CN113259983B

CN113259983B - 5G slice testing method, device, testing system and storage medium

Info

Publication number: CN113259983B
Application number: CN202110690681.9A
Authority: CN
Inventors: 宋丹; 孔露婷; 刘军; 李男
Original assignee: China Mobile Communications Group Co Ltd; China Mobile Communications Ltd Research Institute
Current assignee: China Mobile Communications Group Co Ltd; China Mobile Communications Ltd Research Institute
Priority date: 2021-02-07
Filing date: 2021-06-22
Publication date: 2021-11-19
Anticipated expiration: 2041-06-22
Also published as: CN113259983A; WO2022268117A1; CN112867054A

Abstract

The application discloses a 5G slice testing method, a device, a testing system and a storage medium. Wherein the method comprises the following steps: configuring test parameters associated with the network slice service to a terminal; executing at least two test flows to the terminal in the process of carrying out the network slicing service on the terminal to obtain a test result of a test index of each test flow; the at least two test processes are used for testing the test indexes of the terminal in a multi-network slicing service concurrent scene or testing the test indexes of the terminal in a single-network slicing service scene; and generating a test result aiming at the terminal by using the test result of the test index of each test flow. The method and the device can realize multi-dimension test for network slices, can meet the test requirements of different application scenes on performance indexes, improve the test efficiency, and improve the reliability of the test result in reflecting the authenticity performance of the terminal.

Description

5G slice testing method, device, testing system and storage medium

Technical Field

The present disclosure relates to the field of device testing technologies, and in particular, to a 5G slice testing method, device, testing system, and storage medium.

Background

In order to meet the increasing traffic demand, in the fifth generation communication mobile technology (5G), a network slicing (also referred to as slicing) technology is proposed. The Network slicing technology can provide differentiated services for different user groups, and set different Quality of Service (QoS) and Data Network Names (DNN) per user in a slice; the slice supporting function is a characteristic function for distinguishing 5G from other existing communication systems.

In the related art, no technical scheme for testing the network slice exists.

Disclosure of Invention

In order to solve the related technical problems, embodiments of the present application provide a 5G slice testing method, apparatus, testing system, and storage medium.

The technical scheme of the embodiment of the application is realized as follows:

the embodiment of the application provides a 5G slice testing method, which comprises the following steps:

configuring test parameters associated with the network slice service to a terminal;

executing at least two test flows to the terminal in the process of carrying out the network slicing service on the terminal to obtain a test result of a test index of each test flow; the at least two test processes are used for testing the test indexes of the terminal in a multi-network slicing service concurrent scene or testing the test indexes of the terminal in a single-network slicing service scene;

and generating a test result aiming at the terminal by using the test result of the test index of each test flow.

In the scheme, the indexes of each test flow are different, and the service of each test flow is the same;

or the indexes of each test flow are the same, and the service of each test flow is different;

or, the indexes of each test flow are different, and the service of each test flow is different.

In the scheme, under the scene of concurrent multi-network slicing services, the number of the services is N, and the number of the network slices is M; m is less than or equal to N; m and N are integers.

In the above scheme, at least two services use the same network slice or each service uses one network slice;

a test flow is performed for each service or on a protocol data unit Session (PDU Session) for each network slice.

In the above scheme, the method further comprises: configuring at least two PDU sessions to the terminal;

at least two services are executed in parallel on at least two PDU sessions.

In the above scheme, the method further comprises: configuring one PDU Session on one network slice to the terminal, wherein the PDU Session corresponds to at least two services;

and executing the at least two services in parallel on the PDU Session.

In the above scheme, under the condition that the test indexes of each test flow are different, the test indexes of each test flow are correlated with each other.

In the foregoing solution, the test parameter associated with the network slice service includes at least one of the following:

parameters for characterizing a network slice;

parameters for characterizing service attributes;

a parameter of a terminal routing Policy (URSP, UE Route Selection Policy).

In the foregoing solution, the parameter for characterizing the network slice includes at least one of:

network slice identification;

QoS；

and/or the presence of a gas in the gas,

the parameters for characterizing the service attributes comprise at least one of the following:

an application identifier of the service;

fully Qualified Domain Name (FQDN) information for a service;

internet Protocol (IP) triple information of the service;

DNN information of the service;

connection Capability (CC) information of the service.

In the above scheme, the two test flows include a first test flow and a second test flow; the first test flow and the second test flow comprise one or a combination of the following test indexes:

downlink throughput;

time delay;

an uplink throughput;

power consumption.

In the scheme, a virtual network environment is constructed; the network slicing service performed on the terminal is a service of a virtual network environment.

In the scheme, the terminal is configured with the value of the primary test parameter; in the process of carrying out the network slicing service on the terminal, sequentially executing the at least two test flows on the terminal;

generating a test result aiming at the terminal based on the obtained test result of the test index of the test flow and the corresponding test judgment condition; the test judgment condition is used for judging the success or failure of the corresponding test process.

In the above scheme, when the test result obtained by the first test flow in the at least two test flows does not meet the test judgment condition, the test is ended; the first test flow is the other test flows except the last test flow in at least two test flows executed in sequence.

In the scheme, the terminal is configured with the value of the primary test parameter; in the process of carrying out network slicing service on the terminal, executing the at least two test flows to the terminal in parallel;

generating a test result for the terminal based on the test result of the test index of each test flow and the corresponding test judgment condition; the test judgment condition is used for judging the success or failure of the corresponding test process.

In the scheme, Q times of test parameters are configured for the terminal; q is an integer greater than or equal to 2; when the terminal carries out the network slicing service, the terminal is sequentially executed with the at least two test flows every time the test parameters are configured;

generating a test result aiming at the terminal based on the test result of the test index of the test flow obtained each time and the corresponding test judgment condition; the test judgment condition is used for judging the success or failure of the corresponding test process.

In the above scheme, in the process of sequentially executing the at least two test flows to the terminal each time, when the test value of the test index obtained by the first test flow in the at least two test flows does not satisfy the test judgment condition, the test is ended; the first test flow is the other test flows except the last test flow in at least two test flows executed in sequence.

In the scheme, Q times of test parameters are configured for the terminal; q is an integer greater than or equal to 2; when the terminal carries out the network slicing service, the terminal is executed with the at least two test flows in parallel every time the test parameters are configured;

The embodiment of the present application further provides a 5G slice testing device, including:

the configuration unit is used for configuring test parameters related to the network slice service to the terminal;

the terminal comprises a testing unit and a processing unit, wherein the testing unit is used for executing at least two testing processes on the terminal in the process of carrying out network slicing service on the terminal to obtain a testing result of a testing index of each testing process; the at least two test processes are used for testing the test indexes of the terminal in a multi-network slicing service concurrent scene or testing the test indexes of the terminal in a single-network slicing service scene; and generating a test result aiming at the terminal by using the test result of the test index of each test flow.

The embodiment of the present application further provides a 5G slice test system, including: a processor and a communication interface; wherein the content of the first and second substances,

the processor is configured to:

configuring test parameters associated with the network slice service to the terminal through the communication interface; executing at least two test flows to the terminal in the process of carrying out the network slicing service on the terminal to obtain a test result of a test index of each test flow; the at least two test processes are used for testing the test indexes of the terminal in a multi-network slicing service concurrent scene or testing the test indexes of the terminal in a single-network slicing service scene; and generating a test result aiming at the terminal by using the test result of the test index of each test flow.

The embodiment of the present application further provides a 5G slice test system, including: a processor and a memory for storing a computer program capable of running on the processor,

wherein the processor is configured to perform the steps of any of the above methods when running the computer program.

Embodiments of the present application also provide a storage medium having a computer program stored thereon, where the computer program is executed by a processor to implement the steps of any one of the above methods.

The test method, the test device, the test system and the storage medium provided by the embodiment of the application configure the test parameters associated with the network slice service to the terminal; executing at least two test flows to the terminal in the process of carrying out the network slicing service on the terminal to obtain a test result of a test index of each test flow; the at least two test processes are used for testing the test indexes of the terminal in a multi-network slicing service concurrent scene or testing the test indexes of the terminal in a single-network slicing service scene; according to the scheme of the embodiment of the application, the test is executed through at least two test flows used for the network slice, and the test result of the network slice service is generated through the test result of the test index of each test flow, so that the multi-dimensional test is realized, the test requirements of different application scenes on performance indexes can be met, the test efficiency is improved, and the reliability of the test result in reflecting the terminal authenticity performance can be improved.

Drawings

FIG. 1 is a schematic flow chart of the method for slice testing in example 5G of the present application;

FIG. 2 is a schematic diagram of a multi-dimensional 5G slice test procedure according to an embodiment of the present application;

FIG. 3 is a schematic diagram of another multi-dimensional 5G slice test procedure according to an embodiment of the present application;

FIG. 4 is a schematic structural diagram of a 5G slice test apparatus according to an embodiment of the present application;

fig. 5 is a schematic structural diagram of a 5G slice test system according to this application.

Detailed Description

The present application will be described in further detail with reference to the following drawings and examples.

When testing the network slice service, a single input-single output test mode can be adopted, namely, one path of input measurement quantity is input, one path of input measurement quantity is obtained through the one path of input measurement quantity, and whether the test is passed or not can be judged through the one path of output measurement quantity. A multi-input-single-output test method can also be adopted, namely, multi-path measurement input quantity is input, one path of test output quantity is obtained through the multi-path measurement input quantity, and whether the test passes or not can be judged through the path of test output quantity. That is, a single dimension is used to describe the test results. However, it is often not sufficient for a network slice to describe test results by a single dimension. Therefore, a multi-dimensional test mode needs to be introduced.

Based on this, in various embodiments of the present application, a test is performed through at least two test flows for a network slice service, and a test result is generated using a test result of a performance index of each test flow.

The embodiment of the application provides a 5G slice testing method, which is applied to a testing system, and as shown in fig. 1, the method includes:

step 101: configuring test parameters associated with the network slice service to a terminal;

step 102: executing at least two test flows to the terminal in the process of carrying out the network slicing service on the terminal to obtain a test result of a test index of each test flow; the at least two test processes are used for testing the test indexes of the terminal in a multi-network slicing service concurrent scene or testing the test indexes of the terminal in a single-network slicing service scene;

step 103: and generating a test result aiming at the terminal by using the test result of the test index of each test flow.

In actual application, the test object is a terminal, and thus the terminal may also be referred to as a test terminal or a verification terminal. Because the terminal is subjected to the service performance test of the test network slice, the target performance index can be determined based on the service performance requirement; then, determining target test parameters according to the target performance indexes, and configuring test parameters related to the network slice service to the terminal based on the target test parameters; accordingly, at least two test flows for target performance index measurement based on the target test parameters are generated.

Here, the network slice service refers to a service using a network slice. Specifically, the network slice may be 5G network slice.

In practical application, under a scenario of concurrent multi-network slice services, two factors, namely network slices and services, need to be considered, and therefore, the number of network slices and services may have the following relationship:

the number of the services is N, and the number of the network slices is M; m is less than or equal to N; m and N are integers.

That is, one network slice may be used for a plurality of services, or one network slice may be used for one service, so that in actual application, the following situations may occur:

in the first case, at least two services use the same network slice; executing a test flow for each of the at least two services; illustratively, the terminal has a service A and a service B, both the service A and the service B use the slice 1, a test flow A is executed for the service A, and a test flow B is executed for the service B;

in the second case, at least two services use the same network slice; executing a test flow on the PDU Session of each network slice; the terminal has services A, B and C; wherein, the service A and the service B both use the slice 1, and the service C uses the slice 2; executing a test flow A on the PDU Session corresponding to the slice 1 for the service A, and executing a test flow B on the PDU Session corresponding to the slice 2 for the service C;

in the third case, each service uses one network slice (i.e. each service uses a different network slice), and a test procedure is executed on the PDU Session of each network slice; illustratively, the terminal has a service a and a service B; wherein, the service A uses the slice 1, and the service B uses the slice 2; and executing a test flow A on the PDU Session corresponding to the slice 1 for the service A, and executing a test flow B on the PDU Session corresponding to the slice 2 for the service B.

In the fourth case, at least two services use the same PDU Session on the same network slice, and a test procedure is performed on each service of the PDU Session, that is, multiple test procedures are performed in parallel on the PDU Session. Illustratively, the terminal has a service a and a service B; wherein, the service A and the service B both use PDU Session1 of slice 1; and executing a test flow A on the PDU Session1 of the slice 1 for the service A, and executing a test flow B on the PDU Session1 of the slice 1 for the service B.

As can be seen from the above description, in a scenario where multiple network slices of services are concurrent, at least two services use the same network slice or each service uses one network slice; and a test flow is performed for each service or on a PDU Session of each network slice.

When a test flow is executed on a PDU Session of each network slice, at least two PDU sessions need to be configured to the terminal when configuring test parameters associated with a network slice service to the terminal, so as to implement concurrence of multiple network slice services, that is, at least two services are executed in parallel on at least two PDU sessions.

Wherein, under the condition that the one PDU Session corresponds to at least two services, when configuring a test parameter associated with a network slice service to the terminal, it is required to configure one PDU Session on one network slice to the terminal, where the one PDU Session corresponds to at least two services; and, the at least two services are executed in parallel on the one PDU Session.

Specifically, for each PDU Session, a PDU Session connection establishment request sent by a terminal needs to be received; the request carries corresponding parameters used for representing service attributes and a network slice identifier; and sending a PDU Session establishment completion message to the terminal; the message carries corresponding parameters for representing the network slice and parameters for representing the service attributes.

Exemplarily, assuming that two PDU sessions are configured, for the first PDU Session, the terminal initiates a PDU Session connection establishment request; requesting to carry parameters and network slice identifiers which are corresponding to the first PDU Session and used for representing service attributes; the test system sends a PDU Session establishment completion message to the terminal; the message carries a parameter for representing a network slice and a parameter for representing a service attribute corresponding to the first PDU Session; aiming at the second PDU Session, the terminal also initiates a PDU Session connection establishment request; requesting to carry parameters and network slice identifiers which are corresponding to the second PDU Session and used for representing service attributes; the test system sends a PDU Session establishment completion message to the terminal; the message carries the parameters for representing the network slice and the parameters for representing the service attributes corresponding to the second PDU Session.

Correspondingly, under the condition that the at least two test flows are used for testing the test indexes of the terminal in the single network slice service scene, the at least two test flows are executed on the PDU Session of the first network slice service; illustratively, the terminal has a service a; and the service A uses the slice 1, and executes a test flow A and a test flow B on the PDU Session corresponding to the slice 1.

In actual application, the test indexes of each test flow are different and/or the service of each test flow is different; specifically, indexes of each test flow are different, and the service of each test flow is the same; or, the indexes of each test flow are the same, and the service of each test flow is different; the indexes of each test flow are different, and the service of each test flow is different.

Each test flow can test one test index (also called performance index); under the condition that the test indexes of each test flow are different, the test indexes of each test flow can be correlated with each other.

Here, the association may include at least one of: positive correlation; negative correlation; the test index measured by one test flow and the test index measured by the other test flow are mutual results. The positive correlation means that the test index measured by one test procedure increases with the increase of the test index measured by another test procedure, or decreases with the decrease of the test index measured by another test procedure. Negative correlation means that the test index measured by one test procedure decreases with an increase in the test index measured by another test procedure, or increases with a decrease in the performance index measured by another test procedure. The mutual result means: the test result (e.g., the measured test index) of one test procedure may be used as the test parameter configuration or test condition of another test procedure, and the test result of another test procedure may also be used as the test parameter configuration or test condition of the one test procedure.

In an embodiment, the test parameters associated with the network slice service may include at least one of:

parameters for characterizing a network slice;

parameters for characterizing service attributes;

parameters of URSP.

Wherein, in an embodiment, the parameters for characterizing the network slice include at least one of:

network slice identification;

QoS。

here, in practical applications, the Network Slice identifier may include Single Network Slice Selection Assistance Information (S-NSSAI), and may also be referred to as Network Slice Selection Assistance Information (NSSAI).

In an embodiment, the parameter for characterizing the service attribute includes at least one of:

an application identifier of the service;

FQDN information of the service;

IP triple information of the service;

DNN information of the service;

CC information of the service.

In actual application, one service corresponds to one application, so that a corresponding application identifier, namely, an APP ID, exists. FQDN, which may also be referred to as full domain name. The IP triplet may include: destination IP, destination port, protocol type. A CC may also be referred to as a connection type.

In practical application, according to the requirement of the test, the test result of the test index of the test flow may be: a value, which may also be a packet, etc.

And when the test result of the test indexes of the test flows contains the test value, the test indexes of each test flow are correlated.

Specifically, in one embodiment, the two test flows include a first test flow and a second test flow; the first test flow and the second test flow satisfy one of:

the test index of the first test flow comprises throughput; the test index of the second test flow comprises time delay;

the test index of the first test flow comprises downlink throughput; the test index of the second test flow comprises uplink throughput;

the test index of the first test flow comprises throughput; the test index of the second test flow comprises power consumption.

In practical application, the throughput corresponding to the delay and the power consumption may be an uplink throughput, a downlink throughput, or an uplink throughput and a downlink throughput.

As can be seen from the above description, the first test flow and the second test flow include one or a combination of the following test indicators:

downlink throughput;

time delay;

an uplink throughput;

power consumption.

In an embodiment, the at least two test flows may be used to test a test indicator of the terminal under a network parameter configuration in a multi-network slice service concurrence scenario; the at least two test flows can also be used for testing the test indexes of the terminal under different network parameter configurations in a multi-network slice service concurrency scene.

In an embodiment, the at least two test flows may be used to test a test indicator of the terminal under a network parameter configuration in a single network slice service scenario; the at least two test flows can also be used for testing the test indexes of the terminal under different network parameter configurations in a single network slice service scene.

In practice, the test environment may be virtual. Based on this, in one embodiment, a virtual network environment is constructed; the network slicing service performed on the terminal is a service of a virtual network environment.

Here, the service of the virtual network environment refers to: virtual services, which do not exist in reality, such as those simulated by an application simulator in a test system.

In practical application, in step 102, at least one round of test may be performed on the terminal in the process of performing at least two test procedures on the terminal; and in each round of test, configuring test parameters related to the network slice service to the terminal.

Here, at least two test flows may be executed in sequence in each round of test, or at least two test flows may be executed in parallel; in actual application, which mode is adopted can be determined according to needs.

Specifically, in an embodiment, a value of a primary test parameter is configured to the terminal, that is, a primary test parameter is configured; in the process of carrying out the network slicing service on the terminal, sequentially executing the at least two test flows on the terminal;

When the test result obtained by the first test flow in the at least two test flows does not meet the test judgment condition, ending the test; the first test flow is the other test flows except the last test flow in at least two test flows executed in sequence.

In one embodiment, the terminal is configured with the value of a test parameter once; in the process of carrying out network slicing service on the terminal, executing the at least two test flows to the terminal in parallel;

and generating a test result aiming at the terminal based on the test result of the test index of each test flow and the corresponding test judgment condition.

In one embodiment, Q times of test parameters are configured to the terminal; q is an integer greater than or equal to 2; when the terminal carries out the network slicing service, the terminal is sequentially executed with the at least two test flows every time the test parameters are configured;

and generating a test result aiming at the terminal based on the test result of the test index of the test flow obtained each time and the corresponding test judgment condition.

In an embodiment, in the process of sequentially executing the at least two test flows to the terminal each time, when a test value of a test index obtained by a first test flow in the at least two test flows does not meet a test judgment condition, ending the test; the first test flow is the other test flows except the last test flow in at least two test flows executed in sequence.

In one embodiment, Q times of test parameters are configured to the terminal; q is an integer greater than or equal to 2; when the terminal carries out the network slicing service, the terminal is executed with the at least two test flows in parallel every time the test parameters are configured;

Here, in actual application, the test judgment condition is associated with a type of a test result of a test index of a test flow; for example, when the test result of the test index of the test flow is a test value, the test judgment condition may be a test index threshold; and when the test value of the test index of the test flow does not meet the test index threshold, the test of the test flow is considered to be failed, and when the test index threshold is met, the test of the test flow is considered to be successful. That is, according to the test value of the test index of each round of test and the corresponding test index threshold, it is determined whether the test flow in the corresponding round passes. When the test result of the test index of the test flow is a data packet, the test judgment condition may be a set data packet; and when the data packet of the test flow is not matched with the set data packet, the test of the test flow is considered to be failed, and when the test packet of the test flow is matched with the set data packet, the test of the test flow is considered to be successful.

In each round of test, the values of the test parameters associated with the network slice service configured to the terminal are different.

In practical applications, the values of the test parameters of the current round (which may also be referred to as the current round) may be set according to the test results (the values of the test parameters of each round and the corresponding pass or fail test results) of the P-round tests that have been performed before. Wherein P is an integer greater than or equal to 1.

In each round of test, the set test judgment conditions can be the same and can be different.

When the test of the current round is finished, if all the test flows pass the test, judging that the test of the current round passes;

and when the test of the current round is finished, if the test flow which fails the test exists, judging that the test of the current round fails.

In the embodiment of the present application, a test parameter configuration of a network slice service is performed, and therefore, a test result of the terminal may also be referred to as a test result of the network slice service for the terminal.

In practical application, the specific content of the generated test result aiming at the terminal can be determined according to the requirement; for example, the test result of each test flow and the information of the test success or failure of the corresponding test flow in each round of test can be included; or only contain the information of success or failure of each round of test; the test result of each test flow and the information of the test success or failure of the corresponding test flow can be contained, and the information of the test success or failure of each round can also be contained.

The testing method provided by the embodiment of the application configures testing parameters associated with the network slice service to the terminal; executing at least two test flows to the terminal in the process of carrying out the network slicing service on the terminal to obtain a test result of a test index of each test flow; the at least two test processes are used for testing the test indexes of the terminal in a multi-network slicing service concurrent scene or testing the test indexes of the terminal in a single-network slicing service scene; according to the scheme of the embodiment of the application, the test is executed through at least two test flows used for network slicing, and the test result is generated through the test result of the test index of each test flow, so that the multi-dimensional test is realized, the test requirements of different application scenes on performance indexes can be met, the test efficiency is improved, and the reliability of the test result in reflecting the terminal authenticity performance can be improved.

The present application will be described in further detail with reference to the following application examples.

In an application embodiment, the terminal is in a new air interface (NR) cell. The test system is referred to as a meter and may also be referred to as a test platform.

For the test, NR cell configuration may be performed, and when performing configuration, parameters of the current network may be referred to, which specifically includes:

(1) default parameter configuration

Specific parameters can be found in table 1.

TABLE 1

(2) Network slice configuration, i.e. service configuration

Specific parameters can be found in table 2.

TABLE 2

It is also necessary to configure parameters of a Universal Subscriber Identity Module (USIM), which is generally called a USIM card, and specific parameters can be referred to in table 3.

TABLE 3

During the test, the terminal needs to complete the registration procedure of the NR cell, and specifically, the procedure shown in table 4 may be adopted.

TABLE 4

In table 4, U denotes a terminal, which may also be referred to as User Equipment (UE), and S denotes a test system (i.e., a test platform). -indicating that the test system sends a message to the terminal; - - > represents the terminal sending a message to the test system.

TABLE 5

TABLE 6

TABLE 7

TABLE 8

TABLE 9

Watch 10

TABLE 11

TABLE 12

Watch 13

TABLE 14

Watch 15

Application embodiment 1

In the present application embodiment:

(1) the test purpose is as follows: and verifying whether the service performance of the terminal meets the requirements under the eMBB and uRLLC multi-slice concurrent scene.

(2) The application range is as follows: the method is suitable for the terminal with slicing capability.

(3) And (3) testing conditions are as follows:

[ test System configuration ]

The cell A is an NR cell;

Cell Id=01 TAC = 01；

MCC = 460 MNC = 00；

test band = n 41;

test frequency = f 1;

UL/DL MCS dynamic configuration;

P-MAX=10dBm；

an ideal channel environment;

slice 1 corresponds to 5G quality of service indicator (5 QI) 9;

slice 2 corresponds to 5QI 80.

[ terminal configuration ]

And the terminal is powered off, the USIM card is not inserted into the terminal, and the terminal is not preconfigured with any slice identifier.

[ service configuration ]

Service A: downlink Transmission Control Protocol (TCP) traffic transmission;

and B, service B: PING packet, packet size is 32 bytes.

The test flow comprises the following steps:

step 1: the NR cell remains off;

step 2: the terminal inserts USIM card and starts up;

and step 3: activating an NR cell A;

and 4, step 4: the terminal initiates random access on the cell A, and after the random access is completed, the terminal sends RRC establishment completion (RRCSetupcomplete) and a REGISTRATION REQUEST (REGISTRATION REQUEST);

and 5: executing the steps 5-12 in the table 4 to complete the authentication and the security completion process;

step 6: a meter down REGISTRATION ACCEPT (REGISTRATION ACCEPT) message, configuring different types of authorized network slice selection assistance information (Allowed S-NSSAI) and configured NSSAI (configured NSSAI), see table 5; wherein, table 5 shows the content of the REGISTRATION ACCEPT message;

and 7: the terminal replies a REGISTRATION COMPLETE message;

and 8: the method comprises the steps that a meter sends a downlink non-access stratum (DL NAS TRANSPORT) message to configure routing Policy (URSP) related parameters to a terminal, wherein a load container type (Payload container type) is set to be a 'terminal Policy container' (UE Policy container) ('0101' B), and a management UE Policy COMMAND (MANAGE UE Policy COMMAND) message is carried. The "terminal POLICY part type" in the MANAGE UE POLICY COMMAND message is set to "URSP" ('0001' B), see table 6 and table 7. Wherein, the "UE policy part contents" message carries the URSP configuration information, and maps DNN _1 to slice 1 and DNN _2 to slice 2, specifically refer to tables 8 to 10; wherein, table 6 shows the content of the DL NAS TRANSPORT message; table 7 shows the contents of the message MANAGE UE POLICY COMMAND message of the information element Payload container in table 6; table 8 shows the message contents of the information element UE policy part contents in table 7; table 9 shows the signaling content of an information element Route selection descriptor list in table 8; table 10 shows the signaling content of another information element Route selection descriptor list in table 8;

and step 9: the terminal replies an uplink NAS TRANSPORT (UL NAS TRANSPORT) message carrying a management UE POLICY completion (MANAGE UE POLICY COMPLETE) message;

step 10: a user initiates a service A, and a terminal initiates a PDU Session connection establishment request; specifically, the terminal sends a UL NAS TRANSPORT message and carries a PDU Session Establishment Request (PDU Session Establishment Request), where the PDU Session Establishment Request carries DNN = DNN _1 and a correct single network slice selection assistance information (S-NSSAI) identifier, see table 11 and table 12; wherein, table 11 shows the content of the UL NAS TRANSPORT message; table 12 shows the message contents of the value PDU SESSION ESTABLISHMENT REQUEST of the information element Payload container in table 11.

Step 11: after receiving the request, the meter replies PDU Session Establishment completion (PDU Session Establishment Accept) to perform test parameter configuration such as DNN, S-NSSAI, quality of service class identifier (QCI) and the like to the terminal, and the test parameter configuration refers to a table 13; wherein, table 13 shows the content of PDU Session Establishment Accept message;

step 12: a user initiates a service B, and a terminal initiates a PDU Session connection establishment request; specifically, the terminal sends a UL NAS TRANSPORT message and carries a PDU Session Establishment Request, where the Session Establishment Request carries DNN = DNN _2 and a correct S-NSSAI identifier, see table 11 and table 12;

step 13: the instrument performs test parameter configuration such as DNN, S-NSSAI, QCI and the like to the terminal by replying PDU Session update Accept, see Table 13;

step 14: keeping the terminal executing the service A and the service B in parallel; the terminal executes a throughput testing process of the service A on the first PDU Session to obtain a testing value of the throughput performance index of the service A executed by the terminal;

step 15: setting a throughput performance index threshold according to table 14, and judging whether the test flow of the service a passes or not according to the service a throughput performance test value obtained in step 14 and the throughput performance index threshold set in table 14; if the test is not passed, ending the test of the current round, namely ending the test, and not executing the step 16; specifically, when the throughput performance test value is greater than or equal to the throughput performance index threshold, that is, the threshold is met, the test flow is considered to pass; when the throughput performance test value is smaller than the throughput performance index threshold value, namely the throughput performance index threshold value is not met, the test process is not considered to pass;

step 16: keeping the terminal to run parallel with the service A and the service B, and executing the time delay test flow of the service B on the second PDU Session by the terminal to obtain a test value of the time delay performance index of the service B executed by the terminal;

and step 17: setting a delay performance index threshold according to table 14, and judging whether the test flow of the service B passes or not according to the delay performance test value of the service B obtained in the step 16 and the delay performance index threshold set in table 14; specifically, when the ductility performance test value is less than or equal to the delay performance index threshold, namely the threshold is met, the test flow is considered to pass; when the time delay performance test value is larger than the time delay performance index threshold value, namely the time delay performance index threshold value is not met, the test process is not passed;

step 18: if the step 15 and the step 17 both pass the test, determining that the test passes, ending the test flow of the service A and the service B, and generating a test result of the terminal; if the test flow which fails the test exists in the steps 15 and 17, judging that the test fails, ending the test flows of the service A and the service B, and generating a test result of the terminal; the test result of the terminal may include a result that the test of the throughput test flow of the service a passes or fails, and a result that the test of the delay test flow of the service B passes or fails;

step 19: releasing the RRC link, and powering off the terminal;

step 20: NR cell a is deactivated.

In practical application, the delay performance index threshold may be set according to table 14, and then the set delay performance index threshold is directly used in steps 15 and 17.

In the above flow, the throughput test flow of the service a and the delay test flow of the service B are executed in sequence. The throughput test flow of the service a and the delay test flow of the service B may also be executed in parallel, that is, steps 14 and 16 may also be executed simultaneously, that is, the terminal executes the throughput test flow of the service a on the first PDU Session and executes the delay test flow of the service B on the second PDU Session in parallel.

As can be seen from the above description, steps 1 and 2 are leading steps; step 3 to step 18 are main steps; step 19 and step 20 are end steps.

Application example two

In the present application embodiment:

(1) the test purpose is as follows: and testing the service performance under different network parameter configurations in a multi-slice concurrent scene.

(3) And (3) testing conditions are as follows:

[ test System configuration ]

The cell A is an NR cell;

Cell Id=01 TAC = 01；

MCC = 460 MNC = 00；

test band = n 41;

test frequency = f 1;

UL/DL MCS dynamic configuration

P-MAX=10dBm；

An ideal channel environment.

[ terminal configuration ]

[ service configuration ]

Service A: downlink TCP service transmission;

and B, service B: PING packet, packet size 32 bytes.

The test flow comprises the following steps:

step 1: the NR cell remains off;

step 2: the terminal inserts USIM card and starts up;

and step 3: activating an NR cell A;

and 4, step 4: the terminal initiates random access on the cell A, and after the random access is finished, the terminal sends RRCSetupcomplete and REGISTRATION REQUEST;

step 6: the meter issues a REGISTRATION ACCEPT message, and configures different types of Allowed S-NSSAI and Configured NSSAI, see Table 5; wherein, table 5 shows the content of the REGISTRATION ACCEPT message;

and 7: the terminal replies a REGISTRATION COMPLETE message;

and 8: the instrument sends a downlink non-access stratum (DL NAS TRANSPORT) message to configure URSP related parameters to the terminal, wherein Payload container type is set as 'UE Policy container' ('0101' B) and carries MANAGE UE Policy COMMAND message. The "UE POLICY part type" in the MANAGE UE POLICY COMMAND message is set to "URSP" ('0001' B), see tables 6 and 7. Wherein the "UE policy part contents" message carries URSP configuration information, DNN _1 is mapped to slice 1, and DNN _2 is mapped to slice 2, specifically see tables 8 to 10; wherein, table 8 shows the message content of the information element UE policy part contents in table 7; table 9 shows the signaling content of an information element Route selection descriptor list in table 8; table 10 shows the signaling content of another information element Route selection descriptor list in table 8;

and step 9: the terminal replies UL NAS TRANSPORT message carrying MANAGE UE POLICY COMPLETE message;

step 10: a user initiates a service A, and a terminal initiates a PDU Session connection establishment request; specifically, the terminal sends a UL NAS TRANSPORT message and carries a PDU Session Establishment Request, and carries DNN = DNN _1 and a correct S-NSSAI identifier in the DU Session Establishment Request, see tables 11 and 12; wherein, table 11 shows the content of the UL NAS TRANSPORT message; table 12 shows the message contents of the value PDU SESSION ESTABLISHMENT REQUEST of the information element Payload container in table 11.

Step 11: after the instrument receives the request, test parameter configuration such as DNN, S-NSSAI, QCI and the like is carried out on the terminal by replying PDU Session Establishment Accept, referring to Table 13; wherein, table 13 shows the content of PDU Session Establishment Accept message;

step 14: ensuring that the service A and the service B are executed in parallel;

step 15: the terminal executes a throughput testing process of the service A on the first PDU Session to obtain a testing value of the throughput performance index of the service A;

step 16: and setting a throughput performance index threshold according to the table 14, and judging whether the current test flow passes or not according to the service A throughput performance test value obtained in the step 15 and the throughput performance index threshold set in the table 14. If so, perform step 17; if not, judging that the test of the current round is not passed, ending the test of the current round, and then executing the step 19; specifically, when the throughput performance test value is greater than or equal to the throughput performance index threshold, the test flow is considered to pass; when the throughput performance test value is smaller than the throughput performance index threshold value, namely the throughput performance index threshold value is not met, the test process is not considered to pass;

and step 17: when the throughput performance index of the service A executed by the terminal meets the threshold value, under the condition that the service A and the service B are ensured to be executed in parallel, the terminal executes the time delay test flow of the service B on the second PDU Session to obtain the test value of the time delay performance index of the service B, and then step 18 is executed;

step 18: and setting a delay performance index threshold according to table 14, and judging whether the current test flow passes or not according to the PING packet delay performance index test value of the service B obtained in the step 17 and the delay performance index threshold set in table 14. If the current round passes the test, judging that the current round passes the test, ending the test of the current round, and executing the step 19; if not, judging that the test of the current round is not passed, ending the test of the current round, and executing the step 19;

step 19: the instrument sends a PDU SESSION MODIFICATION COMMAND (PDU SESSION MODIFICATION COMMAND) message to configure the QoS parameters of service a to the terminal, see table 15;

step 20: the terminal replies PDU SESSION MODIFICATION completion (PDU SESSION MODIFICATION COMPLETE) to confirm the parameter MODIFICATION, and then executes step 21;

step 21: under the condition of ensuring that the service A and the service B are executed in parallel, the terminal executes a throughput testing process of the service A on the first PDU Session to obtain a testing value of a throughput performance index of the service A;

step 22: setting a throughput performance index threshold according to table 14, judging whether the current test flow passes according to the service a throughput performance test value obtained in step 21 and the throughput performance index threshold set in table 14, if so, executing step 23; if not, judging that the test of the current round is not passed, and then executing the step 25;

step 23: the terminal executes the throughput performance index of the service A to meet the threshold, and under the condition that the service A and the service B are ensured to be executed in parallel, the terminal executes the time delay test flow of the service B on the second PDU Session in parallel to obtain the test value of the time delay performance index of the service B;

step 24: and setting a delay performance index threshold according to the table 14, and judging whether the current test flow passes or not according to the service B PING packet delay performance index test value obtained in the step 23 and the delay performance index threshold set in the table 14. If the current round passes, judging that the current round passes the test, ending the test of the current round, and executing the step 25; if not, judging that the test of the current round is not passed, ending the test of the current round, and executing the step 25;

step 25: the instrument sends a PDU SESSION MODIFICATION COMMAND message to modify the QoS configuration parameters of the service B, see Table 15;

step 26: the terminal replies PDU Session MODIFICATION COMPLETE to confirm the parameter MODIFICATION, and then executes step 27;

step 27: under the condition of ensuring that the service A and the service B are executed in parallel, the terminal executes the throughput test flow of the service A on the first PDU Session to obtain the test value of the throughput performance index of the service A

Step 28: and setting a throughput performance index threshold according to table 14, and judging whether the current test flow passes or not according to the service a throughput performance test value obtained in step 27 and the throughput performance index threshold set in table 14. If so, perform step 29; if not, judging that the test of the current round is not passed, and ending the test;

step 29: when the throughput performance index of the service A executed by the terminal meets the threshold value, under the condition that the service A and the service B are ensured to be executed in parallel, the terminal executes the time delay test flow of the service B on the second PDU Session in parallel to obtain the test value of the time delay performance index of the service B;

step 30: and setting a delay performance index threshold according to table 14, and judging whether the current test flow passes or not according to the test value of the delay performance index of the service B PING packet obtained in step 29 and the delay performance index threshold set in table 14. If the test result passes, judging that the test in the current round passes, and ending the test; if not, judging that the test of the current round is not passed, and ending the test;

step 31: ending the test flow of the service A and the service B, and generating a test result of the terminal; the test result comprises the QoS parameter (namely 5 QI) of each round of test and the result that the test of each test flow passes or fails;

step 32: releasing the RRC link, and powering off the terminal;

step 33: NR cell a is deactivated.

As can be seen from the above description, in the present application embodiment, whether the service performance can meet the requirement under different network parameter configurations (i.e. different 5QI configurations) is tested, and the test is passed in step 18, step 24 and step 30.

In the above flow, the throughput test flow of the service a and the delay test flow of the service B are executed in sequence. The throughput test flow of the service a and the delay test flow of the service B may also be executed in parallel, that is, steps 15 and 17 are executed simultaneously, that is, the terminal executes the throughput test flow of the service a on the first PDU Session and executes the delay test flow of the service B on the second PDU Session in parallel; likewise, steps 21 and 23 are performed simultaneously, and steps 27 and 29 are performed simultaneously.

In practical application, the delay performance index threshold may be set according to table 14, and then the set delay performance index threshold is directly used in steps 16, 18, 22, 24, 28 and 30.

Step 1, step 2 is a leading step; step 3 to step 31 are main steps; step 32 and step 33 are end steps.

Application example three

In the present application embodiment:

(1) the test purpose is as follows: and verifying whether the service performance of the terminal meets the requirements under the eMBB single-network slicing scene.

(3) And (3) testing conditions are as follows:

[ test System configuration ]

The cell A is an NR cell;

Cell Id=01 TAC = 01；

MCC = 460 MNC = 00；

test band = n 41;

test frequency = f 1;

UL/DL MCS dynamic configuration;

P-MAX=10dBm；

an ideal channel environment;

slice 1 corresponds to 5QI 9.

[ terminal configuration ]

The terminal is powered off, no USIM card is inserted into the terminal, and no slice identifier is preconfigured in the terminal.

[ service configuration ]

Service A: and (4) downlink TCP traffic transmission.

TABLE 16

TABLE 17

Watch 18

Watch 19

Watch 20

TABLE 21

The test flow comprises the following steps:

step 1: the NR cell remains off;

step 2: the terminal inserts USIM card and starts up;

and step 3: activating an NR cell A;

and 5: executing the steps 5-12 in the table 4 to complete the process of the authentication;

step 6: the meter issues a REGISTRATION ACCEPT message, and configures different types of Allowed NSSAIs and Configured NSSAIs, see table 16; wherein, table 16 shows the content of the REGISTRATION ACCEPT message;

and 7: the terminal replies a REGISTRATION COMPLETE message;

and 8: the instrument sends a DL NAS TRANSPORT message to configure URSP related parameters to the terminal, wherein Payload container type is set as 'UE Policy container' ('0101' B), carrying MANAGE UE Policy COMMAND message. The "UE POLICY part type" in the MANAGE UE POLICY COMMAND message is set to "URSP" ('0001' B), see tables 6 and 7. Wherein, the "UE policy part contents" carries the URSP configuration information, and maps DNN _1 to slice 1, see table 17; wherein, table 17 shows the message contents of the information element UE policy part contents in table 7;

and step 9: the verification terminal replies UL NAS TRANSPORT and carries MANAGE UE POLICY COMPLETE message;

step 10: a user initiates a service A, and a terminal initiates a PDU Session connection establishment request; specifically, the terminal sends a UL NAS TRANSPORT message and carries a PDU Session Establishment Request, where the PDU Session Establishment Request carries DNN = DNN _1 and a correct S-NSSAI identifier, see tables 18 and 19; wherein, table 18 shows the contents of the UL NAS TRANSPORT message; table 19 shows the message contents of the value PDU SESSION ESTABLISHMENT REQUEST of the information element Payload container in table 18;

step 11: after the instrument receives the request, test parameter configuration such as DNN, S-NSSAI, QCI and the like is carried out on the terminal by replying PDU Session Establishment Accept, and the operation A is ensured to be normally carried out by referring to the table 20; wherein, table 20 shows the content of PDU Session Establishment Accept message;

step 12: executing the throughput testing process of the service A to obtain a testing value of the throughput performance index of the service A executed by the terminal;

step 13: setting a throughput performance index threshold according to table 21, and judging whether the test flow of the service a passes or not according to the service a throughput performance test value obtained in step 12 and the throughput performance index threshold set in table 21; if the test is not passed, ending the test of the current round, namely ending the test, and not executing the step 14;

step 14: executing the time delay test flow of the service A to obtain a test value of the time delay performance index of the service A executed by the terminal;

step 15: setting a delay performance index threshold according to table 21, and judging whether the delay test flow of the service a passes or not according to the delay performance index test value of the service a obtained in step 14 and the delay performance index threshold set in table 21;

step 16: if the throughput test flow in the step 13 and the delay test flow in the step 15 both pass the test, determining that the round of test passes, ending the test flow of the service A, and generating a test result of the terminal; if the throughput test flow in the step 13 and the time delay test flow in the step 15 have test flows which fail the test, judging that the round of test fails, and generating a test result of the terminal;

and step 17: releasing the RRC link, and powering off the terminal;

step 18: NR cell a is deactivated.

In practical application, the delay performance index threshold may be set according to table 21, and then the set delay performance index threshold is directly used in steps 13 and 15.

In the above flow, the throughput test flow and the delay test flow of the service a are executed in sequence. The throughput testing process and the delay testing process of the service a may also be executed in parallel, that is, the steps 12 and 14 may also be executed simultaneously, that is, the throughput testing process of the service a is executed, and the delay testing process of the service a is executed in parallel.

As can be seen from the above description, steps 1 and 2 are leading steps; step 3 to step 16 are main steps; step 17 and step 18 are end steps.

Application example four

In the present embodiment:

(1) the test purpose is as follows: and verifying whether the throughput performance meets the requirement when the terminal performs bidirectional data transmission under the eMBB single-network slicing scene.

(3) And (3) testing conditions are as follows:

[ test System configuration ]

The cell A is an NR cell;

Cell Id=01 TAC = 01；

MCC = 460 MNC = 00；

test band = n 41;

test frequency = f 1;

UL/DL MCS dynamic configuration

P-MAX=10dBm；

An ideal channel environment;

slice 1 corresponds to 5QI 9.

[ terminal configuration ]

[ service configuration ]

Service A: bidirectional TCP traffic transmission.

TABLE 22

The test flow comprises the following steps:

step 1: the NR cell remains off;

step 2: the terminal inserts USIM card and starts up;

and step 3: activating an NR cell A;

and 7: the terminal replies a REGISTRATION COMPLETE message;

step 10: the instrument and the user initiate service A two-way TCP service transmission, and the terminal initiates a PDU Session connection establishment request; specifically, the terminal sends a UL NAS TRANSPORT message and carries a PDU Session Establishment Request, where the PDU Session Establishment Request carries DNN = DNN _1 and a correct S-NSSAI identifier, see tables 18 and 19; wherein, table 18 shows the contents of the UL NAS TRANSPORT message; table 19 shows the message contents of the value PDU SESSION ESTABLISHMENT REQUEST of the information element Payload container in table 18;

step 11: after the instrument receives the request, test parameter configuration such as DNN, S-NSSAI, QCI and the like is carried out on the terminal by replying PDU Session Establishment Accept so as to ensure that the service A is normally carried out, and the reference is made to a table 20; wherein, table 20 shows the content of PDU Session Establishment Accept message;

step 12: executing a downlink throughput testing process of the service A to obtain a testing value of a downlink throughput performance index of the service A executed by the terminal;

step 13: setting a downlink throughput performance index threshold according to table 22, and judging whether the downlink throughput testing process of the service a passes or not according to the downlink throughput performance testing value of the service a obtained in the step 12 and the downlink throughput performance index threshold set in table 22; if the test is not passed, ending the test of the current round, namely ending the test, and not executing the step 14;

step 14: continuing to perform the downlink throughput test of the service A, and simultaneously executing the uplink throughput test of the service A to obtain an uplink throughput test value of the service A executed by the terminal;

step 15: setting an uplink throughput performance index threshold according to table 22, and judging whether the uplink throughput test flow of the service a passes or not according to the service a uplink throughput test value obtained in step 14 and the uplink throughput index threshold set in table 22;

step 16: if the downlink throughput testing process in the step 13 and the uplink throughput testing process in the step 15 both pass the test, judging that the test passes, ending the testing process of the service A, and generating a testing result of the terminal; if the downlink throughput testing process in the step 13 and the uplink throughput testing process in the step 15 have a testing process which fails the test, judging that the test fails, and generating a testing result of the terminal;

and step 17: releasing the RRC link, and powering off the terminal;

step 18: NR cell a is deactivated.

In practical application, the delay performance index threshold may be set according to the table 22, and then the set delay performance index threshold is directly used in steps 15 and 17.

In the above flow, the uplink throughput testing flow of the service a and the downlink throughput testing flow of the service a are sequentially executed. The throughput testing process of the service a and the downlink throughput testing process of the service a may also be executed in parallel, that is, the steps 12 and 14 are executed simultaneously, that is, the terminal executes the downlink throughput testing process of the service a and executes the uplink throughput testing process of the service a in parallel.

Application example five

In the present application embodiment:

(1) the test purpose is as follows: and verifying whether the service performance of the terminal under different network parameter configurations meets the requirements under the eMBB single network slice scene.

(3) And (3) testing conditions are as follows:

[ test System configuration ]

The cell A is an NR cell;

Cell Id=01 TAC = 01；

MCC = 460 MNC = 00；

test band = n 41;

test frequency = f 1;

UL/DL MCS dynamic configuration

P-MAX=10dBm；

An ideal channel environment;

slice 1 corresponds to 5QI 9.

[ terminal configuration ]

[ service configuration ]

Service A: and (4) downlink TCP traffic transmission.

TABLE 23

The test flow comprises the following steps:

step 1: the NR cell remains off;

step 2: the terminal inserts USIM card and starts up;

and step 3: activating an NR cell A;

and 7: the terminal replies a REGISTRATION COMPLETE message;

and 8: the instrument sends a DL NAS TRANSPORT message to configure URSP related parameters to the terminal, wherein Payload container type is set as 'UE Policy container' ('0101' B), carrying MANAGE UE Policy COMMAND message. The "UE POLICY part type" in the MANAGE UE POLICY COMMAND message is set to "URSP" ('0001' B), see tables 6 and 7. Wherein, the "UE policy part contents" carries the URSP configuration information, and maps DNN _1 to slice 1, see table 17; wherein, table 17 shows the content of the information element UE policy part contents message in table 7;

step 12: executing a throughput testing process of the service A to obtain a testing value of the throughput performance index of the service A;

step 13: and setting a throughput performance index threshold according to the table 21, and judging whether the current test flow passes or not according to the service A throughput performance test value obtained in the step 12 and the throughput performance index threshold set in the table 21. If so, perform step 14; if not, judging that the test of the current round is not passed, ending the test of the current round, and executing the step 16;

step 14: executing the time delay test flow of the service A to obtain the test value of the time delay performance index of the service A

Step 15: and setting a delay performance index threshold according to the table 21, and judging whether the current test flow passes or not according to the service A delay performance index test value obtained in the step 14 and the delay performance index threshold set in the table 21. If the current round passes, judging that the current round passes the test, ending the test of the current round, and executing the step 16; if not, judging that the test of the current round is not passed, ending the test of the current round, and executing the step 16;

step 16: the instrument sends a PDU SESSION MODIFICATION COMMAND message to configure the QoS parameters of the service A to the terminal, see table 23;

and step 17: the terminal replies PDU SESSION MODIFICATION COMPLETE to confirm the parameter MODIFICATION;

step 18: executing a throughput testing process of the service A to obtain a testing value of the throughput performance index of the service A;

step 19: and setting a throughput performance index threshold according to table 21, and judging whether the current test flow passes or not according to the service a throughput performance test value obtained in step 18 and the throughput performance index threshold set in table 21. If so, perform step 20; if not, judging that the test of the current round is not passed, and executing the step 22;

step 20: executing the time delay test flow of the service A to obtain a test value of the time delay performance index of the service A;

step 21: and setting a delay performance index threshold according to the table 21, and judging whether the current test flow passes or not according to the service B PING packet delay performance index test value obtained in the step 20 and the delay performance index threshold set in the table 21. If the current round passes, judging that the current round passes the test, ending the test of the current round, and executing the step 22; if not, judging that the test of the current round is not passed, ending the test of the current round, and executing the step 22;

step 22: the instrument sends a PDU SESSION MODIFICATION COMMAND message to modify the QoS configuration parameters of the service A, see table 23;

step 23: the terminal replies PDU SESSION MODIFICATION COMPLETE to confirm the parameter MODIFICATION;

step 24: executing a throughput testing process of the service A to obtain a testing value of the throughput performance index of the service A;

step 25: and setting a throughput performance index threshold according to the table 21, and judging whether the current test flow passes or not according to the service A throughput performance test value obtained in the step 24 and the throughput performance index threshold set in the table 21. If so, perform step 26; if not, judging that the test of the current round is not passed, and ending the test;

step 26: executing the time delay test flow of the service A to obtain a test value of the time delay performance index of the service A;

step 27: and setting a delay performance index threshold according to table 21, and judging whether the current test flow passes or not according to the service a delay performance index test value obtained in step 26 and the delay performance index threshold set in table 21. If the test result passes, judging that the test in the current round passes, and ending the test; if not, judging that the test of the current round is not passed, and ending the test;

step 28: ending the test flow of the service A and generating a test result of the terminal;

step 29: releasing the RRC link, and powering off the terminal;

step 30: NR cell a is deactivated.

As can be seen from the above description, in the present application embodiment, whether the service performance can meet the requirement under different network parameter configurations (i.e. different 5QI configurations) is tested, and the test is passed in step 15, step 21 and step 27.

In practical application, the delay performance index threshold may be set according to table 21, and then the set delay performance index threshold is directly used in steps 13, 15, 19, 21, 25 and 27.

Step 1, step 2 is a leading step; step 3 to step 28 are main steps; step 29 and step 30 are end steps.

In the above flow, the throughput test flow and the delay test flow of the service a are executed in sequence. The throughput testing process of the service a and the delay testing process of the service a may also be executed in parallel, that is, steps 12 and 14 are executed simultaneously, that is, the throughput testing process of the service a is executed, and the delay testing process of the service a is executed in parallel; likewise, steps 18 and 20 are performed simultaneously, and steps 24 and 26 are performed simultaneously.

Application example six

In the present application embodiment:

(1) the test purpose is as follows: and verifying whether the throughput performance meets the requirement during the bidirectional data transmission of the terminal under different network parameter configurations under the eMBB single network slice scene.

(3) And (3) testing conditions are as follows:

[ test system configuration ];

the cell A is an NR cell;

Cell Id=01 TAC = 01；

MCC = 460 MNC = 00；

test band = n 41;

test frequency = f 1;

UL/DL MCS dynamic configuration;

P-MAX=10dBm；

an ideal channel environment;

slice 1 corresponds to 5QI 9.

[ terminal configuration ]

[ service configuration ]

Service A: bidirectional TCP traffic transmission.

Watch 24

The test flow comprises the following steps:

step 1: the NR cell remains off;

step 2: the terminal inserts USIM card and starts up;

and step 3: activating an NR cell A;

and 7: the terminal replies a REGISTRATION COMPLETE message;

step 12: executing a downlink throughput testing process of the service A to obtain a testing value of a downlink throughput performance index of the service A;

step 13: setting a downlink throughput performance index threshold according to table 22, judging whether the current test flow passes or not according to the downlink throughput performance test value of the service A obtained in the step 12 and the downlink throughput performance index threshold set in the table 22, and if so, executing the step 14; if not, judging that the test of the current round is not passed, ending the test of the current round, and executing the step 16;

step 14: continuing to perform the downlink throughput test of the service A, and simultaneously executing the uplink throughput test of the service A to obtain a test value of the uplink throughput performance index of the service A;

step 15: and setting an uplink throughput performance threshold according to the table 22, and judging whether the current test flow passes or not according to the service A uplink throughput performance test value obtained in the step 14 and the delay performance index threshold set in the table 22. If the current round passes, judging that the current round passes the test, ending the test of the current round, and executing the step 16; if not, judging that the test of the current round is not passed, ending the test of the current round, and executing the step 16;

step 16: the instrument sends a PDU SESSION MODIFICATION COMMAND message to configure the QoS parameters of the service A to the terminal, see table 24;

step 18: executing a downlink throughput testing process of the service A to obtain a testing value of a downlink throughput performance index of the service A;

step 19: setting a downlink throughput performance index threshold according to table 22, judging whether the current test flow passes according to the downlink throughput performance test value of the service a obtained in step 18 and the downlink throughput performance index threshold set in table 22, and if so, executing step 20; if not, judging that the test of the current round is not passed, and executing the step 22;

step 20: continuing to perform the downlink throughput test of the service A, and simultaneously executing the uplink throughput test of the service A to obtain an uplink throughput performance test value of the service A;

step 21: and setting an uplink throughput performance index threshold according to the table 22, and judging whether the current test flow passes or not according to the service A uplink throughput performance test value obtained in the step 20 and the uplink throughput performance index threshold set in the table 22. If the current round passes, judging that the current round passes the test, ending the test of the current round, and executing the step 22; if not, judging that the test of the current round is not passed, ending the test of the current round, and executing the step 22;

step 22: the instrument sends a PDU SESSION MODIFICATION COMMAND message to modify the QoS configuration parameters of the service A, see Table 24;

step 23: the terminal replies PDU Session MODIFICATION COMPLETE to confirm the parameter MODIFICATION

Step 24: executing a downlink throughput testing process of the service A to obtain a testing value of a downlink throughput performance index of the service A;

step 25: and setting a downlink throughput performance index threshold according to the table 22, and judging whether the current test flow passes or not according to the service a downlink throughput performance test value obtained in the step 24 and the downlink throughput performance index threshold set in the table 22. If so, perform step 26; if not, judging that the test of the current round is not passed, and ending the test;

step 26: continuing to perform the downlink throughput test of the service A, and simultaneously executing the uplink throughput test of the service A to obtain a test value of the uplink throughput performance of the service A;

step 27: setting an uplink throughput performance index threshold according to table 22, judging whether the current test flow passes according to the service a uplink throughput performance test value obtained in step 26 and the uplink throughput performance index threshold set in table 22, if so, judging that the test of the current round passes, and ending the test; if not, judging that the test of the current round is not passed, and ending the test;

step 29: releasing the RRC link, and powering off the terminal;

step 30: NR cell a is deactivated.

In practical application, the delay performance index threshold may be set according to the table 22, and then the set delay performance index threshold is directly used in steps 13, 15, 19, 21, 25 and 27.

In the above flow, the uplink throughput testing flow of the service a and the downlink throughput testing flow of the service a are sequentially executed. The throughput testing process of the service a and the downlink throughput testing process of the service a may also be executed in parallel, that is, steps 12 and 14 are executed simultaneously, that is, the downlink throughput testing process of the service a is executed, and the uplink throughput testing of the service a is executed in parallel; likewise, steps 18 and 20 are performed simultaneously; steps 24 and 26 are performed simultaneously.

As can be seen from the above description, in the present application embodiment, whether the performance of the single-network slice service under different network parameter configurations (i.e. different 5QI configurations) can meet the requirement is tested, and the test is passed in step 15, step 21 and step 27. Step 1, step 2 is a leading step; step 3 to step 28 are main steps; step 29 and step 30 are end steps.

Application example seven

In the present application embodiment:

(1) the test purpose is as follows: and verifying whether the terminal supports the DNN associated service and slicing or not and whether the associated sliced service can be normally carried out or not.

(3) And (3) testing conditions are as follows:

[ test System configuration ]

The cell A is an NR cell;

Cell Id=01 TAC = 01；

MCC = 460 MNC = 00；

test band = n 41;

test frequency = f 1;

UL/DL MCS dynamic configuration;

P-MAX=10dBm；

an ideal channel environment.

Service A: generating a data packet with the size of 32 bytes based on the data transmission service generated by the application program simulator, and filling a random number sequence; the corresponding DNN is DNN _ 1;

and B, service B: generating a data packet with the size of 200 bytes based on the data transmission service generated by the application program simulator, and filling a random number sequence; the corresponding DNN is DNN _ 2;

[ terminal configuration ]

TABLE 25

Watch 26

Watch 27

Watch 28

The test flow comprises the following steps:

step 1: the NR cell remains off;

step 2: the terminal inserts USIM card and starts up;

and step 3: activating an NR cell A;

and 4, step 4: the terminal initiates random access on the cell A, and after the random access is finished, the terminal sends RRCSetupcomplete and REGISTRATION REQUEST messages; wherein, the REGISTRATION REQUEST message sent by the terminal does not carry the Requested NSSAI;

step 6: the meter issues a REGISTRATION ACCEPT message, and configures different types of Allowed NSSAIs and Configured NSSAIs, see table 25; wherein, table 25 shows the contents of the REGISTRATION ACCEPT message;

and 7: the terminal replies to the REGISTRATION COMPLETE message, and at this time, the terminal COMPLETEs REGISTRATION and stores the Allowed NSSAI and the Configured NSSAI of the current PLMN;

and 8: the instrument sends a DL NAS TRANSPORT with Payload container type set to "UE Policy container" ('0101' B), carrying MANAGE UE POLICY COMMAND message. The "UE POLICY part type" in the MANAGE UE POLICY COMMAND message is set to "URSP" ('0001' B), "UE POLICY part contents" carries the URSP configuration information, see specifically tables 6 to 10; wherein, table 6 shows the content of the DL NAS TRANSPORT message; table 7 shows the contents of the message MANAGE UE POLICY COMMAND message of the information element Payload container in table 6; table 8 shows the message contents of the information element UE policy part contents in table 7; table 9 shows the signaling content of an information element Route selection descriptor list in table 8; table 10 shows the signaling content of another information element Route selection descriptor list in table 8;

and step 9: the terminal replies UL NAS TRANSPORT and carries MANAGE UE POLICY COMPLETE message;

step 10: a user initiates a service A, and a terminal initiates a PDU Session connection establishment request; specifically, the terminal sends a UL NAS TRANSPORT message and carries a PDU Session Establishment Request, where the PDU Session Establishment Request carries DNN = DNN _1 and a correct S-NSSAI identifier, see tables 26 and 27; wherein, table 26 shows the contents of the UL NAS TRANSPORT message; table 27 shows the message contents of the value PDU SESSION ESTABLISHMENT REQUEST of the information element Payload container in table 26;

step 11: after receiving the request, the meter configures DNN, S-NSSAI, QCI and the like to the terminal by replying PDU Session Establishment Accept, specifically referring to Table 28; wherein, table 28 shows the content of PDU Session Establishment Accept message;

step 12: after the PDU Session is established, the terminal performs a service A through the corresponding PDU Session;

step 13: keeping the normal operation of the service A, initiating a service B by a user, and initiating a PDU Session connection establishment request by a terminal; specifically, the terminal sends a UL NAS TRANSPORT message and carries a PDU Session Establishment Request, where the PDU Session Establishment Request carries DNN = DNN _2 and a correct S-NSSAI identifier, see 26 and table 27;

step 14: after receiving the request, the meter configures DNN, S-NSSAI, QCI and the like to the terminal by replying PDU Session Establishment Accept, specifically referring to Table 28;

step 15: after the PDU Session is established, the terminal carries out the service B through the corresponding PDU Session;

step 16: keeping the service A and the service B simultaneously, and executing a test flow to verify whether the service A is normally carried out on the corresponding PDU Session; specifically, executing a test flow to enable the terminal to transmit a data packet of the service A to the service platform simulator through PDU Session; meanwhile, the terminal directly sends the original simulation data packet of the service A generated by the application program simulator to the service platform simulator; and comparing the service A data packet received through the PDU Session with the service A data packet directly transmitted by the terminal, and comparing the number of bytes of the data packet with the bit-by-bit comparison random number sequence. If the two are consistent, the service A is proved to be mapped to a correct slice, and the following steps are continuously executed; otherwise, the verification of the mapping relation from the service A to the slice fails, the test is judged to fail, and the test is stopped, namely, the step 17 is not executed;

and step 17: keeping the service A and the service B simultaneously, and executing a test flow to verify whether the service B is normally carried out on the corresponding PDU Session; specifically, executing a test flow to enable the terminal to transmit a data packet of the service B to the service platform simulator through the established PDU Session; meanwhile, the terminal directly sends the original simulation data packet of the service B generated by the application program simulator to the service platform simulator; and the service platform simulator compares the service B data packet received through the PDU Session with the service B data packet directly transmitted by the terminal, compares the number of bytes of the data packet and compares the random number sequence bit by bit. If the two are consistent, the service B is proved to be mapped to a correct slice, and the test is judged to be passed; otherwise, the verification of the mapping relation from the service B to the slice fails, and the test failure is judged;

step 18: ending the test flow of the service A and the service B, and generating a test result of the terminal; the test result comprises the result that the test of each test flow passes or fails;

step 19: ending the service A and the service B;

step 20: the instrument issues an RRCRelease message to release an NR link, namely an RRC link;

step 21: NR cell a is deactivated.

In step 16 and step 17, the following steps may be performed in parallel:

after the PDU Session is established, comparing a data packet generated by the terminal service A (an original simulation data packet directly sent to the service platform simulator) with a data packet received by the service platform simulator through the corresponding PDU Session, and verifying whether the service A can be normally performed on the corresponding PDU Session (the first PDU Session); meanwhile, the data packet generated by the terminal service B and the data packet received by the service platform simulator through the corresponding PDU Session are compared to verify whether the service B can be normally performed on the corresponding PDU Session (second PDU Session).

As can be seen from the above description, in this application embodiment, it is tested whether the terminal can associate the correct slice through the DNN information issued by the service and initiate a PDU connection establishment request, and the service can be normally performed after associating the slice. In addition, step 1 and step 2 are leading steps; step 3 to step 18 are main steps; step 19, step 20 and step 21 are end steps.

Application example eight

In the present application embodiment:

(1) the test purpose is as follows: and verifying whether the terminal supports the APP ID association service and the slicing, and whether the service can be normally carried out after the slicing is associated.

(3) And (3) testing conditions are as follows:

[ test System configuration ]

The cell A is an NR cell;

Cell Id=01 TAC = 01；

MCC = 460 MNC = 00；

test band = n 41;

test frequency = f 1;

UL/DL MCS dynamic configuration;

P-MAX=10dBm；

an ideal channel environment.

[ terminal configuration ]

Service A: generating a data packet with the size of 32 bytes based on the data transmission service generated by the application program simulator, and filling a random number sequence;

and B, service B: based on the data transmission service generated by the application program simulator, a data packet with the size of 200 bytes is generated, and a random number sequence is filled.

Watch 29

Watch 30

Watch 31

The test flow comprises the following steps:

step 1: the NR cell remains off;

step 2: the terminal inserts USIM card and starts up;

and step 3: activating an NR cell A;

and 8: the instrument sends a DL NAS TRANSPORT with Payload container type set to "UE Policy container" ('0101' B), carrying MANAGE UE POLICY COMMAND message. The "UE POLICY part type" in the MANAGE UE POLICY COMMAND message is set to "URSP" ('0001' B), "UE POLICY part contents" carries URSP configuration information. See, in particular, tables 6, 7, 29, 9, and 10; wherein, table 29 shows the message contents of the information element UE policy part contents in table 7;

step 10: a user initiates a service A, and a terminal initiates a PDU Session connection establishment request; specifically, the verification terminal sends a UL NAS TRANSPORT message and carries a PDU Session update Request, and carries a correct S-NSSAI identifier in the PDU Session update Request, see table 30 and table 31; wherein, table 30 shows the contents of the UL NAS TRANSPORT message; table 31 shows the message contents of the value PDU SESSION ESTABLISHMENT REQUEST of the information element Payload container in table 30;

step 11: the instrument replies PDU Session update Accept, configures DNN, S-NSSAI and QCI, etc., see Table 28 specifically;

step 13: keeping the normal operation of the service A, initiating a service B by a user, and initiating a PDU Session connection establishment request by a terminal; specifically, the terminal sends a UL NAS TRANSPORT message and carries a PDU Session Establishment Request, and carries a correct S-NSSAI identifier in the PDU Session Establishment Request, see table 30 and table 31; wherein, table 30 shows the contents of the UL NAS TRANSPORT message; table 31 shows the message contents of the value PDU SESSION ESTABLISHMENT REQUEST of the information element Payload container in table 30;

step 14: the instrument replies PDU Session update Accept, configures DNN, S-NSSAI and QCI, etc., see Table 28 specifically;

step 19: ending the service A and the service B;

step 21: NR cell a is deactivated.

In step 16 and step 17, the following steps may be performed in parallel:

after the PDU Session is established, verifying whether the service A can be normally performed on the corresponding PDU Session (the first PDU Session); meanwhile, it is verified whether the service B can be normally performed on the corresponding PDU Session (second PDU Session).

In step 10, the terminal determines a correct S-NSSAI according to the APP ID of the service a in the URSP, so as to associate the service a and the network slice through the APP ID of the service a; accordingly, in step 13, the terminal determines the correct S-NSSAI according to the APP ID of the service B in the URSP, so as to associate the service B and the network slice through the APP ID of the service B.

The specific process of verifying whether the service can be normally performed on the corresponding PDU Session may be understood with reference to the seventh application embodiment, and is not described herein again.

As can be seen from the above description, in this application embodiment, it is tested whether the terminal can associate the correct slice with the APP ID issued by the service and initiate a PDU connection establishment request, and the service can be normally performed after associating the slice. In addition, step 1 and step 2 are leading steps; step 3 to step 18 are main steps; step 19, step 20 and step 21 are end steps.

Application example nine

In the present application embodiment:

(1) the test purpose is as follows: and verifying whether the terminal supports the association of the service and the slice through the FQDN and whether the service can be normally carried out after the association of the slice.

(3) And (3) testing conditions are as follows:

[ test System configuration ]

The cell A is an NR cell;

Cell Id=01 TAC = 01；

MCC = 460 MNC = 00；

test band = n 41;

test frequency = f 1;

UL/DL MCS dynamic configuration;

P-MAX=10dBm；

an ideal channel environment.

[ terminal configuration ]

Watch 32

The test flow comprises the following steps:

step 1: the NR cell remains off;

step 2: the terminal inserts USIM card and starts up;

and step 3: activating an NR cell A;

and 8: the instrument sends a DL NAS TRANSPORT with Payload container type set to "UE Policy container" ('0101' B), carrying MANAGE UE POLICY COMMAND message. The "UE POLICY part type" in the MANAGE UE POLICY COMMAND message is set to "URSP" ('0001' B), "UE POLICY part contents" carries URSP configuration information. See tables 6, 7, 32, 9 and 10; wherein, table 32 shows the message content of the information element UE policy part contents in table 7;

step 10: a user initiates a service A, and a terminal initiates a PDU Session connection establishment request; specifically, the verification terminal sends a UL NAS TRANSPORT message and carries a PDU Session update Request, and carries a correct S-NSSAI identifier in the PDU Session update Request, see table 30 and table 31;

step 13: keeping the normal operation of the service A, initiating a service B by a user, and initiating a PDU Session connection establishment request by a terminal; specifically, the terminal sends a UL NAS TRANSPORT message and carries a PDU Session Establishment Request, and carries a correct S-NSSAI identifier in the PDU Session Establishment Request, see table 30 and table 31;

step 19: ending the service A and the service B;

step 21: NR cell a is deactivated.

In steps 10 to 15, the following steps may be performed in parallel:

In step 10, the terminal determines the correct S-NSSAI according to the FQDN of the service a in the URSP, so as to associate the service a with the network slice through the FQDN of the service a; accordingly, in step 13, the terminal determines the correct S-NSSAI according to the FQDN of the service B in the URSP, so as to associate the service B with the network slice through the FQDN of the service B.

As can be seen from the above description, in this application embodiment, it is tested whether the terminal can associate the correct slice through the FQDN issued by the service and initiate a PDU connection establishment request, and the service can be normally performed after associating the slice. In addition, step 1 and step 2 are leading steps; step 3 to step 18 are main steps; step 19, step 20 and step 21 are end steps.

Application example ten

In the present application embodiment:

(1) the test purpose is as follows: and verifying whether the terminal supports URSP configuration updating and can associate the service to a correct slice after updating the URSP.

(3) And (3) testing conditions are as follows:

[ test System configuration ]

The cell A is an NR cell;

Cell Id=01 TAC = 01；

MCC = 460 MNC = 00；

test band = n 41;

test frequency = f 1;

UL/DL MCS dynamic configuration;

P-MAX=10dBm；

an ideal channel environment.

[ terminal configuration ]

Watch 33

Watch 34

Watch 35

Watch 36

The test flow comprises the following steps:

step 1: the NR cell remains off;

step 2: the terminal inserts USIM card and starts up;

and step 3: activating an NR cell A;

and 8: the instrument sends DL NAS TRANSPORT to configure URSP related parameters to the terminal, wherein, Payload container type is set as 'UE Policy container' ('0101' B) and carries MANAGE UE Policy COMMAND message. The "UE POLICY part type" in the MANAGE UE POLICY COMMAND message is set to "URSP" ('0001' B), "UE POLICY part contents" carries the URSP configuration information, and DNN _1 is mapped to slice 1, specifically see tables 6 to 10;

step 11: the terminal initiates a service A, and the terminal initiates a PDU Session connection establishment request; specifically, the terminal sends a UL NAS TRANSPORT message and carries a PDU Session Establishment Request, and the terminal carries DNN = DNN _1 and a correct S-NSSAI identifier in the PDU Establishment Request, see table 33 and table 34; wherein, table 33 shows the content of the UL NAS TRANSPORT message; table 34 shows the message contents of the value PDU SESSION ESTABLISHMENT REQUEST of the information element Payload container in table 33;

step 12: after the instrument receives the request, the terminal is configured with DNN, S-NSSAI, QCI and the like by replying PDU Session Establishment Accept, see Table 35; wherein, table 35 shows the content of PDU Session Establishment Accept message;

step 13: after the PDU Session is established, executing a test process to verify whether the service A is normally carried out on the first PDU Session;

step 14: the instrument sends DL NAS TRANSPORT to update and configure URSP related parameters to the terminal, wherein, Payload container type is set as UE Policy container ('0101' B) and carries MANAGE UE Policy COMMAND message. "UE POLICY part type" in the MANAGE UE POLICY COMMAND message is set to "URSP" ('0001' B), "UE POLICY part contents" carries the URSP configuration information, and DNN _1 is mapped to slice 2, see table 36 specifically; wherein, table 36 shows the message content of the information element UE policy part contents;

step 15: the terminal replies UL NAS TRANSPORT and carries MANAGE UE POLICY COMPLETE message;

step 16: a user initiates a service A, a terminal initiates a PDU Session connection Establishment Request, specifically, the terminal sends a UL NAS TRANSPORT message and carries a PDU Session Establishment Request, the terminal carries DNN = DNN _1 and a correct S-NSSAI identifier in the PDU Establishment Request, see table 33 and table 34;

and step 17: after the instrument receives the request, the PDU Session Establishment Accept is replied, and DNN, S-NSSAI, QCI and the like are configured, which is shown in a table 35;

step 18: after the PDU Session is established, executing a test process to verify whether the service A is normally carried out on a second PDU Session;

step 19: ending the test flow of the service A and generating a test result of the terminal; the test result comprises the result that the test of each test flow passes or fails;

step 20: ending the service A and the service B;

step 21: the instrument issues an RRCRelease message to release an NR link, namely an RRC link;

step 22: NR cell a is deactivated.

As can be seen from the above description, in the present application embodiment, it is tested whether the terminal can support URSP update, and can correctly associate the traffic onto the updated slice. In addition, step 1 and step 2 are leading steps; step 3 to step 19 are main steps; step 20, step 21 and step 22 are end steps.

According to the scheme provided by the embodiment of the application, one test input quantity 1 can be configured for the terminal, and a plurality of test output quantities 1-P of the terminal can be tested, as shown in FIG. 2; a plurality of test input quantities 1-K can also be configured to the terminal, and a plurality of test output quantities 1-P of the terminal can also be tested, as shown in FIG. 3, where K and P can be equal or different.

In order to implement the method according to the embodiment of the present application, an embodiment of the present application further provides a 5G slice testing apparatus, which is disposed on a testing system, and as shown in fig. 4, the apparatus includes:

a configuration unit 401, configured to configure a test parameter associated with a network slice service to a terminal;

a testing unit 402, configured to execute at least two testing processes on the terminal in a process of performing a network slicing service on the terminal, so as to obtain a testing result of a testing index of each testing process; the at least two test processes are used for testing the test indexes of the terminal in a multi-network slicing service concurrent scene or testing the test indexes of the terminal in a single-network slicing service scene; and generating a test result aiming at the terminal by using the test result of the test index of each test flow.

In an embodiment, the configuration unit 401 is configured to configure at least two PDU sessions to the terminal;

at least two services are executed in parallel on at least two PDU sessions.

In an embodiment, the configuration unit 401 is configured to configure one PDU Session on one network slice to the terminal, where the one PDU Session corresponds to at least two services;

and executing the at least two services in parallel on the PDU Session.

In an embodiment, the configuration unit 401 is configured to configure a value of a test parameter to the terminal once;

the test unit 402 is configured to:

in the process of carrying out the network slicing service on the terminal, sequentially executing the at least two test flows on the terminal;

In an embodiment, the testing unit 402 is configured to, when a test result obtained by a first test procedure of the at least two test procedures does not satisfy a test judgment condition, end the test; the first test flow is the other test flows except the last test flow in at least two test flows executed in sequence.

the test unit 402 is configured to:

in the process of carrying out network slicing service on the terminal, executing the at least two test flows to the terminal in parallel;

In an embodiment, the configuration unit 401 is configured to configure Q test parameters to the terminal; q is an integer greater than or equal to 2;

the test unit 402 is configured to:

when the terminal carries out the network slicing service, the terminal is sequentially executed with the at least two test flows every time the test parameters are configured;

In an embodiment, in the process of sequentially executing the at least two test flows to the terminal each time, when the test value of the test indicator obtained by the first test flow in the at least two test flows does not satisfy the test judgment condition, the test unit 402 ends the test; the first test flow is the other test flows except the last test flow in at least two test flows executed in sequence.

the test unit 402 is configured to:

when the terminal carries out the network slicing service, the terminal is executed with the at least two test flows in parallel every time the test parameters are configured;

In practical applications, the configuration unit 401 and the test unit 402 may be implemented by a processor in a 5G slice test apparatus in combination with a communication interface.

It should be noted that: in the test apparatus provided in the above embodiment, only the division of the program modules is exemplified when performing the test, and in practical applications, the above processing may be distributed to different program modules according to needs, that is, the internal structure of the apparatus may be divided into different program modules to complete all or part of the above-described processing. In addition, the 5G slice testing apparatus and the testing method provided by the above embodiments belong to the same concept, and specific implementation processes thereof are detailed in the method embodiments and are not described herein again.

Based on the hardware implementation of the program module, and in order to implement the method according to the embodiment of the present application, an embodiment of the present application further provides a 5G slice testing system, as shown in fig. 5, the 5G slice testing system 500 includes:

a communication interface 501 capable of performing information interaction with a terminal;

and the processor 502 is connected with the communication interface 501 to realize information interaction with a terminal, and is used for executing the method provided by one or more technical schemes at the 5G slice test system side when running a computer program. And the computer program is stored on the memory 503.

Specifically, the processor 502 is configured to:

configuring test parameters associated with the network slice service to the terminal through the communication interface 501; executing at least two test flows to the terminal in the process of carrying out the network slicing service on the terminal to obtain a test result of a test index of each test flow; the at least two test processes are used for testing the test indexes of the terminal in a multi-network slicing service concurrent scene or testing the test indexes of the terminal in a single-network slicing service scene; and generating a test result aiming at the terminal by using the test result of the test index of each test flow.

In an embodiment, the processor 502 is configured to: configuring at least two PDU sessions to the terminal through the communication interface 501;

at least two services are executed in parallel on at least two PDU sessions.

In one embodiment, the processor 502 is configured to:

configuring the value of a primary test parameter to the terminal;

In an embodiment, the processor 502 is configured to end the test when a test result obtained by a first test procedure of the at least two test procedures does not satisfy a test judgment condition; the first test flow is the other test flows except the last test flow in at least two test flows executed in sequence.

In one embodiment, the processor 502 is configured to:

configuring the value of a primary test parameter to the terminal;

In one embodiment, the processor 502 is configured to:

configuring Q times of test parameters to the terminal; q is an integer greater than or equal to 2;

In an embodiment, in the process of sequentially executing the at least two test flows to the terminal each time, when a test value of a test indicator obtained by a first test flow of the at least two test flows does not satisfy a test judgment condition, the processor 502 ends the test; the first test flow is the other test flows except the last test flow in at least two test flows executed in sequence.

In one embodiment, the processor 502 is configured to:

the terminal is used for configuring Q times of test parameters to the terminal; q is an integer greater than or equal to 2;

It should be noted that: the specific processing of the processor 502 may be understood with reference to the methods described above.

Of course, in practice, the various components in the 5G slice test system 500 are coupled together by a bus system 504. It is understood that the bus system 504 is used to enable communications among the components. The bus system 504 includes a power bus, a control bus, and a status signal bus in addition to a data bus. For clarity of illustration, however, the various buses are labeled as bus system 504 in fig. 5.

The memory 503 in the embodiments of the present application is used to store various types of data to support the operation of the 5G slice test system 500. Examples of such data include: any computer program for operating on 5G slice test system 500.

The method disclosed in the embodiments of the present application may be applied to the processor 502 or implemented by the processor 502. The processor 502 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be implemented by integrated logic circuits of hardware or instructions in the form of software in the processor 502. The Processor 502 described above may be a general purpose Processor, a Digital Signal Processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. The processor 502 may implement or perform the methods, steps, and logic blocks disclosed in the embodiments of the present application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the method disclosed in the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium located in the memory 503, and the processor 502 reads the information in the memory 503 to complete the steps of the foregoing method in combination with the hardware thereof.

In an exemplary embodiment, the 5G slice test system 500 may be implemented by one or more Application Specific Integrated Circuits (ASICs), DSPs, Programmable Logic Devices (PLDs), Complex Programmable Logic Devices (CPLDs), Field Programmable Gate Arrays (FPGAs), general purpose processors, controllers, Micro Controllers (MCUs), microprocessors (microprocessors), or other electronic components for performing the foregoing methods.

It is to be appreciated that the memory 503 of the subject embodiment can be either volatile memory or nonvolatile memory, and can include both volatile and nonvolatile memory. Among them, the nonvolatile Memory may be a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a magnetic random access Memory (FRAM), a Flash Memory (Flash Memory), a magnetic surface Memory, an optical disk, or a Compact Disc Read-Only Memory (CD-ROM); the magnetic surface storage may be disk storage or tape storage. Volatile Memory can be Random Access Memory (RAM), which acts as external cache Memory. By way of illustration and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Enhanced Synchronous Dynamic Random Access Memory (Enhanced DRAM), Synchronous Dynamic Random Access Memory (SLDRAM), Direct Memory (DRmb Access), and Random Access Memory (DRAM). The memories described in the embodiments of the present application are intended to comprise, without being limited to, these and any other suitable types of memory.

In an exemplary embodiment, the present application further provides a storage medium, specifically a computer-readable storage medium, for example, a memory 503 storing a computer program, which can be executed by the processor 502 of the 5G slice test system 500 to complete the steps of the foregoing method. The computer readable storage medium may be Memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface Memory, optical disk, or CD-ROM.

It should be noted that: "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.

The technical means described in the embodiments of the present application may be arbitrarily combined without conflict.

The above description is only a preferred embodiment of the present application, and is not intended to limit the scope of the present application.

Claims

1. A5G slice test method is characterized by comprising the following steps:

generating a test result aiming at the terminal by using the test result of the test index of each test flow;

wherein:

configuring the value of a primary test parameter to the terminal; in the process of carrying out the network slicing service on the terminal, sequentially executing the at least two test flows on the terminal; generating a test result aiming at the terminal based on the obtained test result of the test index of the test flow and the corresponding test judgment condition; the test judgment condition is used for judging the success or failure of the corresponding test process;

alternatively, the first and second electrodes may be,

configuring the value of a primary test parameter to the terminal; in the process of carrying out network slicing service on the terminal, executing the at least two test flows to the terminal in parallel; generating a test result for the terminal based on the test result of the test index of each test flow and the corresponding test judgment condition; the test judgment condition is used for judging the success or failure of the corresponding test process;

alternatively, the first and second electrodes may be,

configuring Q times of test parameters to the terminal; when the terminal carries out the network slicing service, the terminal is sequentially executed with the at least two test flows every time the test parameters are configured; generating a test result aiming at the terminal based on the test result of the test index of the test flow obtained each time and the corresponding test judgment condition; the test judgment condition is used for judging the success or failure of the corresponding test process;

alternatively, the first and second electrodes may be,

configuring Q times of test parameters to the terminal; when the terminal carries out the network slicing service, the terminal is executed with the at least two test flows in parallel every time the test parameters are configured; generating a test result aiming at the terminal based on the test result of the test index of the test flow obtained each time and the corresponding test judgment condition; the test judgment condition is used for judging the success or failure of the corresponding test process;

and Q is an integer greater than or equal to 2.

2. The method of claim 1,

indexes of each test flow are different, and the service of each test flow is the same;

3. The method of claim 1, wherein in a scenario of concurrent multi-network slice services, the number of services is N, and the number of network slices is M; m is less than or equal to N; m and N are integers.

4. The method of claim 3, wherein at least two services use the same network slice or one network slice per service;

a test flow is executed for each service or on the PDU Session of the protocol data unit Session of each network slice.

5. The method of claim 4, further comprising:

configuring at least two PDU sessions to the terminal;

at least two services are executed in parallel on at least two PDU sessions.

6. The method of claim 4, further comprising:

configuring one PDU Session on one network slice to the terminal, wherein the PDU Session corresponds to at least two services;

and executing the at least two services in parallel on the PDU Session.

7. The method of claim 1, wherein the test indexes of each test procedure are correlated with each other when the test indexes of each test procedure are different.

8. The method of claim 1, wherein the network slice traffic associated test parameters comprise at least one of:

parameters for characterizing a network slice;

parameters for characterizing service attributes;

parameters of the terminal routing policy URSP.

9. The method of claim 8, wherein the parameters characterizing the network slice comprise at least one of:

network slice identification;

quality of service, QoS;

and/or the presence of a gas in the gas,

an application identifier of the service;

fully qualified domain name FQDN information of the service;

internet Protocol (IP) triple information of the service;

DNN information of the data network name of the service;

connection capability CC information of the service.

10. The method of claim 1, wherein the two test flows comprise a first test flow and a second test flow; the first test flow and the second test flow comprise one or a combination of the following test indexes:

downlink throughput;

time delay;

an uplink throughput;

power consumption.

11. The method of claim 1, wherein a virtual network environment is constructed; the network slicing service performed on the terminal is a service of a virtual network environment.

12. The method of claim 1,

configuring a value of a test parameter to the terminal once, and in the process of performing a network slicing service on the terminal, under the condition that the terminal is sequentially executed with the at least two test flows:

13. The method of claim 1,

when Q times of test parameters are configured for the terminal and each time of test parameters are configured, the terminal is sequentially executed with the at least two test flows in the process of carrying out the network slicing service by the terminal:

in the process of sequentially executing the at least two test flows to the terminal each time, when the test value of the test index obtained by the first test flow in the at least two test flows does not meet the test judgment condition, ending the test; the first test flow is the other test flows except the last test flow in at least two test flows executed in sequence.

14. A5G slice testing device, comprising:

the terminal comprises a testing unit and a processing unit, wherein the testing unit is used for executing at least two testing processes on the terminal in the process of carrying out network slicing service on the terminal to obtain a testing result of a testing index of each testing process; the testing indexes of each testing process are different and/or the service of each testing process is different; the at least two test processes are used for testing the test indexes of the terminal in a multi-network slicing service concurrent scene or testing the test indexes of the terminal in a single-network slicing service scene; generating a test result aiming at the terminal by using the test result of the test index of each test flow;

wherein:

the configuration unit is further configured to configure a value of a primary test parameter to the terminal; the test unit is further configured to sequentially execute the at least two test flows to the terminal in a process of performing a network slicing service on the terminal; generating a test result aiming at the terminal based on the obtained test result of the test index of the test flow and the corresponding test judgment condition; the test judgment condition is used for judging the success or failure of the corresponding test process;

alternatively, the first and second electrodes may be,

the configuration unit is further configured to configure a value of a primary test parameter to the terminal; the test unit is further configured to: in the process of carrying out network slicing service on the terminal, executing the at least two test flows to the terminal in parallel; generating a test result for the terminal based on the test result of the test index of each test flow and the corresponding test judgment condition; the test judgment condition is used for judging the success or failure of the corresponding test process;

alternatively, the first and second electrodes may be,

the configuration unit is further configured to configure Q test parameters to the terminal; the test unit is further configured to sequentially execute the at least two test flows to the terminal in a process that the terminal performs a network slicing service every time the test parameter is configured; generating a test result aiming at the terminal based on the test result of the test index of the test flow obtained each time and the corresponding test judgment condition; the test judgment condition is used for judging the success or failure of the corresponding test process;

alternatively, the first and second electrodes may be,

the configuration unit is further configured to configure Q test parameters to the terminal; the test unit is further configured to execute the at least two test flows in parallel to the terminal in a process that the terminal performs a network slicing service every time the test parameter is configured; generating a test result aiming at the terminal based on the test result of the test index of the test flow obtained each time and the corresponding test judgment condition; the test judgment condition is used for judging the success or failure of the corresponding test process;

and Q is an integer greater than or equal to 2.

15. A 5G slice test system, comprising: a processor and a communication interface; wherein the content of the first and second substances,

the processor is configured to:

configuring test parameters associated with the network slice service to the terminal through the communication interface; executing at least two test flows to the terminal in the process of carrying out the network slicing service on the terminal to obtain a test result of a test index of each test flow; the at least two test processes are used for testing the test indexes of the terminal in a multi-network slicing service concurrent scene or testing the test indexes of the terminal in a single-network slicing service scene; generating a test result aiming at the terminal by using the test result of the test index of each test flow;

wherein:

the processor is further configured to configure a value of a primary test parameter to the terminal; in the process of carrying out the network slicing service on the terminal, sequentially executing the at least two test flows on the terminal; generating a test result aiming at the terminal based on the obtained test result of the test index of the test flow and the corresponding test judgment condition; the test judgment condition is used for judging the success or failure of the corresponding test process;

alternatively, the first and second electrodes may be,

the processor is further configured to configure a value of a primary test parameter to the terminal; in the process of carrying out network slicing service on the terminal, executing the at least two test flows to the terminal in parallel; generating a test result for the terminal based on the test result of the test index of each test flow and the corresponding test judgment condition; the test judgment condition is used for judging the success or failure of the corresponding test process;

alternatively, the first and second electrodes may be,

the processor is further configured to configure Q test parameters to the terminal; when the terminal carries out the network slicing service, the terminal is sequentially executed with the at least two test flows every time the test parameters are configured; generating a test result aiming at the terminal based on the test result of the test index of the test flow obtained each time and the corresponding test judgment condition; the test judgment condition is used for judging the success or failure of the corresponding test process;

alternatively, the first and second electrodes may be,

the processor is further configured to configure Q test parameters to the terminal; when the terminal carries out the network slicing service, the terminal is executed with the at least two test flows in parallel every time the test parameters are configured; generating a test result aiming at the terminal based on the test result of the test index of the test flow obtained each time and the corresponding test judgment condition; the test judgment condition is used for judging the success or failure of the corresponding test process;

and Q is an integer greater than or equal to 2.

16. A 5G slice test system, comprising: a processor and a memory for storing a computer program capable of running on the processor,

wherein the processor is adapted to perform the steps of the method of any one of claims 1 to 13 when running the computer program.

17. A storage medium having a computer program stored thereon, the computer program, when being executed by a processor, performing the steps of the method of any one of claims 1 to 13.