JP2022533377A - Mobile IoT network active test system and test method using the test system - Google Patents

Abstract

An active test system (1) for a mobile IoT network (2) providing connectivity and services to mobile IoT (MIoT) devices in Low Power Wide Area (LPWA) technology is presented. The test system comprises at least one test probe (3) connected to the MIoT network (2) via the LTE-Uu interface (5) and/or at least one test probe connected to the MIoT network via the S1 interface It has one test probe. The central test unit (5a) is connected (8) to at least one test probe (3) via a wireless backhaul network or a fixed IP network (7). A SIM multiplexer (12) is provided for transferring SIM data to at least one test probe (3) of the test field. An enhanced test system ensures a mobile IoT experience.

Description

This application claims priority from German patent application DE 10 2019 207 051.5 and US patent application US 16/412 459, the contents of which are incorporated herein by reference.

The present invention relates to an active test system for mobile IoT networks. The present invention also relates to a test method using the test system.

Mobile network testing systems are known, for example, from US Pat.

US Pat. No. 5,300,000 discloses a method and apparatus for separating wireless segments in a mobile communication network. US Pat. No. 5,900,000 discloses a method and test system for authenticating mobile test equipment in a mobile communication network.

US 10,097,981 B1 US 7,831,249 B2 WO 2004/049746 A1 US 9,768,893 B1 DE 10 2005 027 027 B4

It is an object of the invention to enhance the performance of such test systems for testing mobile networks.

That object is met by an active test system comprising the features of claim 1 .

A test system according to the present invention can perform testing of mobile Internet of Things (IoT) networks that connect to and provide services to mobile IoT (MIoT) devices. Such tests are active tests, ie require at least one component to actively initiate the respective test method. For example, a central testing unit or part thereof may be a component for actively initiating the testing method.

The mobile IoT network under test is considered a sub-type of installed 4G networks enhanced with LPWA (Low Power Wide Area) technology for power saving devices, enhanced coverage, transmission of small amounts of data and acceptable delays. .

The LPWA technology onboard may be LTE-M and/or NB-IoT. LPWA mobile devices connected to MIoT networks are smart meters, home automation equipment, building automation equipment, part of smart grids, part of industrial production line or pipeline management, part of automobiles, part of transportation equipment and logistics. parts, drones, parts of home security equipment, parts of patient monitoring equipment, parts of agricultural equipment such as irrigation and shadowing, parts of street lighting equipments, parts of tracking equipment, industrial assets It can be part of a management device, part of retail/POS equipment, or part of a wearable device (eg, part of a watch or part of a smart phone). It can also test voice services over LTE-M.

A mobile IoT network may be connected to a MIoT application platform and/or an IoT application platform via an application server.

Such a test system may be used to perform MIoT network connectivity tests and/or MIoT application platform tests. The test system is adapted according to the IoT network architecture and scalability and can introduce one or more test probes. The test probes can be installed at different locations (test fields) within a single IoT network or between multiple interconnected networks. In particular, data communication embodied by the SIM of mobile IoT devices can be simulated and/or emulated by either the LTE-Uu radio interface or the S1 core network interface.

A SIM multiplexer can virtually and/or securely transfer SIM data to at least one test probe.

A SIM multiplexer can be embodied as a support for carrying multiple SIMs, for example up to three or more SIMs.

The test system can be configured to perform mobile IoT test procedures deploying an end-to-end active test methodology between at least one test probe and a MIoT network under test. A test system can be configured to control test probe(s) via a particular active test platform that includes a central test unit. Furthermore, through the central test unit, the test system can automatically execute IoT test procedures, collect test results, and generate test reports and dashboards.

Within the test system, test methods and test sequences are deployed to test MIoT applications and/or services running beyond the IoT connectivity provided by the MIoT network under test.

"End-to-end" testing means that the connectivity between a MIoT device and a MIoT application server and the services provided by a MIoT application to a MIoT device are validated on behalf of and in particular by the MIoT device during round trips to and from the MIoT device. It is meant to be tested using data transfer to and from at least one simulated test probe.

In particular, it tests whether the application's service data flow is working as expected from start to finish. In particular, all steps of the application and/or service are tested.

Download and upload data speeds and/or download or upload bandwidth can be tested.

Data transfer tests can be performed with different sizes of data to be sent and received, in particular different numbers of data packets and/or different amounts of data.

Data transfer quality and data transfer integrity can be tested.

Additionally, the test system can be designed to test the performance of MIoT networks in power save mode (PSM) and/or enhanced discontinuous reception (eDRX) deployments for MIoT applications.

The test system configures at least one test probe to trigger and initiate at least one of a group of power save mode (PSM) or enhanced discontinuous reception (eDRX) modes in the serving MIoT network under test. , can be designed to start. The test system can be specifically designed to negotiate power save and eDRX modes in combination with configuration and activation of test probes to Evolved Packet System (EPS) attachments in the serving MIoT network under test.

The test system can be designed to configure and activate at least one test probe to access and interrogate IoT application servers using various protocols or via device-specific interfaces. . Various protocols include, but are not limited to, oneM2M, Hypercat, constrained application protocol (CoAP), MQTT/MQTT-SN (message queuing telemetry transport), real-time streaming protocol (RTSP). Device-specific interfaces also include JavaScript® Object Notation (JSON) and/or Extensible Markup Language (XML) over HTTP.

Literature on the oneM2M protocol can be found at www. onem2m. org can be accessed. Information on protocol "Hypercat" can be found in John Davies, Hypercat: resource discovery on the Internet of Things (January 12, 2016): IEEE Internet of Things, March 2, 2017. http://iot. IEEE. org. Information about the protocol "CoAP" can be found in RFC7252. This is https://tools. ietf. Available from org/html/rfc7252. Information about JSON can be found in RFC8259 and ECMA-404. Information about RTSP can be found in RFC2326.

Signaling and data exchange according to claim 2 allows testing of the most common signaling messages and data types using the test system.

The arrangement according to claim 3 has proven to be essential for the most common test requirements.

This applies in particular to the test system according to claim 4.

The message structure according to claim 5 is suitable for testing IoT application platforms. Alternatively or additionally, applicable protocols and/or interfaces for communicating with such test systems are oneM2M, Hypercat, CoAP, RTSP, JSON, XML.

The test method according to claim 6 has the advantages described above with respect to the test system according to the invention. This test method is in particular an end-to-end test method. This testing method includes in particular testing an IoT application platform, in particular a server of the platform.

The method of claim 7 enables testing the availability of IoT services of the network by simulating/emulating respective mobile IoT devices with test probes in the IoT network. The testing steps may be repeated periodically during the testing method. Recorded test results may be aggregated and further statistically evaluated.

In the test method of claim 8, mobile IoT connectivity can be performed. Again, the iterative steps may be repeated periodically and the test results aggregated for further statistical evaluation.

Such a ping test can assess the IoT network accessibility of the ping test probe and/or the round trip time of the ping/echo.

The test method according to claim 9 can test the power saving function of each mobile IoT device. Again, the iterative steps may be repeated periodically and the test results aggregated for further statistical evaluation.

As part of such a power saving test methodology, mobile-terminated data transfer combined with power saving features managed by the IoT serving network under test transmit downlink data towards the test probe during T3324 active timer execution. may be tested by verifying that the test probe receives complete downlink data packets, monitoring and recording all test events, and repeating the above test according to a predetermined schedule. Again, the iterative steps may be repeated periodically and the test results aggregated for further statistical evaluation.

Additionally, in such power saving tests, the mobile-terminated SMS combined with power saving features managed by the IoT serving network under test send SMS to the test probe during T3324 active timer running, and the test probe sends the SMS It can be tested by verifying receipt, monitoring and recording all test events, and repeating the above tests according to a predetermined test schedule. Again, the iterative steps may be repeated periodically and the test results aggregated for further statistical evaluation.

A test method according to claim 10 tests the functionality of the eDRX so that the capability of further power saving functionality can be evaluated. Again, the iterative steps may be repeated periodically and the test results aggregated for further statistical evaluation.

Such an eDRX test method involves mobile-terminated data transfer combined with eDRx functionality managed by the IoT serving network under test to transmit downlink data towards the test probe within a paging time window (PTW) and test It can be tested by verifying that the probe receives complete downlink data packets, monitoring and recording all test events, and repeating the above tests according to a predetermined test schedule. Again, the iterative steps may be repeated periodically and the test results aggregated for further statistical evaluation.

Further, in such an eDRX test method, the mobile-terminated SMS combined with the eDRX functionality managed by the IoT serving network under test sends an SMS to the test probe within the paging time window (PTW), and the test probe sends the SMS It may be tested by verifying receipt, monitoring and recording all test events, and repeating the above tests according to a predetermined test schedule. Again, the iterative steps may be repeated periodically and the test results aggregated for further statistical evaluation.

The testing method of claim 11 enables testing connection persistence and unsolicited network-initiated detach requests.

A testing method as claimed in claim 12, wherein mobile originated (MO) data transfer can be tested.

A testing method as claimed in claim 13, wherein mobile terminated (MT) data transfer can be tested.

The testing method of claim 14 allows testing of mobile-originated SMS transmissions.

A testing method according to claim 15, wherein mobile-terminated SMS delivery can be tested.

Data and SMS data delivery can be tested after and during power save mode.

Exemplary embodiments of the invention will be further described with reference to the accompanying drawings.

A key component of an active test system for mobile Internet of Things (IoT) networks, comprising at least one test probe connected to the IoT network via a radio interface. FIG. 2 is a view similar to FIG. 1 and a further embodiment of a test system for a mobile IoT network comprising a test probe connected to the IoT network via an S1 interface; 2 is similar to FIG. 1 and is an embodiment of a test system configured to test an IoT serving network on a test connection path over a roaming interface; FIG. A main component of an embodiment of a test system including two test probes configured to communicate with an IoT application platform of an IoT service via MQTT/MQTT-SN messages.

FIG. 1 shows the main components of an active test system 1 for a mobile Internet of Things (IoT) network 2 represented by the various communication lines shown in FIG. A communication link is either a pure signaling path, a signaling path with embedded IoT data, or an IoT data transport path. A mobile IoT (MIoT) network 2 provides connectivity and services to mobile IoT devices in Low Power Wide Area (LPWA) technology. The LPWA frequency bands used are in regularly licensed frequency bands. The LPWA technology introduced may be LTE-M and/or NB-IoT.

Throughout this application, reference is made to the following documents, particularly with respect to standardized specifications for IoT networks:
- GSM Association; official document CLP. 28-NB-IoT Deployment Guide to Basic Feature set Requirements, version 1.0, August 2, 2017 (GSMA White Paper).
- Technical specifications 3GPP TS 23.682, V. 15.8.0, Release 15, March 2019.

The mobile IoT network tested via the test system 1 may also be equipped with EC-GSM-IoT (Extended Coverage GSM IoT). Other communication techniques may also be used for additional network access for machine-to-machine communication. For example, examples of short-range wireless communication include Bluetooth (registered trademark) Mesh Networking, Light-Fidelity (Li-Fi), Near-Field Communication (NFC), Wi-Fi, ZigBee (registered trademark). ) or Z-Wave. Examples of medium-range wireless communication include LTE-Advanced or LTE-Standard, examples of long-range wireless communication include Lo-RaWan, Sigfox, or Weightless, or very small earth stations (VSAT), and examples of wired communication include Ethernet or power line communication.

The test system 1 of FIG. 1 includes one or more test probes 3 that are components of local units 4 of the test system 1 . Some examples of such local units 4 are shown in FIG. A local unit 4 may include from 1 to 4 or more, eg up to 15 or more test probes 3 . A test probe 3 is connected to the IoT network 2 via a radio interface 5 . Each interface 5 is schematically shown in FIG. 1 with an LTE RAN (Radio Access Network) and may be embodied by multiple antenna sites.

The central test unit 5a is connected to the test probes 3 via internet networks 7,8. Such a connection could be, for example, a permanent secure IP connection via a VPN server and an LTE/GPRS/EDGE/HSPA modem, or established via the VPN server when needed during testing. It may be a semi-permanent IP connection.

Network 7, which in the embodiment of FIG. 1 is a wireless backhaul network or a fixed IP network, includes the Internet 8 and further includes components of the Evolved Packet Core (EPC) of the 3GPP LTE communication standard. In general, components of network 7 are also components of IoT network 2 , but this is not required and some of the components of network 7 may be independent of IoT network 2 .

A further part of the test system 1 shown on the left side of FIG. 1 is the test client 5b which is also connected to the networks 7, 8 of the test system 1.

Figure 1 shows two main test communication paths between a test probe 3 and an application server (AS) 6 acting as an IoT platform. First, the NB-IoT test communication line 9 extends from the local radio interface 5 it belongs to via the MME (Mobile Management Entity), and is further divided into two ways: the Serving Gateway (S-GW) and the Packet Data Network Gateway ( P-GW) and network 7 to AS / IoT platform 6 (so-called direct mode), test line 11, SCEF (Service Cap-ability Exposure Function), SCS (Services Capability Server), and network 7 to reach the AS/IoT platform 6 (so-called indirect mode).

A further LTE-M test communication path 10 extends directly from the local radio interface 5 to the S-GW (i.e. without going through the MME) and further between the Serving Gateway (S-GW) and the Packet Data Network Gateway (P-GW). and the network 7 to reach the AS/IoT platform 6 .

All test probes are equipped with a SIM. The SIM multiplexer 12 is connected to the Internet 8 and virtually transmits SIM data to individual test probes in a completely secure and reliable manner. A SIM multiplexer that is part of the test system 1 is known from US Pat.

Local unit 4 and local unit 16 are equipped with SIM multiplexer support.

Instead of SIM multiplexer support, the local unit 4 may include support for carrying multiple SIMs, eg up to three or more SIMs.

A local unit 4 is placed in a given test field and connected to the test system. In practice, a plurality of test probes 3 are located at different locations, especially in a wide range of locations covering national or inter-national areas. As a result, at least one test probe 3 is configured to be placed either in a home IoT network under test for national IoT service testing or in a visited IoT network under test for international IoT roaming service testing. there is

The test system 1 is configured to perform a mobile IoT test procedure deploying an end-to-end active test method between at least one test probe 3 and an IoT network 2 under test.

Further, the test system 1 is configured to control at least one test probe 3, configured to automatically perform IoT test procedures, configured to collect test results, and test reports and/or or configured to create dashboards.

The test system 1 is configured to exchange signaling messages and is configured to transmit IP and/or non-IP data and/or SMS to/from the IoT network 2 under test.

A test communication path between the test probe 3 and the IoT platform 6 through the Internet network 8 includes an MQTT/MQTT-SN client/server structure, where the test probe 3 is the MQTT client and the IoT platform 6 is the server/MQTT is a broker. IoT application data stored in the IoT platform can be obtained, evaluated and verified via the messaging protocol MQTT (Message Queuing Telemetry Transport).

As further components in the network and/or communication path, SCS (Service Capability Server) and/or AS (Application Server) may function. For such an SCS/AS arrangement, refer to Technical Specification 3GPP TS 23.682, in particular Figure 4.2-1a.

Further possible interfaces may operate according to standardized S6a, S8, SGd or T7 roaming interfaces.

Via a test method or procedure implemented by the test system 1, the availability and quality of the serving IoT network 2 under test can be tested. During service availability testing, in particular the timing of each test event can be monitored and recorded in the central testing unit 5a.

The test methods described are controlled by a central test unit 5a unless otherwise indicated.

An example of such a test method includes the following steps.

Each test probe 3 is configured via a test client 5b and initiated to an EPS (Evolved Packet System) attach on the serving IoT network 2 under test. After starting the EPS attach, completion of the attach procedure, the received message from the IoT network under test by the test probe 3 is verified by the central test unit 5a.

All test events are monitored and recorded during the setup, start-up and verification steps. These test steps are repeated according to a predetermined test schedule. In particular, such repetition may be periodic re-execution of the test steps. Furthermore, such multiple test results are aggregated, transferred to statistical evaluation, and the test results are presented to the tester (human tester) via the test client 5b.

In a particular test method, a test probe is initiated to ping a server or IoT network component (eg, P-GW) installed in the serving IoT network 2 under test for network connectivity.

This "ping" is done using the respective IP software utility.

After such a ping procedure, its completion is verified, again all test events are monitored and recorded, and the test steps are repeated according to a predetermined test schedule.

A further test method can test power saving features managed by the serving IoT network 2 under test. Testing of such power saving features includes enabling a power saving mode (PSM) on each test probe 3, which results in a T3324 active timer and an extended T3412 timer on the local unit 4 test probe 3. set the value of It then initiates an EPS attach of the test probe 3 in the serving IoT network 2 to verify completion of the attach procedure. Additionally, it is verified whether the timer value is accepted by the serving IoT network 2 . This is done by comparing these values with the values requested by the respective test probes 3 . Additionally, it is verified whether an extended periodic Tracking Area Update (TAU) procedure has been completed. Again, all test events are monitored and recorded during this test method and the above tests are repeated according to a predetermined schedule.

In a further test method, mobile-terminated data transfer in combination with power saving features (PSM) managed by the serving IoT network 2 under test is tested. For this purpose, the downlink data to each test probe 3 is transmitted during the period when the T3324 active timer is running. It is verified that each test probe 3 receives complete downlink data packets.

The PSM verifies whether each test probe 3 can wake up from the power save state to the communication state. Such a wakeup is performed, for example, once a week for two minutes. The EPS attach procedure negotiates such a wakeup duty cycle. The test method tests wake-up functionality and, in particular, state changes between a power save state and a communication state.

Again, all test events are monitored and recorded during this test method and the above tests are repeated according to a predetermined test schedule.

In a further test method for testing mobile-terminated SMS in combination with power saving features managed by the serving IoT network 2 under test, the SMS is sent to the test probe 3 while the T3324 active timer is running.

It is then verified whether the SMS has reached the test probe correctly. All test events during this test method are again monitored and recorded and the above test method is repeated according to the predetermined test schedule.

In a further test method, the eDRX (Extended Discontinuous Reception) functionality managed by the serving IoT network 2 under test is tested. In this method, eDRX is enabled and eDRX cycle length and paging time window (PTW) values are set on each test probe 3 of the local unit 4 . Furthermore, the EPS attach of the test probe 3 to the serving IoT network 2 under test is initiated. Completion of such an attach procedure is verified. Furthermore, it is verified whether the eDRX cycle length and PTW values are accepted by the serving IoT network 2 by comparing these values with the values requested by the respective test probes 3 . All test events of this method are monitored and recorded, and the above test method is repeated according to a predetermined test schedule.

In a further test method, mobile-terminated data transfer in combination with eDRX functionality managed by the serving IoT network 2 under test is tested. Here, within the PTW (paging time window), downlink data is sent towards each test probe 3 . It is verified whether the test probe 3 receives complete downlink data packets. All test events of this test method are monitored and recorded and the test method is repeated according to a predetermined test schedule.

In a further test method a mobile-terminated SMS in combination with eDRX functionality managed by the serving IoT network 2 under test is tested. Here an SMS is sent to each test probe 3 within the PTW. It is verified whether the SMS is delivered correctly to the test probe. All test events during this test method are monitored and recorded. The above test method is repeated according to a predetermined test schedule. In a further test method the connection persistence of the IoT network 2 is tested. Here, it is verified whether each test probe 3 is requested to detach from the serving IoT network 2 after EPS attach or after mobile originated (MO) or mobile terminated (MT) data transfer. This verification step is repeated multiple times. Multiple test results of this test method are aggregated. From this aggregation, a default EPS bearer context cutoff ratio is calculated.

In a further method the IoT MO data transfer provided by the serving IoT network 2 under test is tested. Here, the TCP (Transmission Control Protocol) transport protocol is deployed. In addition, a mobile-originated IoT data transfer is initiated from each test probe 3 to an application server 6 located in the home network (HPMN). It is verified whether the IoT data has been received correctly at the application server 6 . This verification step was repeated multiple times and the results of multiple tests were aggregated to give the default EPS bearer context cutoff ratio. Also, UDP (User Datagram Protocol) is introduced and the above IoT MO data transfer test is repeated. In addition, a non-IP data delivery mechanism over NAS (non-access stratum) signaling is introduced. Again, the IoT MO data transfer test is repeated.

In a further test method the IoT MT (Mobile Termination) data transfer provided by the serving IoT network 2 under test is tested. Here, after deployment of the TCP transport protocol, an application server, ie a server of the IoT platform 6 located in the home network, is started to send IoT data to the respective test probes 3 . It is verified whether the IoT data is completely received by each test probe 3 . Further steps of this test method, including deployment of the UDP transport protocol and deployment of non-IP data delivery mechanisms over NAS signaling, correspond to those described above with respect to the MO data transfer test method.

In a further test method, MO SMS transmissions through the serving IoT network 2 under test are tested. Here each test probe 3 is initiated to send an SMS to the partner test probe 3 of the local unit 4 in the home network (HPMN). It is then verified whether the SMS was received correctly at the partner test probe 3 . This test is repeated multiple times, and the multiple test results are aggregated for further statistical evaluation.

In a further test method MT SMS delivery over the serving IoT network 2 under test is tested. Here the partner test probe 3 in the home network (HPMN) is initiated to send an SMS to the test probe 3 in the serving IoT network 2. It is verified whether the SMS sent by the partner test probes 3' is correctly delivered to the respective test probes 3 in the serving IoT network 2. Again, this test is repeated multiple times and multiple test results are aggregated.

A further embodiment of a test system 15 for mobile IoT networks is described with respect to FIG. Components and functions corresponding to those described above with respect to FIG. 1 are denoted by the same reference numerals and will not be described in detail again. Not shown in FIG. 2 are the central test units, test clients and SIM multiplexers that may be present in the test system 15 in addition to the local units.

In the test system 15 a local unit 16 containing the test probes 3 is embodied as an S1 core unit connected to the IoT network 2 via an S1 interface 17 . A communication line 18 via the S1 interface 17 is implemented as an emulated eNodeB (Evolved NodeB). Details regarding the implementation of the S1 interface and its protocol can be found in 3GPP TS 36.413 V.3. 15.5.0, Release 15, March 2019: "Evolved Universal Terrestrial Access Network (E-UTRAN); S1 Application Protocol (S1AP)".

FIG. 3 shows another embodiment of test system 20 . Components and functions corresponding to those already described with respect to Figures 1 and 2 have the same reference numerals and will not be described in detail again.

The test system 20 provides testing for connectivity and services to roaming mobile IoT devices. Here, communication over lines 9, 10 and 11 takes place across a boundary 21 between a Home Public Mobile Network (HPMN) and a Visiting Public Mobile Network (VPMN). will be For this purpose, in the communication line 11, in addition to the HPMN SCEF module, an interworking SCEF (IWK-SCEF) module is also arranged in the VPMN.

This roaming scheme is possible with the radio interface 5 shown again in FIG. 3 and the S1 interface 17 according to FIG. 2 (not shown in FIG. 3).

Figure 4 shows components of a further embodiment of a test system for mobile IoT networks, including details regarding test data transfer between different test probes 3, 3'. Such different test probes 3, 3' may be by different e.g. home/visited public mobile networks. Components and functions corresponding to those described above with respect to Figures 1 to 3 have the same reference numerals and will not be described in detail again.

A first of the test probes, test probe 3 , is connected to IoT platform 6 via test communication line 22 , which may include wireless interface 5 or S1 interface 17 . The test probe 3 includes an MQTT/MQTT-SN client 23 that communicates via a communication line 22 with an MQTT/MQTT-SN server/broker 24 of the IoT platform 6 under test.

Another test probe, test probe 3', communicates with the IoT platform 6 under test via another test communication line 25, which in FIG. For this purpose the further test probe 3' also contains an MQTT/MQTT-SN client 23'.

Alternatively or in addition to MQTT/MQTT-SN, at least one of the following further protocols and/or one of the following device specific interfaces may be used: oneM2M, Hypercat, CoAP, RTSP, JSON. , XML.

The use of specific protocols/interfaces depends on each IoT device and/or each application. As an example, CoAP is suitable for constrained networks with low bandwidth and low power.

The test probe 3, 3' may be part of a mobile device, i.e. part of a vehicle (e.g. bicycle or car). In this case, each time a respective test probe 3 enters a new tracking area within the MIoT network 2, the test device may initiate updating of the tracking area. After such a tracking area update, the data assigned to the PSM and/or the data assigned to the eDRX functionality may be renegotiated and/or overwritten upon initiation of the test system.

By using in particular one of the test methods described above, relevant IoT data transmitted by the test probe 3 and stored in the platform under test 6 can be read by the test probe 3'. . The read data is compared with the original data that was sent. A corresponding test verdict can be assigned.

The test probes 3, 3' may be located within the same home network. Alternatively, the test probes 3, 3' may be placed in separate networks, as shown in FIG. For example, as shown in Figure 4, a test probe 3 may be located in the visited public network VPMN and a further test probe 3' located in the home public mobile network HPMN. With such a configuration, the IoT application platform testing under IoT device roaming described above can be performed globally.

1, 15, 20 active test system 2 IoT network 3, 3' test probe 4 local unit 5 radio interface 5a central test unit 5b test client 6 AS/IoT platform 7, 8 Internet network 9 NB - IoT test communication line 10 LTE - M Test Communication Lines 11, 18 Communication Lines 12 SIM Multiplexer 16 Local Unit 17 S1 Interface 21 Boundary 22, 25 Test Communication Lines 23, 23' MQTT/MQTT-SN Client 24 MQTT/MQTT-SN Server/Broker

Claims (15)

In an active test system (1; 15; 20) for a mobile IoT network (2) providing connectivity and services to mobile IoT (MIoT) devices of Low Power Wide Area (LPWA) technology,
- the test system is designed to test the quality of MIoT services of the serving MIoT network under test and to test the availability of MIoT services;
-- at least one test probe (3; 3, 3') connected to said MIoT network (2) via an LTE-Uu interface (5); and/or
-- at least one test probe (3; 3, 3') connected to said MIoT network (2) via an S1 interface (17);
- a central test unit (5a) connected to said at least one test probe (3; 3, 3') via a wireless backhaul network or a fixed IP network (7);
- a SIM multiplexer (12) for transferring SIM data to said at least one test probe (3; 3, 3') of the test field;
- The test system configures and initiates test probes on EPS attachments in the serving MIoT network under test to verify completion of test procedures, monitor and record all test events, and follow the test schedule. An active test system characterized by repeating the above test procedure. 2. A test system according to claim 1, adapted to exchange signaling messages and to transmit either IP data, non-IP data or SMS to/from the MIoT network (2) under test. Said at least one test probe (3; 3, 3') is a serving network, i.e. a home IoT network under test (2, HPMN) for national MIoT service testing or an international MIoT roaming service testing 3. Testing system according to claim 1 or 2, characterized in that it is arranged to be located in a visited MIoT network (2, VPMN) to be tested. Different test connections and communication across different MIoT network (2) components via MME, S-GW, P-GW, SCEF, IWK-SCEF, SCS, AS and across roaming interfaces S6a, S8, SGd, T7 4. The test system of claim 3, configured to test a MIoT serving network under test on the path. communicate with the MIoT platform under test (6) via MQTT/MQTT-SN messages;
ensuring the availability and connectivity of the MIoT platform (6) through the underlying MIoT network, end-to-end data transfer and data integrity between the MIoT platform (6) and the mobile IoT device; 5. A test system according to any one of claims 1 to 4, adapted for verification. A test method for testing the MIoT service quality of the serving MIoT network under test using a test system according to any one of claims 1-5. testing the availability of MIoT services;
- setting up and starting a test probe to an EPS attach in the serving MIoT network under test;
- verifying completion of said attach procedure;
- monitoring and recording all test events;
- repeating the above test steps according to a test schedule;
7. The test method of claim 6, comprising: testing connectivity retention of IoT networks,
- initiating the test probe to ping a server in the serving MIoT network under test; - verifying completion of the ping procedure;
- monitoring and recording all test events;
- repeating the above test steps according to a test schedule;
The test method according to any one of claims 6 to 7, comprising testing power saving features managed by the MIoT serving network under test;
- enabling PSM (power saving mode), thereby setting the values of the T3324 active timer and the extended T3412 timer in the test probe;
- initiating an EPS attach of the test probe in the serving MIoT network;
- verifying completion of said attach procedure;
- verifying whether the values of the timers are acceptable by the serving MIoT network by comparing them with the values requested by the test probe;
- verifying if an extended periodic TAU (Tracking Area Update) procedure is acceptable;
- monitoring and recording all test events;
- repeating the above test steps according to a test schedule;
The test method according to any one of claims 6 to 8, comprising testing eDRX functionality managed by the serving IoT network under test;
- enabling eDRX (Extended Discontinuous Reception), thereby setting eDRX cycle length and paging time window (PTW) values in said test probe;
- initiating an EPS attach of the test probe at the serving IoT network under test;
- verifying completion of said attach procedure;
- verifying whether the values of the eDRX cycle length and the PTW are acceptable to the serving IoT network by comparing them with the values requested by the test probe;
- monitoring and recording all test events;
- repeating the above test steps according to a test schedule;
The test method according to any one of claims 6 to 9, comprising testing the connectivity retention of an IoT network;
- verifying whether the test probe is requested from the serving IoT network for detach after EPS attach or after Mobile Originated (MO) or Mobile Terminated (MT) data transfer;
- repeating this verification step multiple times;
- aggregating the multiple test results to indicate a default EPS bearer context cutoff ratio;
The test method according to any one of claims 6 to 10, comprising testing IoT mobile originated (MO) data transfers provided by the serving IoT network under test;
- deploying the TCP transport protocol;
- initiating a mobile-originated IoT data transfer from the test probe to an application server located in the home network (HPMN);
- verifying whether the IoT data is correctly received by the application server;
- repeating this verification step multiple times;
- aggregating the multiple test results to indicate the default EPS bearer context cutoff ratio;
- deploying the UDP transport protocol;
- repeating the IoT MO data transfer test;
- deploying non-IP data delivery mechanisms over NAS signaling;
- repeating the IoT MO data transfer test;
The test method according to any one of claims 6 to 11, comprising testing an IoT mobile terminated (MT) data transfer provided by the serving IoT network under test;
- deploying the TCP transport protocol;
- starting an application server located in said home network (HPMN) to send IoT data to said test probe;
- verifying whether the IoT data is completely received by the test probe;
- repeating this verification step multiple times;
- aggregating the multiple test results to indicate the default EPS bearer context cutoff ratio;
- deploying the UDP transport protocol;
- repeating the IoT MT data transfer test;
- deploying non-IP data delivery mechanisms over NAS signaling;
- repeating the IoT MT data transfer test;
The test method according to any one of claims 6 to 12, comprising testing MO SMS transmission over the serving IoT network under test;
- initiating the test probe to send an SMS to a partner test probe in the home network (HPMN);
- verifying whether the SMS is delivered correctly to the partner testing probe;
- repeating the test multiple times;
- aggregating the multiple test results;
The test method according to any one of claims 6 to 13, comprising testing MT SMS delivery over the serving IoT network under test;
- initiating a partner testing probe in the home network to send an SMS to the testing probe in the serving IoT network;
- verifying whether the SMS sent by the partner test probe is correctly delivered to the test probe in the serving IoT network;
- repeating the test multiple times;
- aggregating the multiple test results;
The test method according to any one of claims 6 to 14, comprising

Images