WO2025145337A1

WO2025145337A1 - Devices, methods, and medium for communication

Info

Publication number: WO2025145337A1
Application number: PCT/CN2024/070397
Authority: WO
Inventors: Wei Chen; Peng Guan; Zhen He; Rao SHI; Gang Wang
Original assignee: NEC Corp
Current assignee: NEC Corp
Priority date: 2024-01-03
Filing date: 2024-01-03
Publication date: 2025-07-10
Anticipated expiration: 2026-07-03

Abstract

Example embodiments of the present disclosure relate to devices, methods, and computer storage medium for communication. An LMF may transmit, to a communication device deployed with a model for positioning, a first message indicating to the communication device to provide requested information for model monitoring if an accuracy of a location determined based on at least one model output is lower than a threshold. The LMF may receive a second message comprising the requested information, and further determine a reliability of the model based on the second message. As such, a process of model monitoring may be performed in case a model accuracy is lower than a threshold. In this way, a reliability of the model for positioning may be determined, and thus the positioning may be guaranteed.

Description

DEVICES, METHODS, AND MEDIUM FOR COMMUNICATION

FIELD

Example embodiments of the present disclosure generally relate to the field of communication techniques and in particular, to devices, methods, and a computer readable medium for communication.

BACKGROUND

Supporting various positioning methods to provide reliable, timely and accurate user equipment (UE) location is one of the key features of the third generation partnership project (3GPP) standard. It has been agreed to investigate the potential for artificial intelligence (AI) /machine learning (ML) in air interface to improve comprehensive performance in 5G-adcanced. AI/ML based mechanism to improve the positioning accuracy is one of the use cases to apply AI/ML in air interface.

An AI/ML model may be deployed at a terminal device (such as a UE) , a network device (such as one or more gNBs or transmission reception points (TRPs) ) , or a core network entity (such as a location management function (LMF) ) . The AI/ML model may be used for positioning, e.g. determining a positon (or location) of a UE, however, an accuracy of the positioning may be varied. Thus, a model monitoring is needed to ensure an accuracy of the positioning.

SUMMARY

In general, example embodiments of the present disclosure provide devices, methods, and a computer storage medium for communication.

In a first aspect, there is provided an LMF. The LMF comprises at least one processor configured to cause the LMF at least to: in accordance with a determination that an accuracy of a location determined based on at least one model output is lower than a threshold, transmit, to a communication device deployed with a model for positioning, a first message comprising assistance information indicating to the communication device to provide requested information for model monitoring; receive, from the communication device, a second message comprising the requested information; and determine a reliability of the model based on the second message.

In a second aspect, there is provided a communication device. The communication device comprises at least one processor configured to cause the communication device at least to: receive, from an LMF, a first message comprising assistance information indicating to the communication device to provide requested information for model monitoring; determine the requested information based on the first message; and transmit, to the LMF, a second message comprising the requested information, wherein the communication device comprises a terminal device or a network device deployed with a model for positioning.

In a third aspect, there is provided a terminal device. The terminal device comprises at least one processor configured to cause the terminal device at least to: in accordance with a determination that an accuracy of a location determined based on at least one model output is lower than a threshold, transmit, to an LMF, a first message comprising an indication of at least one network device, wherein the first message is used for requesting at least one geographical coordinate of the at least one network device, wherein there is at least one model deployed at the terminal device; receive, from the LMF, a second message comprising the at least one geographical coordinate of the at least one network device; determine a difference between a model output and a ground truth, wherein the ground truth is determined based on the second message; and transmit, to the LMF, a report comprising the difference.

In a fourth aspect, there is provided an LMF. The LMF comprises at least one processor configured to cause the LMF at least to: receive, from a terminal device deployed with at least one model, a first message comprising an indication of at least one network device, wherein the first message is used for requesting at least one geographical coordinate of the at least one network device; transmit, to the terminal device, a second message comprising the at least one geographical coordinate of the at least one network device; and receive, from the terminal device, a report comprising a difference between a model output and a ground truth, wherein the ground truth is determined based on the second message.

In a fifth aspect, there is provided a method of communication. The method comprises: in accordance with a determination that an accuracy of a location determined based on at least one model output is lower than a threshold, transmitting, at an LMF to a communication device deployed with a model for positioning, a first message comprising assistance information indicating to the communication device to provide requested information for model monitoring; receiving, from the communication device, a second message comprising the requested information; and determining a reliability of the model based on the second message.

In a sixth aspect, there is provided a method of communication. The method comprises: receiving, at a communication device from an LMF, a first message comprising assistance information indicating to the communication device to provide requested information for model monitoring; determine the requested information based on the first message; and transmit, to the LMF, a second message comprising the requested information, wherein the communication device comprises a terminal device or a network device deployed with a model for positioning.

In a seventh aspect, there is provided a method of communication. The method comprises: in accordance with a determination that an accuracy of a location determined based on at least one model output is lower than a threshold, transmitting, at a terminal device to an LMF, a first message comprising an indication of at least one network device, wherein the first message is used for requesting at least one geographical coordinate of the at least one network device, wherein there is at least one model deployed at the terminal device; receiving, from the LMF, a second message comprising the at least one geographical coordinate of the at least one network device; determining a difference between a model output and a ground truth, wherein the ground truth is determined based on the second message; and transmitting, to the LMF, a report comprising the difference.

In an eighth aspect, there is provided a method of communication. The method comprises: receiving, at an LMF from a terminal device deployed with at least one model, a first message comprising an indication of at least one network device, wherein the first message is used for requesting at least one geographical coordinate of the at least one network device; transmitting, to the terminal device, a second message comprising the at least one geographical coordinate of the at least one network device; and receiving, from the terminal device, a report comprising a difference between a model output and a ground truth, wherein the ground truth is determined based on the second message.

In a ninth aspect, there is provided a computer readable medium having instructions stored thereon, the instructions, when executed on at least one processor, causing the at least one processor to carry out the method according to any one of the fifth to eighth aspects above.

It is to be understood that the summary section is not intended to identify key or essential features of embodiments of the present disclosure, nor is it intended to be used to limit the scope of the present disclosure. Other features of the present disclosure will become easily comprehensible through the following description.

BRIEF DESCRIPTION OF THE DRAWINGS

Through the more detailed description of some example embodiments of the present disclosure in the accompanying drawings, the above and other objects, features and advantages of the present disclosure will become more apparent, wherein:

FIG. 1 illustrates an example positioning system by wireless network in which some embodiments of the present disclosure can be implemented;

FIG. 2 illustrates a signalling chart illustrating communication process in accordance with some example embodiments of the present disclosure;

FIG. 3A illustrates an example schematic for determining a monitoring metric at an LMF in accordance with some embodiments of the present disclosure;

FIGS. 3B-3E illustrate signalling charts of processes between the network device and the LMF in accordance with some embodiments of the present disclosure;

FIG. 4A illustrates an example schematic for determining a monitoring metric at an LMF in accordance with some embodiments of the present disclosure;

FIGS. 4B-4D illustrate signalling charts of processes between the terminal device and the LMF in accordance with some embodiments of the present disclosure;

FIG. 5 illustrates a signalling chart illustrating communication process in accordance with some example embodiments of the present disclosure;

FIG. 6A illustrates an example schematic for determining a monitoring metric at a terminal device in accordance with some embodiments of the present disclosure;

FIG. 6B illustrates a signalling chart of a process between the terminal device and the LMF in accordance with some embodiments of the present disclosure;

FIG. 7 illustrates a flowchart of an example method implemented at an LMF in accordance with some embodiments of the present disclosure;

FIG. 8 illustrates a flowchart of an example method implemented at a communication device in accordance with some embodiments of the present disclosure;

FIG. 9 illustrates a flowchart of an example method implemented at a terminal device in accordance with some embodiments of the present disclosure;

FIG. 10 illustrates a flowchart of an example method implemented at an LMF in accordance with some embodiments of the present disclosure; and

FIG. 11 illustrates a simplified block diagram of a device that is suitable for implementing embodiments of the present disclosure.

Throughout the drawings, the same or similar reference numerals represent the same or similar element.

DETAILED DESCRIPTION

Principle of the present disclosure will now be described with reference to some example embodiments. It is to be understood that these embodiments are described only for the purpose of illustration and help those skilled in the art to understand and implement the present disclosure, without suggesting any limitation as to the scope of the disclosure. Embodiments described herein can be implemented in various manners other than the ones described below.

In the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skills in the art to which this disclosure belongs.

References in the present disclosure to “one embodiment, ” “an embodiment, ” “an example embodiment, ” and the like indicate that the embodiment described may include a particular feature, structure, or characteristic, but it is not necessary that every embodiment includes the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.

It shall be understood that although the terms “first” and “second” etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element could be termed a second element, and similarly, a second element could be termed a first element, without departing from the scope of example embodiments. As used herein, the term “and/or” includes any and all combinations of one or more of the listed terms.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments. As used herein, the singular forms “a” , “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” , “comprising” , “has” , “having” , “includes” and/or “including” , when used herein, specify the presence of stated features, elements, and/or components etc., but do not preclude the presence or addition of one or more other features, elements, components and/or combinations thereof.

In some examples, values, procedures, or apparatus are referred to as “best, ” “lowest, ” “highest, ” “minimum, ” “maximum, ” or the like. It will be appreciated that such descriptions are intended to indicate that a selection among many used functional alternatives can be made, and such selections need not be better, smaller, higher, or otherwise preferable to other selections.

As used herein, the term “communication network” refers to a network following any suitable communication standards, such as New Radio (NR) , Long Term Evolution (LTE) , LTE-Advanced (LTE-A) , Wideband Code Division Multiple Access (WCDMA) , High-Speed Packet Access (HSPA) , Narrow Band Internet of Things (NB-IoT) and so on. Furthermore, the communications between a terminal device and a network device in the communication network may be performed according to any suitable generation communication protocols, including, but not limited to, the first generation (1G) , the second generation (2G) , 2.5G, 2.75G, the third generation (3G) , the fourth generation (4G) , 4.5G, the fifth generation (5G) , 5.5G, 5G-Advanced networks, or the sixth generation (6G) communication protocols, and/or any other protocols either currently known or to be developed in the future. Embodiments of the present disclosure may be applied in various communication systems. Given the rapid development in communications, there will of course also be future type communication technologies and systems with which the present disclosure may be embodied. It should not be seen as limiting the scope of the present disclosure to only the aforementioned system.

As used herein, the term “terminal device” refers to any device having wireless or wired communication capabilities. Examples of terminal device include, but not limited to, user equipment (UE) , personal computers, desktops, mobile phones, cellular phones, smart phones, personal digital assistants (PDAs) , portable computers, tablets, wearable devices, internet of things (IoT) devices, Ultra-reliable and Low Latency Communications (URLLC) devices, Internet of Everything (IoE) devices, machine type communication (MTC) devices, device on vehicle for V2X communication where X means pedestrian, vehicle, or infrastructure/network, devices for Integrated Access and Backhaul (IAB) , Space borne vehicles or Air borne vehicles in Non-terrestrial networks (NTN) including Satellites and High Altitude Platforms (HAPs) encompassing Unmanned Aircraft Systems (UAS) , eXtended Reality (XR) devices including different types of realities such as Augmented Reality (AR) , Mixed Reality (MR) and Virtual Reality (VR) , the unmanned aerial vehicle (UAV) commonly known as a drone which is an aircraft without any human pilot, devices on high speed train (HST) , or image capture devices such as digital cameras, sensors, gaming devices, music storage and playback appliances, or Internet appliances enabling wireless or wired Internet access and browsing and the like. The ‘terminal device’ can further has ‘multicast/broadcast’ feature, to support public safety and mission critical, V2X applications, transparent IPv4/IPv6 multicast delivery, IPTV, smart TV, radio services, software delivery over wireless, group communications and IoT applications. It may also be incorporated one or multiple Subscriber Identity Module (SIM) as known as Multi-SIM. The term “terminal device” can be used interchangeably with a UE, a mobile station, a subscriber station, a mobile terminal, a user terminal or a wireless device.

As used herein, the term “network device” refers to a device which is capable of providing or hosting a cell or coverage where terminal devices can communicate. Examples of a network device include, but not limited to, a satellite, an unmanned aerial systems (UAS) platform, a Node B (NodeB or NB) , an evolved NodeB (eNodeB or eNB) , a next generation NodeB (gNB) , a transmission reception point (TRP) , a remote radio unit (RRU) , a radio head (RH) , a remote radio head (RRH) , an IAB node, a low power node such as a femto node, a pico node, a reconfigurable intelligent surface (RIS) , and the like.

In one embodiment, the terminal device may be connected with a first network device and a second network device. One of the first network device and the second network device may be a master node and the other one may be a secondary node. The first network device and the second network device may use different radio access technologies (RATs) . In one embodiment, the first network device may be a first RAT device and the second network device may be a second RAT device. In one embodiment, the first RAT device is eNB and the second RAT device is gNB. Information related with different RATs may be transmitted to the terminal device from at least one of the first network device and the second network device. In one embodiment, first information may be transmitted to the terminal device from the first network device and second information may be transmitted to the terminal device from the second network device directly or via the first network device. In one embodiment, information related with configuration for the terminal device configured by the second network device may be transmitted from the second network device via the first network device. Information related with reconfiguration for the terminal device configured by the second network device may be transmitted to the terminal device from the second network device directly or via the first network device.

Communications discussed herein may conform to any suitable standards including, but not limited to, New Radio Access (NR) , Long Term Evolution (LTE) , LTE-Evolution, LTE-Advanced (LTE-A) , Wideband Code Division Multiple Access (WCDMA) , Code Division Multiple Access (CDMA) , cdma2000, and Global System for Mobile Communications (GSM) and the like. Furthermore, the communications may be performed according to any generation communication protocols either currently known or to be developed in the future. Examples of the communication protocols include, but not limited to, the first generation (1G) , the second generation (2G) , 2.5G, 2.85G, the third generation (3G) , the fourth generation (4G) , 4.5G, the fifth generation (5G) , and the sixth (6G) communication protocols. The techniques described herein may be used for the wireless networks and radio technologies mentioned above as well as other wireless networks and radio technologies. The embodiments of the present disclosure may be performed according to any generation communication protocols either currently known or to be developed in the future. Examples of the communication protocols include, but not limited to, the first generation (1G) , the second generation (2G) , 2.5G, 2.75G, the third generation (3G) , the fourth generation (4G) , 4.5G, the fifth generation (5G) communication protocols, 5.5G, 5G-Advanced networks, or the sixth generation (6G) networks.

The terminal device or the network device may work on several frequency ranges, e.g. FR1 (410 MHz –7125 MHz) , FR2 (24.25GHz to 71GHz) , frequency band larger than 200GHz as well as Tera Hertz (THz) . It can further work on licensed/unlicensed/shared spectrum. The terminal device may have more than one connection with the network device under Multi-Radio Dual Connectivity (MR-DC) application scenario. The terminal device or the network device can work on full duplex, flexible duplex and cross division duplex modes.

The embodiments of the present disclosure may be performed in test equipment, e.g., signal generator, signal analyzer, spectrum analyzer, network analyzer, test terminal device, test network device, or channel emulator.

The term “circuitry” used herein may refer to hardware circuits and/or combinations of hardware circuits and software. For example, the circuitry may be a combination of analog and/or digital hardware circuits with software/firmware. As a further example, the circuitry may be any portions of hardware processors with software including digital signal processor (s) , software, and memory (ies) that work together to cause an apparatus, such as a terminal device or a network device, to perform various functions. In a still further example, the circuitry may be hardware circuits and or processors, such as a microprocessor or a portion of a microprocessor, that requires software/firmware for operation, but the software may not be present when it is not needed for operation. As used herein, the term circuitry also covers an implementation of merely a hardware circuit or processor (s) or a portion of a hardware circuit or processor (s) and its (or their) accompanying software and/or firmware.

As used herein, the singular forms “a” , “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. The term “includes” and its variants are to be read as open terms that mean “includes, but is not limited to. ” The term “based on” is to be read as “based at least in part on. ” The term “one embodiment” and “an embodiment” are to be read as “at least one embodiment. ” The term “another embodiment” is to be read as “at least one other embodiment. ” The terms “first, ” “second, ” and the like may refer to different or same objects. Other definitions, explicit and implicit, may be included below.

The terminal device or the network device may have AI or ML capability. It generally includes a model which has been trained from numerous collected data for a specific function, and can be used to predict some information.

As used herein, a model may be equivalent to at least one of the following: an AI/ML model, an ML model, an AI model, a data-driven, a data processing model, an algorithm, a functionality, a procedure, a process, an entity, a function, a feature, a feature group, a model identifier (ID) , an ID, a functionality ID, a configuration ID, a scenario ID, a site ID, or a dataset ID. As a result, the above terms may be used interchangeably.

In some embodiments, the model may be represented by or associated with a channel, a resource, a resource set, a reference signal (RS) resource, an RS resource set, an RS port, a set of RS ports, an RS port ID, or a set of RS port IDs.

In some embodiments, the model may comprise a set of weights values that may be learned during training, e.g. for a specific architecture or configuration, where a set of weights values may also be called a parameter set.

In some embodiments, the model may be used to predict a target cell, or measurements of a set of beams of a set of candidate cells in future based on at least historical measurements (e.g., layer 1 (L1) -reference signal received power (RSRP) , L1-signal to interference plus noise ratio (SINR) ) of a set of beams of a set of candidate cells.

In some embodiments, an input of the AI/ML model (i.e., AI input) may refer to the input of a model and indicate data inputted into the model, which may be equivalent to data.

In some embodiments, an output of AI/ML model (i.e., AI output) may refers to the output of a model and indicate result (s) outputted by the model, which is equivalent to label/data.

In some embodiments, “ground truth” , “ground truth label” , “ground truth label of data” , “input label” , “input data” and “data” can be used interchangeably.

In some embodiments, a ground truth label of data (or ground-truth label) for monitoring or training the ML model (i.e., AI output) may refers to the authoritative, accepted data, or true answer or outcome for AI/ML model.

In some embodiments, the ground truth can be interpreted as actual/factual (i.e. actual/factual measured) data/values/results/collections/parameters, which can be used as reference, compared to prediction or inference.

AI/ML techniques play a significant role in enhancing the accuracy and reliability of positioning, which is particularly useful in indoor environments where global position system (GPS) signals might be weak or unavailable. To guarantee the ongoing fulfillment of performance requirements, it is imperative to incorporate model monitoring as a crucial step when deploying the AI/ML approach for positioning.

AI/ML based positioning has been introduced to enhance the positioning accuracy, especially for indoor scenario with low line of sight (LOS) condition probability. For the study of benefit (s) and potential specification impact for AI/ML based positioning accuracy enhancement, one-sided model whose inference is performed entirely at the UE or at the network is prioritized in Rel-18 study item (SI) .

For AI/ML-assisted positioning, a “single-TRP construction” and a “multi-TRP construction” are being discussed. Single-TRP construction: the input of the ML model is the channel measurement between the target UE and a single TRP, and the output of the ML model is for the same pair of UE and TRP. Multi-TRP construction: the input of the ML model contains N sets of channel measurements between the target UE and N (N>1) TRPs, and the output of the ML model contains N sets of values, one for each of the N TRPs.

Release 18 has explored two approaches for implementing model monitoring. The first involves model monitoring without the need for a ground truth label (or its approximation) , while the second requires the availability of a provided ground truth label (or its approximation) . In the context current definition, the ground truth consistently corresponds to the same type of model output. This implies that monitoring always involves contrasting intermediate measurements but lacks a mechanism for comparing the final result at least for AI/ML assisted positioning.

Embodiments of the present disclosure provide a solution of communication. In the solution, an LMF transmits a first message to a communication device deployed with a model for positioning, where the first message may indicate to the communication device to provide requested information for model monitoring. The LMF may further determine a reliability of the model based on the request information included in a second message from the communication device. As such, a model monitoring may be performed by the LMF and accordingly the model reliability may be guaranteed. Principles and implementations of the present disclosure will be described in detail below with reference to the figures.

FIG. 1 illustrates an example communication system 100 in which some embodiments of the present disclosure can be implemented. The communication system 100 may also be called as a network environment, a network system, a communication environment, a communication network, or the like, the present disclosure does not limit this aspect. The communication system 100 includes a terminal device 110, multiple network devices 120-1 to 120-N, and an LMF 130. It should be appreciated that the LMF 130 may be located in the access network or core network.

The multiple network devices 120-1 to 120-N (N is a positive integer, e.g. N=3) may be separately or collectively be referred to as a network device 120, which may be a gNB or a TRP.

In the system 100, the network device 120 can communicate/transmit data and control information to the terminal device 110, and the terminal device 110 can also communicate/transmit data and control information to the network device 120. A link from the network device 120 to the terminal device 110 is referred to as a downlink (DL) , while a link from the terminal device 110 to the network device 120 is referred to as an uplink (UL) . DL may comprise one or more logical channels, including but not limited to a Physical Downlink Control Channel (PDCCH) and a Physical Downlink Shared Channel (PDSCH) . UL may comprise one or more logical channels, including but not limited to a Physical Uplink Control Channel (PUCCH) and a Physical Uplink Shared Channel (PUSCH) . As used herein, the term “channel” may refer to a carrier or a part of a carrier consisting of a contiguous set of resource blocks (RBs) on which a channel access procedure is performed in shared spectrum.

Communications in the system 100, between the network device 120 and the terminal device 110 for example, may be implemented according to any proper communication protocol (s) , comprising, but not limited to, cellular communication protocols of the first generation (1G) , the second generation (2G) , the third generation (3G) , the fourth generation (4G) , the fifth generation (5G) and the sixth generation (6G) and on the like, wireless local network communication protocols such as Institute for Electrical and Electronics Engineers (IEEE) 802.11 and the like, and/or any other protocols currently known or to be developed in the future. Moreover, the communication may utilize any proper wireless communication technology, comprising but not limited to: Code Divided Multiple Address (CDMA) , Frequency Divided Multiple Address (FDMA) , Time Divided Multiple Address (TDMA) , Frequency Divided Duplexer (FDD) , Time Divided Duplexer (TDD) , Multiple-Input Multiple-Output (MIMO) , Orthogonal Frequency Divided Multiple Access (OFDMA) and/or any other technologies currently known or to be developed in the future.

In the system 100, the terminal device 110 can communicate with the LMF 130 according to any proper communication protocol, such as an LTE positioning protocol (LPP) . It is to be understood that other protocol may also be applied and will not be listed herein.

Embodiments of the present disclosure can be applied to any suitable scenarios. For example, embodiments of the present disclosure can be implemented at reduced capability NR devices. Alternatively, embodiments of the present disclosure can be implemented in one of the followings: NR multiple-input and multiple-output (MIMO) , NR sidelink enhancements, NR systems with frequency above 52.6GHz, an extending NR operation up to 71GHz, narrow band-Internet of Thing (NB-IOT) /enhanced Machine Type Communication (eMTC) over non-terrestrial networks (NTN) , NTN, UE power saving enhancements, NR coverage enhancement, NB-IoT and LTE-MTC, Integrated Access and Backhaul (IAB) , NR Multicast and Broadcast Services, or enhancements on Multi-Radio Dual-Connectivity.

It is to be understood that the numbers of devices (i.e., the terminal devices 110 and the network device 120) and their connection relationships and types shown in FIG. 1 are only for the purpose of illustration without suggesting any limitation. The system 100 may include any suitable numbers of devices adapted for implementing embodiments of the present disclosure.

It has been agreed to study and provide inputs on benefit (s) and potential specification impact at least for the following cases of AI/ML based positioning accuracy enhancement:

● Case 1: UE-based positioning with UE-side model, direct AI/ML or AI/ML assisted positioning;

● Case 2a: UE-assisted/LMF-based positioning with UE-side model, AI/ML assisted positioning;

● Case 2b: UE-assisted/LMF-based positioning with LMF-side model, direct AI/ML positioning;

● Case 3a: NG-RAN node assisted positioning with gNB-side model, AI/ML assisted positioning;

● Case 3b: NG-RAN node assisted positioning with LMF-side model, direct AI/ML positioning.

In case an AI/ML model is deployed for positioning, a device responsible for model monitoring may derive a monitoring metric, e.g. based on a difference between a real location and an inferred location from measurements output from the AI/ML model. For example, Table 1 below provides a comparison for different entities for model monitoring.

Table 1

Reference is further made to FIG. 2, which illustrates a signalling chart illustrating communication process 200 in accordance with some example embodiments of the present disclosure. The process 200 may involve a communication device 202 deployed with an AI/ML model and the LMF 130. With reference to FIG. 1, the communication device 202 may be the terminal device 110 or the network device 120.

In the process 200, the LMF 130 determines that an accuracy of a location is lower than a threshold at 210. In some implementations, the LMF 130 may determine an inferred location, e.g. based on output of the AI/ML model at the communication device 202. In some implementations, the LMF 130 may determine an accurate location, e.g. based on a GPS system.

In some implementations, the LMF 130 may determine (or calculate) a difference between the inferred location and the accurate location, and further determine whether the difference is lower than a predefined threshold. In some examples, if the difference is lower than (or not larger than) the predefined threshold, the LMF 130 may determine that the accuracy of the inferred location is higher (or not less) than a threshold (or an accuracy threshold) . In some other examples, if the difference is high than (or nor lower than) the predefined threshold, the LMF 130 may determine that the accuracy of the inferred location is less (or not larger) than the threshold (or the accuracy threshold) .

In the process 200, if the accuracy is lower than the threshold, the LMF 130 transmits a first message to the communication device 202 at 220. In some implementations, the first message may include assistance information indicating to the communication device 202 to provide required information for model monitoring. In some example embodiments, the assistance information in the first message may be called as first information, while the required information may be called as second information.

In some implementations, the communication device 202 may be the network device 120, in this case, the first message may include a new radio positioning protocol A (NRPPa) message. Details of which may refer to FIGS. 3A-3E below.

In some implementations, the communication device 202 may be the terminal device 110, in this case, the first message may include an LPP message. Details of which may refer to FIGS. 4A-4D below.

In the process 200, the communication device 202 determines the required information at 230 based on the first message. In some implementations, the required information may be generated by using the deployed AI/ML model based on the assistance information (i.e. first information) .

In the process 200, the communication device 202 transmits a second message to the LMF 130 at 240, where the second message includes the required information (i.e. second information) .

In addition, the LMF 130 determines a reliability of the model at 250. In some implementations, the LMF 130 may determine whether there is any abnormal data among the second information in the second message. In some implementations, the LMF 130 may determine an availability of the AI/ML model at the communication device 202. For example, the LMF 130 may indicate to the communication device 202 to update the AI/ML model or use another model for positioning.

FIG. 3A illustrates an example schematic for determining 310 a monitoring metric at an LMF in accordance with some embodiments of the present disclosure. It is assumed that there is an AI/ML model deployed at each network device 120, that is, a scenario of an NG-RAN node assisted positioning with gNB-side model (case 3a discussed above) is assumed.

For example, an input of the AI/ML model may be any of channel impulse response (CIR) , power delay profile (PDP) , or delay of path (DP) , and an output of the AI/ML model may be a time of arrival (TOA) .

In some implementations, each network device 120 may report its output (such as TOA) to the LMF 130. In some implementations, there may be a calculator and a comparator at the LMF 130. In some examples, the calculator at the LMF 130 may determine an estimated UE position (i.e. an inferred location) based on the TOAs from multiple network devices 120. In some examples, the comparator at the LMF 130 may compare the estimated UE position with a real UE position (i.e. an accurate location) to obtain a monitoring metric 312.

It is to be understood that the AI/ML models deployed at different network devices 120 may be different or may be the same. For example, there may be N models for N network devices (such as TRPs or gNBs) . For example, there may be a same model for N network devices (such as TRPs or gNBs) .

It is to be understood that the output of the AI/ML model is not limited to TOA, for example, other timing estimation is also applied, and the present disclosure does not limit this aspect.

In some implementations, a positioning accuracy based on deployed AI/ML model (s) may be determined based on the monitoring metric 312. For example, if the positioning accuracy degrades to an unacceptable level, e.g. the monitoring metric 312 is larger (or not less) than a predefined threshold, the LMF 130 may further perform model monitoring for the individual network device, such as any of the processes 320-350.

FIG. 3B illustrates a signalling chart of a process 320 between the network device 120 and the LMF 130 in accordance with some embodiments of the present disclosure.

The network device 120 may refer to some or all of the N network devices. In some examples, the LMF 130 may select one or more from the N network devices, and the process 320 may relate to the one or more selected network devices 120. For example, the selection of the network devices 120 may depend on the LMF implementation.

The LMF 130 transmits an NRPPa message to the network device 120 at 321. In some implementations, the NRPPa message at 321 may request model input information.

In some implementations, the NRPPa message at 321 may be regarded as the first message discussed in FIG. 2. In some implementations, the NRPPa message at 321 may include first information which may be one or more of: a timestamp of a model output associated with the at least one model input, an identification of the model output, or a number of the at least one model input.

In some examples, the first information in the NRPPa message at 321 may assist the network device 120 to identify a specific model input.

In the process 320, the network device 120 transmits a further NRPPa message to the LMF 130 at 322. In some implementations, the further NRPPa message at 322 may include at least one input of the model requested by the NRPPa message at 321. For example, the further NRPPa message at 322 may include the specific model input corresponding to the first information in the NRPPa message at 321.

In addition or alternatively, the LMF 130 may identify (or determine or detect) any abnormal (or strange) input from the at least one input of the model provided by the network device 120.

In some examples, the LMF 130 may receive respective inputs from respective network devices 120. In some examples, the LMF 130 may combine all inputs to obtain a cluster of inputs, and in addition may determine which model is inappropriate, e.g. the inappropriate model may have a borderline input.

As such, the model monitoring may be performed without changing the environment, this solution may have a minimal impact for the network device 120, especially when a same AI/ML model is used for multiple network devices 120.

FIG. 3C illustrates a signalling chart of a process 330 between the network device 120 and the LMF 130 in accordance with some embodiments of the present disclosure.

The network device 120 may refer to some or all of the N network devices. In some examples, the LMF 130 may select one or more from the N network devices, and the process 330 may relate to the one or more selected network devices 120. For example, the selection of the network devices 120 may depend on the LMF implementation.

The LMF 130 transmits an NRPPa message to the network device 120 at 331. In some implementations, the NRPPa message at 331 may request a difference between a model output and a ground truth.

In some implementations, the NRPPa message at 331 may be regarded as the first message discussed in FIG. 2. In some implementations, the NRPPa message at 331 may include first information which may be one or more of: a location of a terminal device 110 being used as the ground truth, a timestamp of the model output, or an identification of the model output.

In some examples, the first information may include a location of the terminal device 110 or its approximate, for example, it may be used by the network device 120 to determine the ground truth. In some examples, the first information may be associated with a specific model output, such as the timestamp of the model output or the identification of the model output.

The network device 120 may determine the model output based in the first information, such as the timestamp of the model output or the identification of the model output. The network device 120 may then determine (or calculate) the difference between the model output and the ground truth, for example, an absolute value of the difference may be determined. For example, if the model output is TOA1, and the ground truth is TOA2, then the difference may be represented as |TOA1-TOA2|.

In the process 330, the network device 120 transmits a further NRPPa message to the LMF 130 at 332. In some implementations, the further NRPPa message at 332 may include the difference between the model output and the ground truth.

In addition or alternatively, the LMF 130 may determine whether an input corresponding to the model output is abnormal based on the difference. In some examples, the LMF 130 may determine whether to update the model or use a new model.

FIG. 3D illustrates a signalling chart of a process 340 between the network device 120 and the LMF 130 in accordance with some embodiments of the present disclosure.

The network device 120 may refer to some or all of the N network devices. In some examples, the LMF 130 may select one or more from the N network devices, and the process 340 may relate to the one or more selected network devices 120. For example, the selection of the network devices 120 may depend on the LMF implementation.

The LMF 130 transmits an NRPPa message to the network device 120 at 341. In some implementations, the NRPPa message at 341 may request at least one statistical parameter of a plurality of model inputs.

In some implementations, the NRPPa message at 341 may be regarded as the first message discussed in FIG. 2. In some implementations, the NRPPa message at 341 may include first information which may be one or more of: an identification of the plurality of model inputs, or a number of the plurality of model inputs.

In some examples, the first information in the NRPPa message at 341 may assist the network device 120 to identify the plurality of model inputs.

The network device 120 may determine the plurality of model inputs, e.g. based on the first information in the NRPPa message at 341. The network device 120 may then determine (or calculate) the at least one statistical parameter of the plurality of model inputs. In some examples, the at least one statistical parameter may include a correlation matrix of the plurality of model inputs.

In the process 340, the network device 120 transmits a further NRPPa message to the LMF 130 at 342. In some implementations, the further NRPPa message at 342 may include the at least one statistical parameter of the plurality of model inputs requested by the NRPPa message at 341. For example, the further NRPPa message at 342 may include the correlation matrix of the plurality of model inputs.

In addition or alternatively, the LMF 130 may identify (or determine or detect) any abnormal (or strange) input from the plurality of model inputs based on the at least one statistical parameter.

In some examples, the LMF 130 may combine the plurality of model inputs to obtain a cluster of inputs, and in addition may determine which model is inappropriate, e.g. the inappropriate model may have a borderline input.

As such, the model monitoring may be performed based on the at least one statistical parameter, this solution may have a minimal impact for the network device 120, especially when a same AI/ML model is used for multiple network devices 120.

FIG. 3E illustrates a signalling chart of a process 350 between the network device 120 and the LMF 130 in accordance with some embodiments of the present disclosure.

The network device 120 may refer to some or all of the N network devices. In some examples, the LMF 130 may select one or more from the N network devices, and the process 350 may relate to the one or more selected network devices 120. For example, the selection of the network devices 120 may depend on the LMF implementation.

The LMF 130 transmits an NRPPa message to the network device 120 at 351. In some implementations, the NRPPa message at 351 may indicate test data, and the NRPPa message at 351 may request a test result associated with the test data.

In some implementations, the NRPPa message at 351 may be regarded as the first message discussed in FIG. 2. In some implementations, the NRPPa message at 351 may include first information which may indicate the test data.

In some examples, the first information may include the test data. For example, the test data may be directly included in the first message. For example, the test data may include corner cases or borderline data within the deployment scenario. For example, the test data may be at least one pair of a model input and a model output. For example, the test data may be at least one model input.

In some other examples, the first information may include an ID of a data set which includes the test data. In some embodiments, multiple test data sets with multiple data set IDs may be predefined. For example, the multiple test data sets may be provided to the network device 120 together with the AI/ML model deployed at the network device 120. For example, signalling used for providing the multiple test data sets may be an NRPPa message or a message being not an NRPPa message. For example, the data set may specify the test data associated with the deployment scenario. For example, one or more following features may be different drops, different cluster parameters, different InF scenarios, or different synchronization errors.

The network device 120 may determine the test result based on the test data. In some implementations, if the test data includes a pair of a model input and a model output, the network device 120 may determine an inferred model output based on the model input indicated by the test data, and further calculate a difference between the model output indicated by the test data and the inferred model output. For example, the test result may include the difference. In some implementations, if the test data includes a model input (e.g. model output is not included) , the network device 120 may determine an inferred model output based on the model input indicated by the test data, for example, the test result may include the inferred model output.

In the process 350, the network device 120 transmits a further NRPPa message to the LMF 130 at 352. In some implementations, the further NRPPa message at 352 may include the test result which is corresponding to the test data indicated by the NRPPa message at 341.

In addition or alternatively, the LMF 130 may identify (or determine or detect) whether the model input is abnormal based on the test result. In some examples, the LMF 130 may determine whether the model is inappropriate, e.g. the inappropriate model may have a borderline input. As such, the model monitoring may be performed based on the test result, this solution may have a minimal impact for the network device 120, especially when a same AI/ML model is used for multiple network devices 120.

It is to be understood that any NRPPa message in any of processes 320-350 may be implemented as an existing NRPPa message or a newly defined NRPPa message, the present disclosure does not limit this aspect.

FIG. 4A illustrates an example schematic for determining 410 a monitoring metric at an LMF in accordance with some embodiments of the present disclosure. It is assumed that there is an AI/ML model deployed at the terminal device 110, that is, a scenario of an UE-assisted positioning or LMF-based positioning with UE-side model (case 2a discussed above) is assumed.

For example, an input of the AI/ML model may be any of CIR, PDP, or DP, and an output of the AI/ML model may be a TOA.

In some implementations, the terminal device 110 may report its output (such as TOA) to the LMF 130. For example, multiple TOAs associated with multiple network devices 120 may be transmitted to the LMF 130.

In some implementations, there may be one AI/ML model deployed at the terminal device 110, where the AI/ML model is used for multiple network devices 120 (such as TRPs or gNBs) . In some implementations, there may be multiple AI/ML models deployed at the terminal device 110, where a specific AI/ML model is used for a specific network devices 120 (such as TRP or gNB) .

In some implementations, there may be a calculator and a comparator at the LMF 130. In some examples, the calculator at the LMF 130 may determine an estimated UE position (i.e. an inferred location) based on the TOAs from the terminal device 110. In some examples, the comparator at the LMF 130 may compare the estimated UE position with a real UE position (i.e. an accurate location) to obtain a monitoring metric 412.

In some implementations, a positioning accuracy may be determined based on the monitoring metric 412. For example, if the positioning accuracy degrades to an unacceptable level, e.g. the monitoring metric 412 is larger (or not less) than a predefined threshold, the LMF 130 may further perform model monitoring, such as any of the processes 420-440.

FIG. 4B illustrates a signalling chart of a process 420 between the terminal device 110 and the LMF 130 in accordance with some embodiments of the present disclosure.

The LMF 130 transmits an LPP provide assistance data message to the terminal device 110 at 421. In some implementations, the LPP provide assistance data message at 421 may include multiple geographical coordinates of multiple network devices 120. In some examples, the multiple network devices 120 may refer to some or all of the N network devices. In some examples, the LMF 130 may select multiple from the N network devices, and the LPP provide assistance data message at 421 may relate to the multiple selected network devices 120. For example, the selection of the network devices 120 may depend on the LMF implementation. For example, the selection of the network devices may depend on the network devices involved in the positioning procedure.

In some implementations, the LPP provide assistance data message at 421 may be regarded as the first message discussed in FIG. 2. In some implementations, the LPP provide assistance data message at 421 may include first information which may be multiple geographical coordinates of the multiple network devices 120. In some examples, the first message may request a difference between a model output and a ground truth.

In the process 420, the terminal device 110 determines the difference between the model output and the ground truth based on the first message at 422. In some implementations, the terminal device 110 may determine the ground truth based on the multiple geographical coordinates of the multiple network devices 120, optional further based on other apparatus of the terminal device 110 (such as the sensor, GNSS, etc. ) . In some implementations, the terminal device 110 may determine the ground truth based on a deformation, which may e a straight-line distance between the terminal device 110 and a network device 120 divided by a speed of light.

The terminal device 110 may then determine (or calculate) the difference between the model output and the ground truth, for example, an absolute value of the difference may be determined. For example, if the model output is TOA1, and the ground truth is TOA2, then the difference may be represented as |TOA1-TOA2|.

In the process 420, the terminal device 110 transmits a further LPP provide assistance data message to the LMF 130 at 423. In some implementations, the further LPP provide assistance data message at 423 may include the difference between the model output and the ground truth.

In addition or alternatively, the LMF 130 may determine whether a certain model output is abnormal based on the difference. In some examples, the LMF 130 may determine whether to update the model or use a new model.

FIG. 4C illustrates a signalling chart of a process 430 between the terminal device 110 and the LMF 130 in accordance with some embodiments of the present disclosure.

The LMF 130 transmits an LPP model input statistical parameters request message to the terminal device 110 at 431. In some implementations, the LPP model input statistical parameters request message at 431 may request at least one statistical parameter of a plurality of model inputs.

In some implementations, the LPP model input statistical parameters request message at 431 may be regarded as the first message discussed in FIG. 2. In some implementations, the LPP model input statistical parameters request message at 431 may include first information which may be one or more of: an identification of the plurality of model inputs, or a number of the plurality of model inputs.

In some examples, the first information in the LPP model input statistical parameters request message at 431 may assist the terminal device 110 to identify the plurality of model inputs.

In the process 430, the terminal device 110 determines the at least one statistical parameter of a plurality of model inputs at 432 based on the first message. In some implementations, the terminal device 110 may determine the plurality of model inputs, e.g. based on the first information in the LPP model input statistical parameters request message at 431. For example, the first information may indicate a model or indicate a network device, and the plurality of model inputs may be those of the indicated model or those of a specific model used for the indicated network device. The terminal device 110 may then determine (or calculate) the at least one statistical parameter of the plurality of model inputs. In some examples, the at least one statistical parameter may include a correlation matrix of the plurality of model inputs.

In the process 430, the terminal device 110 transmits an LPP model input statistical parameters response message to the LMF 130 at 433. In some implementations, the LPP model input statistical parameters response message at 433 may include the at least one statistical parameter of the plurality of model inputs requested by the LPP model input statistical parameters request message at 431. For example, the LPP model input statistical parameters response message at 433 may include the correlation matrix of the plurality of model inputs.

In some examples, the LMF 130 may determine whether the model deployed at the terminal device 110 is inappropriate, e.g. the inappropriate model may have a borderline input.

As such, the model monitoring may be performed based on the at least one statistical parameter, this solution may have a minimal impact for the terminal device 110, especially when a same AI/ML model is used for multiple network devices 120.

FIG. 4D illustrates a signalling chart of a process 440 between the terminal device 110 and the LMF 130 in accordance with some embodiments of the present disclosure.

The LMF 130 transmits an LPP test data message to the terminal device 110 at 441. In some implementations, the LPP test data message at 441 may indicate test data, and the LPP test data message at 441 may request a test result associated with the test data.

In some implementations, the LPP test data message at 441 may be regarded as the first message discussed in FIG. 2. In some implementations, the LPP test data message at 441 may include first information which may indicate the test data.

In some other examples, the first information may include an ID of a data set which includes the test data. In some embodiments, multiple test data sets with multiple data set IDs may be predefined. For example, the multiple test data sets may be provided to the terminal device 110 together with the AI/ML model deployed at the terminal device 110. For example, signalling used for providing the multiple test data sets may be an LPP message or a message being not an LPP message. For example, the data set may specify the test data associated with the deployment scenario. For example, one or more following features may be different drops, different cluster parameters, different InF scenarios, or different synchronization errors.

In some examples, there may be multiple different AI/ML models deployed at the terminal device 110, and the multiple different AI/ML models are corresponding to multiple different network device 120 (such as N network devices 120) , in this case, the first information may further include one or more IDs of one or more AI/ML models among the multiple different AI/ML models, or include one or more IDs of one or more network devices 120 among the multiple different network device 120. In other words, the first information may indicate one or more AI/ML models to be tested.

The terminal device 110 may determine the test result based on the test data at 442. In some examples, if there is one AI/ML model deployed at the terminal device 110, the terminal device 110 may run the model to determine the test result. In some other examples, if there are multiple different AI/ML models deployed at the terminal device 110 and the first information indicate one or more AI/ML models, the terminal device 110 may run the one or more models to determine the test result.

In some implementations, if the test data includes a pair of a model input and a model output, the terminal device 110 may determine an inferred model output based on the model input indicated by the test data, and further calculate a difference between the model output indicated by the test data and the inferred model output. For example, the test result may include the difference. In some implementations, if the test data includes a model input (e.g. model output is not included) , the terminal device 110 may determine an inferred model output based on the model input indicated by the test data, for example, the test result may include the inferred model output.

In the process 440, the terminal device 110 transmits an LPP provide monitoring information message to the LMF 130 at 443. In some implementations, the LPP provide monitoring information message at 443 may include the test result which is corresponding to the test data indicated by the LPP provide test data message at 441.

In addition or alternatively, the LMF 130 may identify (or determine or detect) whether the model input is abnormal based on the test result. In some examples, the LMF 130 may determine whether the model is inappropriate, e.g. the inappropriate model may have a borderline input. As such, the model monitoring may be performed based on the test result, this solution may have a minimal impact for the terminal device 110, especially when a same AI/ML model is used for multiple network devices 120.

It is to be understood that any message between the terminal device 110 and the LMF 130 in any of processes 420-440 may be implemented as an existing LPP message or a newly defined LPP message, the present disclosure does not limit this aspect.

According to some embodiments discussed with reference to FIGS. 2-4D, a process of model monitoring may be performed in case a model accuracy is lower than a threshold. In this way, a reliability of the model for positioning may be determined, and thus the positioning may be guaranteed.

Reference is further made to FIG. 5, which illustrates a signalling chart illustrating communication process 500 in accordance with some example embodiments of the present disclosure. The process 500 may involve the terminal device 110 and the LMF 130 as shown in FIG. 1.

It is assumed that there may be one or more AI/ML models deployed at the terminal device 110, that is, a scenario of an UE-assisted positioning or LMF-based positioning with UE-side model (case 2a discussed above) is assumed.

In the process 500, the terminal device 110 determines that an accuracy of a location is lower than a threshold at 510. In some implementations, the terminal device 110 may determine an inferred location, e.g. based on output of the AI/ML model. In some implementations, the terminal device 110 may determine an accurate location, e.g. based on a GPS system.

In some implementations, the terminal device 110 may determine (or calculate) a difference between the inferred location and the accurate location, and further determine whether the difference is lower than a predefined threshold. In some examples, if the difference is lower than (or not larger than) the predefined threshold, the terminal device 110 may determine that the accuracy of the inferred location is higher (or not less) than a threshold (or an accuracy threshold) . In some other examples, if the difference is high than (or nor lower than) the predefined threshold, the terminal device 110 may determine that the accuracy of the inferred location is less (or not larger) than the threshold (or the accuracy threshold) .

In the process 500, if the accuracy is lower than the threshold, the terminal device 110 transmits a first message to the LMF 130 at 520. In some implementations, the first message is used for requesting at least one geographical coordinate of the at least one network device 120.

In the process 500, the LMF 130 transmits a second message to the terminal device 110 at 530, where the second message may include the at least one geographical coordinate of the at least one network device 120.

In addition, the terminal device 110 determines a difference between a model output and a ground truth at 540. In some implementations, the terminal device 110 may determine the ground truth based on the second message, i.e. the at least one geographical coordinate of the at least one network device 120. The terminal device 110 further transmits a report to the LMF 130 at 550, and the report may include the difference.

In addition or alternatively, the LMF 130 determines a reliability of the model deployed at the terminal device 110. In some implementations, the LMF 130 may determine an availability of the AI/ML model at the terminal device 110. For example, the LMF 130 may indicate to the terminal device 110 to update the AI/ML model or use another model for positioning.

FIG. 6A illustrates an example schematic for determining 610 a monitoring metric at a terminal device in accordance with some embodiments of the present disclosure. It is assumed that there is one or multiple AI/ML model (s) deployed at the terminal device 110, that is, a scenario of an UE-assisted positioning or LMF-based positioning with UE-side model (case 2a discussed above) is assumed.

In some implementations, there may be one AI/ML model deployed at the terminal device 110, where the AI/ML model is used for multiple network devices 120 (such as TRPs or gNBs) . In some implementations, there may be multiple AI/ML models deployed at the terminal device 110, where a specific AI/ML model is used for a specific network device 120 (such as TRP or gNB) .

In some implementations, the terminal device 110 may determine multiple model outputs (such as TOAs) corresponding to the multiple network devices 120 (such as TRPs or gNBs) respectively.

In some implementations, there may be a calculator and a comparator at the terminal device 110. In some examples, the calculator at the terminal device 110 may determine an estimated UE position (i.e. an inferred location) based on the TOAs. In some examples, the comparator at the terminal device 110 may compare the estimated UE position with a real UE position (i.e. an accurate location) to obtain a monitoring metric 612.

In some implementations, a positioning accuracy may be determined based on the monitoring metric 612. For example, if the positioning accuracy degrades to an unacceptable level, e.g. the monitoring metric 612 is larger (or not less) than a predefined threshold, the terminal device 110 may further perform model monitoring, such as the process shown in FIG. 6B.

FIG. 6B illustrates a signalling chart of a process 620 between the terminal device and the LMF in accordance with some embodiments of the present disclosure.

In the process 620, the terminal device 110 transmits an LPP request assistance data message to the LMF 130 at 621. In some implementations, the LPP request assistance data message at 621 may indicate one or more network devices 120 among all network devices (such as N TRPs or N gNBs) . In some examples, the one or more network devices 120 may refer to some or all of the N network devices. In some examples, the terminal device 110 may select one or more from the N network devices. For example, the selection of the one or more network devices 120 may depend on UE implementation. For example, the selection of the one or more network devices may depend on the models involved in the positioning procedure.

In some implementations, the LPP request assistance data message at 621 may request one or more geographical coordinates of the one or more network devices 120.

In the process 620, the LMF 130 transmits a further LPP provide assistance data message to the terminal device 110 at 622. In some implementations, the further LPP provide assistance data message at 622 may include the one or more geographical coordinates of the one or more network devices 120.

The terminal device 110 determines, at 623, a difference between a model output and a ground truth. In some implementations, the terminal device 110 may determine the ground truth based on the multiple geographical coordinates of the multiple network devices 120, optional further based on other apparatus of the terminal device 110 (such as the sensor, GNSS, etc. ) . In some implementations, the terminal device 110 may determine the ground truth based on a deformation, which may a straight-line distance between the terminal device 110 and a network device 120 divided by a speed of light.

In the process 620, the terminal device 110 transmits an LPP provide monitoring information message to the LMF 130 at 624. In some implementations, the LPP provide monitoring information message at 624 may include the difference between the model output and the ground truth.

According to some embodiments discussed with reference to FIGS. 5-6B, a process of model monitoring may be performed in case a model accuracy is lower than a threshold. In this way, a reliability of the model for positioning may be determined, and thus the positioning may be guaranteed.

FIG. 7 illustrates a flowchart of an example method 700 implemented at an LMF in accordance with some embodiments of the present disclosure. For the purpose of discussion, the method 700 will be described from the perspective of the LMF 130 with reference to FIG. 1.

At block 710, if an accuracy of a location determined based on at least one model output is lower than a threshold, the LMF transmits, to a communication device deployed with a model for positioning, a first message comprising assistance information indicating to the communication device to provide requested information for model monitoring. At block 720, the LMF receives, from the communication device, a second message comprising the requested information. At block 730, the LMF determines a reliability of the model based on the second message.

In some example embodiments, the requested information comprises at least one input of the model, and the first message comprises at least one of: a timestamp of a model output associated with the at least one model input, an identification of the model output, or a number of the at least one model input.

In some example embodiments, the requested information comprises at least one statistical parameter of a plurality of model inputs, and the first message comprises at least one of: an identification of the plurality of model inputs, or a number of the plurality of model inputs.

In some example embodiments, the at least one statistical parameter comprises a correlation matrix of the plurality of model inputs.

In some example embodiments, the first message indicates test data, and the requested information comprises a test result associated with test data, and the test data comprises at least one of: at least one pair of a model input and a model output, or at least one model input.

In some example embodiments, the first message comprises an identifier of a data set comprising the test data.

In some example embodiments, the communication device comprises a terminal device, and the first message comprises information about a plurality of models for a plurality of network devices.

In some example embodiments, the communication device comprises a network device, the requested information comprises a difference between a model output and a ground truth, and the first message comprises at least one of: a location of a terminal device being used as the ground truth, a timestamp of the model output, or an identification of the model output.

In some example embodiments, the communication device comprises a terminal device, the requested information comprises a difference between a model output and a ground truth, and the first message comprises a plurality of geographical coordinates of a plurality of network devices.

In some example embodiments, the LMF receives, from the communication device, the at least one model output; the LMF determines the location based on the at least one model output; and the LMF determines that the accuracy of the location is lower than the threshold by comparing the location and a ground truth of the location.

FIG. 8 illustrates a flowchart of an example method 800 implemented at a communication device in accordance with some embodiments of the present disclosure. For the purpose of discussion, the method 800 will be described from the perspective of the communication 202 deployed with a model for positioning which may be a terminal device 110 or a network device 120 with reference to FIG. 1.

At block 810, the communication device receives, from an LMF, a first message comprising assistance information indicating to the communication device to provide requested information for model monitoring. At block 820, the communication device determines the requested information based on the first message. At block 830, the communication device transmits, to the LMF, a second message comprising the requested information.

In some example embodiments, the requested information comprises at least one model input of the model, and wherein the first message comprises at least one of: a timestamp of a model output associated with the at least one model input, an identification of the model output, or a number of the at least one model input.

In some example embodiments, the requested information comprises at least one statistical parameter of a plurality of model inputs, and wherein the first message comprises at least one of: an identification of the plurality of model inputs, or a number of the plurality of model inputs.

In some example embodiments, the first message indicates test data, and the requested information comprises a test result associated with test data, and wherein the test data comprises at least one of: at least one pair of a model input and a model output, or at least one model input.

In some example embodiments, the communication device comprises a terminal device, and wherein the first message comprises information about a plurality of models for a plurality of network devices.

In some example embodiments, the communication device comprises a network device, the requested information comprises a difference between a model output and a ground truth, and wherein the first message comprises at least one of: a location of the terminal device being used as the ground truth, a timestamp of the model output, or an identification of the model output.

In some example embodiments, the communication device comprises a terminal device, the requested information comprises a difference between a model output and a ground truth, and wherein the first message comprises a plurality of geographical coordinates of a plurality of network devices.

FIG. 9 illustrates a flowchart of an example method 900 implemented at a terminal device in accordance with some embodiments of the present disclosure. For the purpose of discussion, the method 900 will be described from the perspective of the terminal device 110 with reference to FIG. 1.

At block 910, in accordance with a determination that an accuracy of a location determined based on at least one model output is lower than a threshold, the terminal device transmits, to an LMF, a first message comprising an indication of at least one network device, wherein the first message is used for requesting at least one geographical coordinate of the at least one network device, wherein there is at least one model deployed at the terminal device. At block 920, the terminal device receives, from the LMF, a second message comprising the at least one geographical coordinate of the at least one network device. At block 930, the terminal device determines a difference between a model output and a ground truth, wherein the ground truth is determined based on the second message. At block 940, the terminal device transmits, to the LMF, a report comprising the difference.

FIG. 10 illustrates a flowchart of an example method 1000 implemented at an LMF in accordance with some embodiments of the present disclosure. For the purpose of discussion, the method 1000 will be described from the perspective of the LMF 130 with reference to FIG. 1.

At block 1010, the LMF receives, from a terminal device deployed with at least one model, a first message comprising an indication of at least one network device, wherein the first message is used for requesting at least one geographical coordinate of the at least one network device. At block 1020, the LMF transmits, to the terminal device, a second message comprising the at least one geographical coordinate of the at least one network device. At block 1030, the LMF receives, from the terminal device, a report comprising a difference between a model output and a ground truth, wherein the ground truth is determined based on the second message.

Details of some embodiments according to the present disclosure have been described with reference to FIGS. 2-10. Now an example implementation of the terminal device and the network device will be discussed below.

In some example embodiments, an LMF comprises circuitry configured to: in accordance with a determination that an accuracy of a location determined based on at least one model output is lower than a threshold, transmit, to a communication device deployed with a model for positioning, a first message comprising assistance information indicating to the communication device to provide requested information for model monitoring; receive, from the communication device, a second message comprising the requested information; and determine a reliability of the model based on the second message.

In some example embodiments, the requested information comprises at least one input of the model, and wherein the first message comprises at least one of: a timestamp of a model output associated with the at least one model input, an identification of the model output, or a number of the at least one model input.

In some example embodiments, the communication device comprises a network device, the requested information comprises a difference between a model output and a ground truth, and wherein the first message comprises at least one of: a location of a terminal device being used as the ground truth, a timestamp of the model output, or an identification of the model output.

In some example embodiments, the LMF comprises circuitry configured to: receive, from the communication device, the at least one model output; determine the location based on the at least one model output; and determine that the accuracy of the location is lower than the threshold by comparing the location and a ground truth of the location.

In some example embodiments, a communication device comprises circuitry configured to: receive, from an LMF, a first message comprising assistance information indicating to the communication device to provide requested information for model monitoring; determine the requested information based on the first message; and transmit, to the LMF, a second message comprising the requested information, wherein the communication device comprises a terminal device or a network device deployed with a model for positioning.

In some example embodiments, a terminal device comprises circuitry configured to: in accordance with a determination that an accuracy of a location determined based on at least one model output is lower than a threshold, transmit, to an LMF, a first message comprising an indication of at least one network device, wherein the first message is used for requesting at least one geographical coordinate of the at least one network device, wherein there is at least one model deployed at the terminal device; receive, from the LMF, a second message comprising the at least one geographical coordinate of the at least one network device; determine a difference between a model output and a ground truth, wherein the ground truth is determined based on the second message; and transmit, to the LMF, a report comprising the difference.

In some example embodiments, an LMF comprises circuitry configured to: receive, from a terminal device deployed with at least one model, a first message comprising an indication of at least one network device, wherein the first message is used for requesting at least one geographical coordinate of the at least one network device; transmit, to the terminal device, a second message comprising the at least one geographical coordinate of the at least one network device; and receive, from the terminal device, a report comprising a difference between a model output and a ground truth, wherein the ground truth is determined based on the second message.

FIG. 11 illustrates a simplified block diagram of a device 1100 that is suitable for implementing embodiments of the present disclosure. The device 1100 can be considered as a further example implementation of a network device, an LMF, or a terminal device as described above. Accordingly, the device 1100 can be implemented at or as at least a part of the terminal device 110, the network device 120, or the LMF 130 as shown in FIG. 1.

As shown, the device 1100 includes a processor 1110, a memory 1120 coupled to the processor 1110, a suitable transceiver 1140 coupled to the processor 1110, and a communication interface coupled to the transceiver 1140. The memory 1120 stores at least a part of a program 1130. The transceiver 1140 may be for bidirectional communications or a unidirectional communication based on requirements. The transceiver 1140 may include at least one of a transmitter and a receiver. The transmitter and the receiver may be functional modules or physical entities. The transceiver 1140 has at least one antenna to facilitate communication, though in practice an Access Node mentioned in this application may have several ones. The communication interface may represent any interface that is necessary for communication with other network elements, such as X2/Xn interface for bidirectional communications between eNBs/gNBs, S1/NG interface for communication between a Mobility Management Entity (MME) /Access and Mobility Management Function (AMF) /serving gateway (SGW) /user plane function (UPF) and the eNB/gNB, Un interface for communication between the eNB/gNB and a relay node (RN) , or Uu interface for communication between the eNB/gNB and a terminal device.

The program 1130 is assumed to include program instructions that, when executed by the associated processor 1110, enable the device 1100 to operate in accordance with the embodiments of the present disclosure, as discussed herein with reference to FIGS. 2-10. The embodiments herein may be implemented by computer software executable by the processor 1110 of the device 1100, or by hardware, or by a combination of software and hardware. The processor 1110 may be configured to implement various embodiments of the present disclosure. Furthermore, a combination of the processor 1110 and memory 1120 may form processing means 1150 adapted to implement various embodiments of the present disclosure.

The memory 1120 may be of any type suitable to the local technical network and may be implemented using any suitable data storage technology, such as a non-transitory computer readable storage medium, semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory and removable memory, as non-limiting examples. While only one memory 1120 is shown in the device 1100, there may be several physically distinct memory modules in the device 1100. The processor 1110 may be of any type suitable to the local technical network, and may include one or more of general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs) and processors based on multicore processor architecture, as non-limiting examples. The device 1100 may have multiple processors, such as an application specific integrated circuit chip that is slaved in time to a clock which synchronizes the main processor.

In summary, embodiments of the present disclosure may provide the following solutions.

The present disclosure provides an LMF, comprising at least one processor configured to cause the LMF at least to: in accordance with a determination that an accuracy of a location determined based on at least one model output is lower than a threshold, transmit, to a communication device deployed with a model for positioning, a first message comprising assistance information indicating to the communication device to provide requested information for model monitoring; receive, from the communication device, a second message comprising the requested information; and determine a reliability of the model based on the second message.

In one embodiment, the LMF as above, the requested information comprises at least one input of the model, and wherein the first message comprises at least one of: a timestamp of a model output associated with the at least one model input, an identification of the model output, or a number of the at least one model input.

In one embodiment, the LMF as above, the requested information comprises at least one statistical parameter of a plurality of model inputs, and wherein the first message comprises at least one of: an identification of the plurality of model inputs, or a number of the plurality of model inputs.

In one embodiment, the LMF as above, the at least one statistical parameter comprises a correlation matrix of the plurality of model inputs.

In one embodiment, the LMF as above, the first message indicates test data, and the requested information comprises a test result associated with test data, and wherein the test data comprises at least one of: at least one pair of a model input and a model output, or at least one model input.

In one embodiment, the LMF as above, the first message comprises an identifier of a data set comprising the test data.

In one embodiment, the LMF as above, the communication device comprises a terminal device, and wherein the first message comprises information about a plurality of models for a plurality of network devices.

In one embodiment, the LMF as above, the communication device comprises a network device, the requested information comprises a difference between a model output and a ground truth, and wherein the first message comprises at least one of: a location of a terminal device being used as the ground truth, a timestamp of the model output, or an identification of the model output.

In one embodiment, the LMF as above, the communication device comprises a terminal device, the requested information comprises a difference between a model output and a ground truth, and wherein the first message comprises a plurality of geographical coordinates of a plurality of network devices.

In one embodiment, the LMF as above, the at least one processor is further configured to cause the LMF to: receive, from the communication device, the at least one model output; determine the location based on the at least one model output; and determine that the accuracy of the location is lower than the threshold by comparing the location and a ground truth of the location.

The present disclosure provides a communication device, comprising at least one processor configured to cause the communication device at least to: receive, from an LMF, a first message comprising assistance information indicating to the communication device to provide requested information for model monitoring; determine the requested information based on the first message; and transmit, to the LMF, a second message comprising the requested information, wherein the communication device comprises a terminal device or a network device deployed with a model for positioning.

In one embodiment, the communication device as above, the requested information comprises at least one model input of the model, and wherein the first message comprises at least one of: a timestamp of a model output associated with the at least one model input, an identification of the model output, or a number of the at least one model input.

In one embodiment, the communication device as above, the requested information comprises at least one statistical parameter of a plurality of model inputs, and wherein the first message comprises at least one of: an identification of the plurality of model inputs, or a number of the plurality of model inputs.

In one embodiment, the communication device as above, the at least one statistical parameter comprises a correlation matrix of the plurality of model inputs.

In one embodiment, the communication device as above, the first message indicates test data, and the requested information comprises a test result associated with test data, and wherein the test data comprises at least one of: at least one pair of a model input and a model output, or at least one model input.

In one embodiment, the communication device as above, the first message comprises an identifier of a data set comprising the test data.

In one embodiment, the communication device as above, the communication device comprises a terminal device, and wherein the first message comprises information about a plurality of models for a plurality of network devices.

In one embodiment, the communication device as above, the communication device comprises a network device, the requested information comprises a difference between a model output and a ground truth, and wherein the first message comprises at least one of: a location of the terminal device being used as the ground truth, a timestamp of the model output, or an identification of the model output.

In one embodiment, the communication device as above, the communication device comprises a terminal device, the requested information comprises a difference between a model output and a ground truth, and wherein the first message comprises a plurality of geographical coordinates of a plurality of network devices.

The present disclosure provides a terminal device, comprising at least one processor configured to cause the terminal device at least to: in accordance with a determination that an accuracy of a location determined based on at least one model output is lower than a threshold, transmit, to an LMF, a first message comprising an indication of at least one network device, wherein the first message is used for requesting at least one geographical coordinate of the at least one network device, wherein there is at least one model deployed at the terminal device; receive, from the LMF, a second message comprising the at least one geographical coordinate of the at least one network device; determine a difference between a model output and a ground truth, wherein the ground truth is determined based on the second message; and transmit, to the LMF, a report comprising the difference.

The present disclosure provides an LMF, comprising at least one processor configured to cause the LMF at least to: receive, from a terminal device deployed with at least one model, a first message comprising an indication of at least one network device, wherein the first message is used for requesting at least one geographical coordinate of the at least one network device; transmit, to the terminal device, a second message comprising the at least one geographical coordinate of the at least one network device; and receive, from the terminal device, a report comprising a difference between a model output and a ground truth, wherein the ground truth is determined based on the second message.

The present disclosure provides a method of communication, comprising the operations implemented at the LMF discussed above. The present disclosure provides a method of communication, comprising the operations implemented at the communication device discussed above. The present disclosure provides a method of communication, comprising the operations implemented at the terminal device discussed above.

The present disclosure provides a network device, comprising: a processor; and a memory storing computer program codes; the memory and the computer program codes configured to, with the processor, cause the network device to perform the method implemented at the communication device discussed above.

The present disclosure provides an LMF, comprising: a processor; and a memory storing computer program codes; the memory and the computer program codes configured to, with the processor, cause the LMF to perform the method implemented at the LMF discussed above.

The present disclosure provides a terminal device, comprising: a processor; and a memory storing computer program codes; the memory and the computer program codes configured to, with the processor, cause the terminal device to perform the method implemented at the terminal device or the communication device discussed above.

The present disclosure provides a non-transient computer readable medium having instructions stored thereon, the instructions, when executed by a processor of an apparatus, causing the apparatus to perform the method implemented at an LMF, a communication device, or a terminal device discussed above.

The present disclosure provides a computer program product having instructions stored thereon, the instructions, when executed by a processor of an apparatus, causing the apparatus to perform the method implemented at an LMF, a communication device, or a terminal device discussed above.

Generally, various embodiments of the present disclosure may be implemented in hardware or special purpose circuits, software, logic or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device. While various aspects of embodiments of the present disclosure are illustrated and described as block diagrams, flowcharts, or using some other pictorial representation, it will be appreciated that the blocks, apparatus, systems, techniques or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

The present disclosure also provides at least one computer program product tangibly stored on a non-transitory computer readable storage medium. The computer program product includes computer-executable instructions, such as those included in program modules, being executed in a device on a target real or virtual processor, to carry out the process or method as described above. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, or the like that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or split between program modules as desired in various embodiments. Machine-executable instructions for program modules may be executed within a local or distributed device. In a distributed device, program modules may be located in both local and remote storage media.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowcharts and/or block diagrams to be implemented. The program code may execute entirely on a machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

The above program code may be embodied on a machine readable medium, which may be any tangible medium that may contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine readable medium may be a machine readable signal medium or a machine readable storage medium. A machine readable medium may include but not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine readable storage medium would include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM) , a read-only memory (ROM) , an erasable programmable read-only memory (EPROM or Flash memory) , an optical fiber, a portable compact disc read-only memory (CD-ROM) , an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are contained in the above discussions, these should not be construed as limitations on the scope of the present disclosure, but rather as descriptions of features that may be specific to particular embodiments. Certain features that are described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable sub-combination.

Although the present disclosure has been described in language specific to structural features and/or methodological acts, it is to be understood that the present disclosure defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

A location management function (LMF) comprising at least one processor configured to cause the LMF at least to:

in accordance with a determination that an accuracy of a location determined based on at least one model output is lower than a threshold, transmit, to a communication device deployed with a model for positioning, a first message comprising assistance information indicating to the communication device to provide requested information for model monitoring;

receive, from the communication device, a second message comprising the requested information; and

determine a reliability of the model based on the second message.
The LMF of claim 1, wherein the requested information comprises at least one input of the model, and wherein the first message comprises at least one of:

a timestamp of a model output associated with the at least one model input,

an identification of the model output, or

a number of the at least one model input.
The LMF of claim 1, wherein the requested information comprises at least one statistical parameter of a plurality of model inputs, and wherein the first message comprises at least one of:

an identification of the plurality of model inputs, or

a number of the plurality of model inputs.
The LMF of claim 3, wherein the at least one statistical parameter comprises a correlation matrix of the plurality of model inputs.
The LMF of claim 1, wherein the first message indicates test data, and the requested information comprises a test result associated with test data, and wherein the test data comprises at least one of:

at least one pair of a model input and a model output, or

at least one model input.
The LMF of claim 5, wherein the first message comprises an identifier of a data set comprising the test data.
The LMF of claim 5, wherein the communication device comprises a terminal device, and wherein the first message comprises information about a plurality of models for a plurality of network devices.
The LMF of claim 1, wherein the communication device comprises a network device, the requested information comprises a difference between a model output and a ground truth, and wherein the first message comprises at least one of:

a location of a terminal device being used as the ground truth,

a timestamp of the model output, or

an identification of the model output.
The LMF of claim 1, wherein the communication device comprises a terminal device, the requested information comprises a difference between a model output and a ground truth, and wherein the first message comprises a plurality of geographical coordinates of a plurality of network devices.
The LMF of claim 1, wherein the at least one processor is further configured to cause the LMF to:

receive, from the communication device, the at least one model output;

determine the location based on the at least one model output; and

determine that the accuracy of the location is lower than the threshold by comparing the location and a ground truth of the location.
A communication device comprising at least one processor configured to cause the communication device at least to:

receive, from a location management function (LMF) , a first message comprising assistance information indicating to the communication device to provide requested information for model monitoring;

determine the requested information based on the first message; and

transmit, to the LMF, a second message comprising the requested information,

wherein the communication device comprises a terminal device or a network device deployed with a model for positioning.
The communication device of claim 11, wherein the requested information comprises at least one model input of the model, and wherein the first message comprises at least one of:

a timestamp of a model output associated with the at least one model input,

an identification of the model output, or

a number of the at least one model input.
The communication device of claim 11, wherein the requested information comprises at least one statistical parameter of a plurality of model inputs, and wherein the first message comprises at least one of:

an identification of the plurality of model inputs, or

a number of the plurality of model inputs.
The communication device of claim 13, wherein the at least one statistical parameter comprises a correlation matrix of the plurality of model inputs.
The communication device of claim 11, wherein the first message indicates test data, and the requested information comprises a test result associated with test data, and wherein the test data comprises at least one of:

at least one pair of a model input and a model output, or

at least one model input.
The communication device of claim 15, wherein the first message comprises an identifier of a data set comprising the test data.
The communication device of claim 11, wherein the communication device comprises a network device, the requested information comprises a difference between a model output and a ground truth, and wherein the first message comprises at least one of:

a location of the terminal device being used as the ground truth,

a timestamp of the model output, or

an identification of the model output.
The communication device of claim 11, wherein the communication device comprises a terminal device, the requested information comprises a difference between a model output and a ground truth, and wherein the first message comprises a plurality of geographical coordinates of a plurality of network devices.
A terminal device comprising at least one processor configured to cause the terminal device at least to:

in accordance with a determination that an accuracy of a location determined based on at least one model output is lower than a threshold, transmit, to a location management function (LMF) , a first message comprising an indication of at least one network device, wherein the first message is used for requesting at least one geographical coordinate of the at least one network device, wherein there is at least one model deployed at the terminal device;

receive, from the LMF, a second message comprising the at least one geographical coordinate of the at least one network device;

determine a difference between a model output and a ground truth, wherein the ground truth is determined based on the second message; and

transmit, to the LMF, a report comprising the difference.
A location management function (LMF) comprising at least one processor configured to cause the LMF at least to:

receive, from a terminal device deployed with at least one model, a first message comprising an indication of at least one network device, wherein the first message is used for requesting at least one geographical coordinate of the at least one network device;

transmit, to the terminal device, a second message comprising the at least one geographical coordinate of the at least one network device; and

receive, from the terminal device, a report comprising a difference between a model output and a ground truth, wherein the ground truth is determined based on the second message.