WO2025202004A1

WO2025202004A1 - System and apparatus for determining a model accuracy and a method in association thereto

Info

Publication number: WO2025202004A1
Application number: PCT/EP2025/057579
Authority: WO
Inventors: Rajat PUSHKARNA; Rikin SHAH; Reuben GEORGE STEPHEN; Hojin Kim
Original assignee: Continental Automotive Technologies GmbH
Current assignee: Aumovio Germany GmbH
Priority date: 2024-03-26
Filing date: 2025-03-20
Publication date: 2025-10-02
Anticipated expiration: 2026-09-26
Also published as: DE102024108668A1

Abstract

System (100), device (104) and a method (300) for determining a model accuracy are disclosed. The method (300) includes configuring data associated with a reference dataset of the model; transmitting the configured data; and determining the model accuracy of the user device based on the configured data.

Description

SYSTEM AND APPARATUS FOR DETERMINING A MODEL ACCURACY AND A METHOD IN ASSOCIATION THERETO

Field Of Invention

[001] The present disclosure generally relates to one or both of a system and a device for determining a model accuracy in association with, for example, a User Equipment (UE) usable for communication. The present disclosure further relates a method which can be associated with the system and/or the device.

Background of Invention

[002] Generally, energy efficiency and power saving would be helpful in communication networks, for example, a 3rd Generation Partnership Project (3GPP) 5G (fifth generation) New Radio (NR) standard-based telecommunications network.

[003] Current techniques may not address the issue of determining a model accuracy when providing inference to a model, for example an Artificial Intelligence/Machine Learning (AIML) model, during a Life Cycle Management (LCM) cycle and may lead to problems such as unnecessary energy consumption. Thus, the current techniques may not facilitate energy efficiency and power saving in an optimal manner.

[004] The present disclosure contemplates that it would be helpful to address or at least mitigate one or more issues in relation to conventional techniques for facilitating energy efficiency and power saving when determining a model accuracy.

Summary of the Invention

[005] According to a first aspect of the present invention, there is provided a method for determining a model accuracy, the method comprising: configuring data associated with a reference dataset of the model; transmitting the configured data; and determining the model accuracy of the user device based on the configured data [006] Advantageously, the method as described herein can provide a standardized and/or a reference dataset to determine model accuracy so that model performance reliability can be ensured.

[007] In an embodiment, transmitting the data comprises transmitting via at least one of: system information message, Radio Resource Control (RRC) reconfiguration and/or a signal configured for a model.

[008] In an embodiment, the reference dataset comprises a mapping table associated with online field data and offline field data.

[009] In an embodiment, the method further includes training the model online based on the online field data.

[0010] In an embodiment, the method further includes training the model offline based on the offline field data.

[0011] In an embodiment, determining the model accuracy comprises determining the model accuracy at a pre-determined time period or at a pre-determined event.

[0012] In an embodiment, configuring the data comprises configuring a predetermined validation threshold for model accuracy.

[0013] In an embodiment, the method further includes analyzing the model accuracy to determine a reliability of the model.

[0014] In an embodiment, the data associated with a reference dataset of the model comprises dataset identification based on meta information of the model.

[0015] In an embodiment, dataset identification comprises at least one of: dataset statistics, dataset format and/or information related to an entity obtaining the dataset. [0016] In an embodiment, the model is an Artificial Intelligence/Machine Learning (AIML) model.

[0017] In an embodiment, there is provided a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of the first aspect.

[0018] In an embodiment, there is provided a computer readable storage medium having data stored therein representing software executable by a computer, the software including instructions, when executed by the computer, to carry out the method of the first aspect.

[0019] In an embodiment, there is provided an apparatus for determining a model accuracy comprising: a first module configured to receive at least one input signal having data associated with a reference dataset of the model; a second module configured to at least one of process and facilitate the method of the first aspect to generate at least one output signal; and a third module configured to communicate at least one output signal, wherein the output signal corresponds to a control signal for determining the model accuracy.

[0020] In an embodiment, the apparatus corresponds to a User Equipment (UE) communicable with a device corresponding to a base station, and wherein the base station corresponds to a Next generation Node B (gNB) configured to transmit the at least one input signal to the UE.

[0021] In an embodiment, there is provided a system comprising: at least one apparatus(es); and at least one device(s), wherein the apparatus(es) and the device(s) are capable of being coupled via at least one of wired coupling and wireless coupling. [0022] Advantageously, the system as described herein can provide fundamental mechanisms of interworking and data information flow in radio access network collaboration for AIML support. Accordingly, AIML performance for wireless communication can be improved.

Brief Description of the Drawings

[0023] Embodiments of the disclosure are described hereinafter with reference to the following drawings, in which:

[0024] Fig. 1A shows a schematic diagram illustrating a system for determining a model accuracy which can include at least one apparatus, according to an embodiment of the invention.

[0025] Figs. 1 B shows an example scenario in association with the system of Fig. 1A, according to an embodiment of the invention.

[0026] Fig. 2 shows a schematic diagram illustrating the apparatus of Fig. 1A in further detail, according to an embodiment of the invention.

[0027] Fig. 3 shows a method in association with the system of Fig. 1A, according to an embodiment of the invention.

[0028] Fig. 4A to Fig. 4B show schematic diagrams illustrating the flow of information in association with the method of Fig. 3, according to an embodiment of the invention.

Detailed Description

[0029] The present specification discloses apparatus and/or device for performing the operations of the methods. Such apparatus and/or device may be specially constructed for the required purposes, or may comprise a computer or other device selectively activated or reconfigured by a computer program stored in the computer. The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Various machines may be used with programs in accordance with the teachings herein. Alternatively, the construction of more specialized apparatus to perform the required method steps may be appropriate. The structure of a computer will appear from the description below.

[0030] In addition, the present specification also implicitly discloses a computer program, in that it would be apparent to the person skilled in the art that the individual steps of the method described herein may be put into effect by computer code. The computer program is not intended to be limited to any particular programming language and implementation thereof. It will be appreciated that a variety of programming languages and coding thereof may be used to implement the teachings of the disclosure contained herein. Moreover, the computer program is not intended to be limited to any particular control flow. There are many other variants of the computer program, which can use different control flows without departing from the spirit or scope of the disclosure.

[0031] Furthermore, one or more of the steps of the computer program may be performed in parallel rather than sequentially. Such a computer program may be stored on any computer readable medium. The computer readable medium may include storage devices such as magnetic or optical disks, memory chips, or other storage devices suitable for interfacing with a computer. The computer readable medium may also include a hard-wired medium such as exemplified in the Internet system, or wireless medium such as exemplified in the mobile telephone system. The computer program when loaded and executed on such a computer effectively results in an apparatus and/or a device that implements the steps of the preferred method.

[0032] The present disclosure generally contemplates the facilitation and optimization of a network (for example in association with 3GPP based standard/specification etc.) and/or user equipment (UE) efficiency (for example energy efficiency or power saving), in accordance with an embodiment of the invention. Specifically, the present disclosure contemplates the possibility of determining a model accuracy during a Life Cycle Management (LCM) process in connection with 3GPP standard(s). [0033] The present disclosure generally contemplates that it may be important to have a consistent model accuracy in order for a model (e.g. Artificial Intelligence/Machine Learning (AIML) model) to provide accurate inference during the LCM process. The present disclosure contemplates the possibility to determine if the model (e.g. AIML model) is providing inference accurately during its LCM cycle.

[0034] The present disclosure also contemplates that an entity performing model inference can configure a reference dataset (ground truth) with respect to an individual model identification for an entity performing model training. The present disclosure further contemplates the possibility of having a standardized dataset related to the model (e.g. AIML model) in order to test the model accuracy.

[0035] In the above manner, accurate inference can be provided in the model (e.g. AIML model) based on the consistency of the model accuracy during the LCM process. Power saving and energy consumption efficiency can possibly be facilitated in the network, in accordance with an embodiment of the invention.

[0036] The foregoing will be discussed in further detail with reference to Fig. 1 to Fig. 4 hereinafter.

[0037] Referring to Fig. 1A, a schematic diagram illustrating a system 100 for determining a model accuracy is shown, according to an embodiment of the invention. The system 100 can, for example, be suitable for facilitating energy and improve power efficiency, in accordance with an embodiment of the invention.

[0038] As shown, the system 100 can include one or more apparatuses 102, at least one device 104 and, optionally, a communication network 106, in accordance with an embodiment of the invention.

[0039] The apparatus(es) 102 can be coupled to the device(s) 104. Specifically, the apparatus(es) 102 can, for example, be coupled to the device(s) 104 via the communication network 106, in accordance with an embodiment of the invention. [0040] In one embodiment, the apparatus(es) 102 can be coupled to the communication network 106 and the device(s) 104 can be coupled to the communication network 106. Coupling can be by manner of one or both of wired coupling and wireless coupling. The apparatus(es) 102 can, in general, be configured to communicate with the device(s) 104 via the communication network 106, according to an embodiment of the invention.

[0041] The apparatus(es) 102 can, for example, be associated with or correspond to or include one or more user equipment (UE) which can carry one or more computers, in accordance with an embodiment of the invention. For example, an apparatus 102 can correspond to a UE carrying at least one computer (e.g. an electronic device or module having computing capabilities such as an electronic mobile device which can be carried into a vehicle or an electronic module which can be installed in a vehicle, in accordance with an embodiment of the invention) which can be configured to perform one or more processing tasks in association with adaptive/dynamic/gradual control, in accordance with an embodiment of the invention.

[0042] In an embodiment, the apparatus(es) 102 can, for example, be configured to receive one or more input signals and perform at least one processing task based on the input signal(s) in a manner to generate one or more output signals. The input signal(s) can, for example, be communicated from the device(s) 104 and received by the apparatus(es) 102, in accordance with an embodiment of the invention. The input signal can include data associated with a reference dataset of the model. As a possible option, the output signal(s) can, for example, be communicated from the apparatus(es) 102, in accordance with an embodiment of the invention. The output signal may correspond to a control signal for determining a model, for example an Artificial Intelligence/Machine Learning (AIML) model. The apparatus(es) 102 will be discussed later in further detail with reference to Fig. 2, according to an embodiment of the invention.

[0043] The device(s) 104 can, for example, be associated with/correspond to at least one base station, where the at least one base station can be a Next Generation Node B (gNB). Moreover, the device(s) 104 can, for example, be configured to carry/be associated with/include one or more computers (e.g., an electronic device/module having computing capabilities) which can, for example, be configured to perform one or more processing tasks in association with the base station. The device(s) 104 can be configured to generate one or more input signals which can be communicated to the apparatus(es) 102, in accordance with an embodiment of the invention. For example, the device(s) 104 can configure data associated with a reference dataset of the model (e.g. AIML model). This will be discussed later in further detail in the context of an example scenario, in accordance with an embodiment of the invention.

[0044] The communication network 106 can, for example, correspond to an Internet communication network, a cellular-based communication network, a wired-based communication network, a Global Navigation Satellite System (GNSS) based communication network, a wireless-based communication network, or any combination thereof. Communication (e.g., between the apparatuses 102 and/or between the apparatus(es) 102 and the device(s) 104) via the communication network 106 can be by manner of one or both of wired communication and wireless communication.

[0045] As mentioned, the apparatus(es) 102 can, for example, be configured to receive at least one input signal and perform at least one processing task in association with dynamic/adaptive/gradual control on the input signal(s) in a manner so as to generate at least one output signal. Moreover, the device(s) 104 can, for example, be configured to generate (and communicate) the input signal(s) to the apparatus(es) 102, in accordance with an embodiment of the invention. Accordingly, the device(s) 104 can configure data associated with a reference dataset of the model and also communicate the data to the apparatus(es) 102. This will be discussed, in accordance with an embodiment of the invention, in the context of an example scenario with reference to Fig. 1 B, hereinafter.

[0046] Fig. 1 B shows an example scenario in association with the system of Fig. 1A, according to an embodiment of the invention. Specifically, Fig. 1 B shows an example of model inference, for example an Artificial Intelligence/Machine Learning (AIML) model, in a wireless network. As shown in the Figure, a base station (e.g. Next generation Node B gNB) performs a model (e.g. an AIML model) inference. The base station (or gNB) may be in communication with multiple user devices or User Equipment (UE) A and B where each of the user device (or UE) may perform model training (e.g. AIML model).

[0047] In such communications, model identification can be supported for UE-side models. Advantageously, this can identify a model (e.g. AI/ML model) for common understanding between the base station (e.g. gNB) and the UE, in accordance with an embodiment of the invention. Consistency between training and inference with additional conditions can be ensured and model applicability can be checked. In addition, configuration matching with the model input can be provided and the gain of localized model (e.g. AIML model) can be harvested, in accordance with an embodiment of the invention. The present disclosure thus contemplates that it can be important that model consistency is crucial for model (e.g. AIML model) inference.

[0048] In an embodiment, the present disclosure contemplates radio environment (e.g. construction of a new building) or mobility profiles (e.g. advent of electric bikes) may reduce the accuracy of models (e.g. Machine Learning ML models) and may require re-training. Therefore, coordinated ML Life-Cycle Management (LCM) and monitoring may be required and ML-enabled solutions should be robust and failsafe even in unforeseen situations. In the embodiment shown in Fig. 1 B, the data collection done at the UE at A may not be the same at point B due to changes in the radio environment. Accordingly, a method can be provided to test a model (e.g. an AIML model) for a dataset which is not prone to such changes. This can ensure that the model (e.g. AIML) itself is not undergoing performance degradation.

[0049] In an example embodiment, the AIML Model can be a data driven algorithm that applies AIML techniques to generate a set of outputs based on a set of inputs while AIML model Inference can be a process of using a trained AIML model to produce a set of outputs based on a set of inputs. In another example embodiment, AIML model validation can be a subprocess of training to evaluate the quality of an AIML model using a dataset different from one used for model training. This can help to select model parameters that generalize beyond the dataset used for model training. In yet another example embodiment, data collection can be a process of collecting data by the network nodes, management entity or the UE for the purpose of AIML model training, data analytics and inference. In a further embodiment, offline field data can refer to data collected from the field and used for offline training of the AIML model while online field data can refer to data collected from the field and used for online training of the AIML model.

[0050] The above-described aspect(s) of the system 100 of the present invention can also apply analogously (all) the aspect(s) of a below described apparatus 102 of the present invention. Likewise, all below described aspect(s) of the apparatus 102 of the invention can also apply analogously (all) the aspect(s) of above-described system 100 of the invention.

[0051] The aforementioned apparatus(es) 102 or User Equipment (UE) or user device will be discussed in further detail with reference to Fig. 2 hereinafter.

[0052] Referring to Fig. 2, a schematic diagram illustrating an apparatus 102 is shown in further detail in the context of an example implementation 200, according to an embodiment of the invention.

[0053] In the example implementation 200, the apparatus 102 can correspond to an electronic module 200a. The electronic module 200a can, in one example, correspond to a mobile device which can, for example, be carried into the vehicle by a user, in accordance with an embodiment of the invention. In another example, the electronic module 200a can correspond to an electronic device which can be installed/mounted in the vehicle, in accordance with an embodiment of the invention. In this regard, the electronic module 200a can be considered to be carried by the vehicle (e.g., either carried into the vehicle by a user or installed/mounted in the vehicle).

[0054] It is contemplated that the electronic module 200a can be capable of performing one or more processing tasks in association with adaptive/dynamic/gradual control related processing, in accordance with an embodiment of the invention.

[0055] The electronic module 200a can, for example, include a casing 200b. Moreover, the electronic module 200a can, for example, carry any one of a first module 202, a second module 204, a third module 206, or any combination thereof.

[0056] In one embodiment, the electronic module 200a can carry a first module 202, a second module 204 and/or a third module 206. In a specific example, the electronic module 200a can carry a first module 202, a second module 204 and a third module 206, in accordance with an embodiment of the invention.

[0057] In this regard, it is appreciable that, in one embodiment, the casing 200b can be shaped and dimensioned to carry any one of the first module 202, the second module 204 and the third module 206, or any combination thereof.

[0058] The first module 202 can be coupled to one or both of the second module 204 and the third module 206. The second module 204 can be coupled to one or both of the first module 202 and the third module 206. The third module 206 can be coupled to one or both of the first module 202 and the second module 204. In one example, the first module 202 can be coupled to the second module 204 and the second module 204 can be coupled to the third module 206, in accordance with an embodiment of the invention. Coupling between the first module 202, the second module 204 and/or the third module 206 can, for example, be by manner of one or both of wired coupling and wireless coupling. Each of the first module 202, the second module 204 and the third module 206 can correspond to one or both of a hardware-based module and a software-based module, according to an embodiment of the invention.

[0059] In one example, the first module 202 can correspond to a hardware-based receiver which can be configured to receive one or more input signals. The input signal(s) can, for example, be communicated from the device(s) 104 (or base station e.g., a Next Generation Node B gNB), in accordance with an embodiment of the invention. [0060] The second module 204 can, for example, correspond to a hardware-based processor which can be configured to perform one or more processing tasks (e.g., in a manner so as to generate one or more output signals) as will be discussed later in further detail with reference to Fig. 3, in accordance with an embodiment of the invention.

[0061] The third module 206 can correspond to a hardware-based transmitter which can be configured to communicate one or more output signals from the electronic module 200a. The output signal(s) can, for example, include one or more instructions/commands/control signals in association with the aforementioned dynamic/adaptive/gradual control configuration/determination strategy so as to facilitate efficiency (e.g., power/energy efficiency and/or communication efficiency), in accordance with an embodiment of the invention. For example, the output signal(s) can be a control signal(s) to a model accuracy.

[0062] The present disclosure contemplates the possibility that the first and second modules 202, 204 can be an integrated software-hardware based module, for example, an electronic part which can carry a software program or algorithm in association with receiving and processing functions or an electronic module programmed to perform the functions of receiving and processing. The present disclosure further contemplates the possibility that the first and third modules 202, 206 can be an integrated software-hardware based module, for example an electronic part which can carry a software program or algorithm in association with receiving and transmitting functions or an electronic module programmed to perform the functions of receiving and transmitting. The present disclosure yet further contemplates the possibility that the first and third modules 202, 206 can be an integrated hardware module, for example a hardware-based transceiver, capable of performing the functions of receiving and transmitting.

[0063] The apparatus 102 (or user device or UE) can, for example, be further configured to process the input signal(s), as will be discussed later in further detail with reference to Fig. 3, in a manner so as to generate one or more output signals in a manner so as to facilitate efficiency, for example power efficiency or energy efficiency, in accordance with an embodiment of the invention. In one specific example, the output signal(s) can include one or more control signals to facilitate some form of dynamic/adaptive/gradual control configuration/determination strategy so as to facilitate efficiency, for example power efficiency or energy efficiency, in accordance with an embodiment of the invention. For example, the output signal(s) can be a control signal(s) for determining a model accuracy.

[0064] The above-described aspect(s) of the apparatus 102 of the present invention can also apply analogously (all) the aspect(s) of a below described processing/communication method of the present invention. Likewise, all below described aspect(s) of the method of the invention can also apply analogously (all) the aspect(s) of above described apparatus 102 of the invention. It is to be appreciated that these remarks apply analogously to the earlier discussed system 100 of the present disclosure.

[0065] Referring to Fig. 3, a method 300 (or a communication method) for determining a model accuracy in association with the system 100 is shown, according to an embodiment of the invention.

[0066] The method 300 can, for example, be suitable for facilitating energy efficiency, network optimization and power saving in accordance with an embodiment of the invention.

[0067] The method 300 can include any one of an input step 302, a processing step 304 and an output step 306, or any combination thereof, in accordance with an embodiment of the invention.

[0068] In an embodiment, the processing method 300 can include the input step 302. In another embodiment, the processing method 300 can include the input step 302 and the processing step 304. In another embodiment, the processing method 300 can include the input step 302, the processing step 304 and the output step 306. In yet another embodiment, the processing method 300 can include the processing step 304 and one or both of the input step 302 and the output step 306. In yet a further embodiment, the processing method 300 can include the input step 302, the processing step 304 and the output step 306. In yet a further additional embodiment, the processing method 300 can include the processing step 304. In yet another further additional embodiment, the processing method 300 can include any one of or any combination of the input step 302, the processing step 304 and the output step 306 (i.e. , the input step 302, the processing step 304 and/or the output step 306).

[0069] With regard to the input step 302, one or more input signal(s) can be received. For example, the input signal(s) can be communicated from the device 104 and can be received by the apparatus 102, in accordance with an embodiment of the invention.

[0070] The input step 302 can include receiving at least one input signal including data associated with a reference dataset of the model. The input signal may be transmitted by the device 104 (or base station or gNB) via at least one of: system information message, Radio Resource Control (RRC) reconfiguration and/or a signal configured for a model. The reference dataset may include a mapping table associated with online field data and offline field data. Data associated with a reference dataset of the model may include dataset identification based on meta information (or meta data) of the model and the dataset identification may include at least one of: dataset statistics, dataset format and/or information related to an entity obtaining the dataset.

[0071] With regard to the processing step 304, at least a processing task can be performed in association with the received input signal(s) in a manner so as to generate one or more output signals, in accordance with an embodiment of the invention.

[0072] The processing step 304 may include at least one of: determining the model accuracy of the user device based on the configured data; determining the model accuracy at a pre-determined time period or at a pre-determined event; training the model online based on the online field data; and training the model offline based on the offline field data. [0073] The processing step 304 may also include determining the model accuracy at a pre-determined time period or at a pre-determined event; configuring a predetermined validation threshold for model accuracy; analyzing the model accuracy to determine a reliability of the model. The model can be an Artificial Intelligence/Machine Learning (AIML) model.

[0074] In an implementation, an entity performing model inference (e.g. a base station or gNB) can configure a reference dataset (ground truth) with respect to an individual model identification for an entity performing model training (e.g. a user device or UE). The present disclosure contemplates the possibility of having the reference dataset for determining the model accuracy.

[0075] In an embodiment, the base station (or gNB) may configure the reference dataset for model training which can be used by the UE to check model (e.g. AIML model) validation. The model accuracy analysis may be performed by the user device (or UE) periodically or at an event configured by the base station (or gNB). The base station (or gNB) may also provide the reference dataset as a mapping table for the user device (or UE), which can be signaled to the user device (or UE) via RRC Reconfiguration and/or System Information message. An example of the mapping table is shown in Table 1 below.

Table 1

[0076] In an example embodiment, the online field data of Table 1 above could be used for online training and the offline field data could be used for offline training. The user device (or UE) may use this to perform AIML model validation periodically or at an event configured by the base station (or gNB) upon receiving the reference dataset. In an alternate embodiment, the base station (or gNB) may set a model (e.g. AIML model) validation threshold for the UE. If the AIML model accuracy is determined to be below AIML model validation threshold, then the user device (or UE) may perform the model accuracy analysis and check if the model (e.g. AIML model) is reliable.

[0077] In another example embodiment, the base station (or gNB) may provide a mapping table for the user device (or UE) based on meta information (or meta data). In this embodiment, the dataset ID could indicate a number of datapoints that the user device (or UE) may collect for model accuracy performance or a part of dataset that is already collected or a dataset collection method. The dataset ID may also include information about dataset statistics, dataset format and information on which entity to obtain the dataset from. An example of the mapping table as described herein is shown in Table 2 below.

Table 2

[0078] With regards to the output step 306, the output signal(s) can, for example, be communicated, as an option, in accordance with an embodiment of the invention. For example, the output signal(s) can optionally be communicated from the apparatus 102. In a more specific example, the output signal(s) can optionally be communicated from the apparatus 102 to one or both of at least one apparatus 102 or at least one device 104, in accordance with an embodiment of the invention.

[0079] The present disclosure further contemplates a computer program (not shown) which can include instructions which, when the program is executed by a computer (not shown), cause the computer to carry out the input step 302, the processing step 304 and/or the output step 306 as discussed with reference to the method 300. For example, the computer program can include instructions which, when the program is executed by a computer, cause the computer to carry out the input step 302 and/or the processing step 304, in accordance with an embodiment of the invention.

[0080] The present disclosure yet further contemplates a computer readable storage medium (not shown) having data stored therein representing software executable by a computer (not shown), the software including instructions, when executed by the computer, to carry out the input step 302, the processing step 304 and/or the output step 306 as discussed with reference to the method 300. For example, the computer readable storage medium can have data stored therein representing software executable by a computer, the software including instructions, when executed by the computer, cause the computer to carry out the input step 302 and/or the processing step 304, in accordance with an embodiment of the invention.

[0081] Further in view of the foregoing, it is appreciable that the present disclosure generally contemplates an apparatus 102 for determining a model accuracy (e.g. AIML model) which can include a first module 202, a second module 204 and/or a third module 206.

[0082] The first module 202 can be configured to receive one or more input signals. The input signal(s) can, for example, include data associated with a reference dataset of a model (e.g. AIML model).

[0083] The second module 204 can be configured to process and/or facilitate processing of the input signal(s) according to the method 300 as discussed earlier to generate one or more output signals.

[0084] The third module 206 can be configured to communicate one or more output signals. The output signal(s) can, for example, correspond to one or more control signals for determining a model accuracy (e.g. AIML model).

[0085] In one embodiment, the apparatus 102 can correspond to a User Equipment (UE) which can communicate with a device 104 corresponding to a base station. The base station can, for example, correspond to a Next generation Node B (gNB) which can be configured to communicate one or more signals (e.g., input signal(s)) to the UE.

[0086] Yet further in view of the foregoing, it is appreciable that the present disclosure generally contemplates a system 100 which can include one or more apparatuses 102 and one or more devices 104. The apparatus(es) 102 and the device(s) 104 can, for example, be capable of being coupled via wired coupling and/or wireless coupling.

[0087] It should be appreciated that the embodiments described above can be combined in any manner as appropriate (e.g., one or more embodiments as discussed in the “Detailed Description” section can be combined with one or more embodiments as described in the “Summary of the Invention” section).

[0088] It should be further appreciated by the person skilled in the art that variations and combinations of embodiments described above, not being alternatives or substitutes, may be combined to form yet further embodiments.

[0089] In one example, the possibility of the output signal(s) being communicated from the apparatus(es) 102 was discussed. It is appreciable that the output signal(s) need not necessarily be communicated from the apparatus(es) 102. Specifically, the possibility that the output signal(s) need not necessarily be communicated outside of the apparatus(es) 102 is contemplated, in accordance with an embodiment of the invention. More specifically, the output signal(s) can, for example, correspond to internal command(s)/instruction(s) (e.g., communicated only within an apparatus 102) for adaptively controlling operational configuration of an apparatus 102, in accordance with an embodiment of the invention.

[0090] In another example, application(s) of the present disclosure in association with/in the context of low power wake up radio and/or ambient loT (Internet of Things) type device(s) can be possible, in accordance with an embodiment of the invention. [0091] Fig. 4A to Fig. 4B show schematic diagrams illustrating the flow of information in association with the method of Fig. 3, according to an embodiment of the invention

[0092] In the example context as shown in Fig. 4A, a User Equipment UE (or user device) can, for example, be configured to perform AIML model accuracy analysis. The UE (or user device) can perform the analysis at a configured or pre-determined periodicity, even and/or threshold, in accordance with an embodiment of the invention. In the example context as shown in Fig. 4B, the gNB (or base station) can, for example, configure a true dataset (or reference dataset or ground truth) associated with an individual model identity of the UE (or user device), in accordance with an embodiment of the invention.

[0093] In the foregoing manner, various embodiments of the disclosure are described for addressing at least one of the foregoing disadvantages. Such embodiments are intended to be encompassed by the following claims and are not to be limited to specific forms or arrangements of parts so described and it will be apparent to one skilled in the art in view of this disclosure that numerous changes and/or modification can be made, which are also intended to be encompassed by the following claims.

Claims

Claim(s)

1 . A method (300) for determining a model accuracy, the method comprising: configuring data associated with a reference dataset of the model; transmitting the configured data; and determining the model accuracy of the user device based on the configured data.

2. The method (300) according to claim 1 , wherein transmitting the data comprises transmitting via at least one of: system information message, Radio Resource Control (RRC) reconfiguration and/or a signal configured for a model.

3. The method (300) according to claim 1 , wherein the reference dataset comprises a mapping table associated with online field data and offline field data.

4. The method (300) according to claim 3, further comprising training the model online based on the online field data.

5. The method (300) according to claim 3, further comprising training the model offline based on the offline field data.

6. The method (300) according to claim 1 , wherein determining the model accuracy comprises determining the model accuracy at a pre-determined time period or at a pre-determined event.

7. The method (300) according to claim 1 , wherein configuring data comprises configuring a pre-determined validation threshold for model accuracy.

8. The method (300) according to claim 1 , further comprising analyzing the model accuracy to determine a reliability of the model.

9. The method (300) according to claim 1 , wherein the data associated with a reference dataset of the model comprises dataset identification based on meta information of the model.

10. The method (300) according to claim 9, wherein dataset identification comprises at least one of: dataset statistics, dataset format and/or information related to an entity obtaining the dataset.

11. The method (300) according to any of the preceding claims, wherein the model is an Artificial Intelligence/Machine Learning (AIML) model.

12. A computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method (300) according to any of the preceding claims.

13. A computer readable storage medium having data stored therein representing software executable by a computer, the software including instructions, when executed by the computer, to carry out the method (300) according to any one of claims 1-11.

14. An apparatus (102) for determining a model accuracy comprising: a first module (202) configured to receive at least one input signal having data associated with a reference dataset of the model; a second module (204) configured to at least one of process and facilitate the method (300) of claim 1 to claim 11 to generate at least one output signal; and a third module (206) configured to communicate at least one output signal, wherein the output signal corresponds to a control signal for determining the model accuracy.

15. The apparatus (102) according to claim 14, wherein the apparatus (102) corresponds to a User Equipment (UE) communicable with a device (104) corresponding to a base station, and wherein the base station corresponds to a Next generation Node B (gNB) configured to transmit the at least one input signal to the UE.

16. A system (100) comprising: at least one apparatus (102) according to any of claims 14 and 15; and at least one device (104) according to claim 15, wherein the apparatus (102) and the device (104) are capable of being coupled via at least one of wired coupling and wireless coupling.