EP4616347A1

EP4616347A1 - Feedback on executed field service operation

Info

Publication number: EP4616347A1
Application number: EP23889259.0A
Authority: EP
Inventors: Athanasios KARAPANTELAKIS; Konstantinos Vandikas; Lackis ELEFTHERIADIS; Marin ORLIC; Leonid Mokrushin; Pratyush Kiran UPPULURI; Wenfeng HU; Anders Holst
Original assignee: Telefonaktiebolaget LM Ericsson AB
Current assignee: Telefonaktiebolaget LM Ericsson AB
Priority date: 2022-11-08
Filing date: 2023-11-07
Publication date: 2025-09-17
Also published as: WO2024102051A1

Abstract

A computer-implemented method is provided performed by a computing device (102, 6200) to provide feedback on an executed field service operation for a communication network. The method includes comparing (410) a first model including a first representation of a plurality of semantics of the executed field service operation to a second model including a second representation of a plurality of semantics of a workflow to execute the field service operation to obtain a first similarity metric. The method further includes determining (412) an indication that indicates whether the first similarity metric satisfies a criteria for the executed field service operation; and transmitting (414) the indication to a network node.

Description

FEEDBACK ON EXECUTED FIELD SERVICE OPERATION

TECHNICAL FIELD

[0001] The present disclosure relates generally to computer-implemented methods performed by a computing device to provide feedback on an executed field service operation for a communications network, and related methods and apparatuses.

BACKGROUND

[0002] Artificial intelligence (Al) and machine learning (ML) algorithms may be broadly used in a telecommunications networks not only for optimization, but also to try to ensure proper network operation. For example, Al algorithms may predict faults before they happen, which may trigger automated preventive maintenance loops that may reduce the possibility of faults occurring in the future. Other algorithms may predict utilization of a telecommunications network and may schedule resources in such a way that may serve future users in a fair way and consistent with service level agreements.

SUMMARY

[0003] There currently exist certain challenges. While some approaches may include use of sensory modalities to diagnostically evaluate whether an installed piece of equipment works, such approaches may not address a scalable method to supervise, based on inclusion of a ML model(s), that manual installation of a component(s) is properly implemented (e.g., via feedback on a step(s) of an installation process). See e.g., https://cloud.google.com/blog/products/ai-machine-learning/improve-manufacturing- quality-control-with-visual-inspection-ai (accessed 28 October 2022); "PCB Component Detection using Computer Vision for Hardware Assurance", Wenwei Zhao, Suprith Gurudu, Shayan Taheri, Shajib Ghosh, Mukhil Azhagan Mallaiyan Sathiaseelan, Navid Asadizanjani, arXiv:2202.08452 (17 February 2022).

[0004] Certain aspects of the disclosure and their embodiments may provide solutions to these or other challenges. [0005] In various embodiments of the present disclosure, a computer- implemented method is provided that is performed by a computing device to provide feedback on an executed field service operation for a communication network. The method includes comparing a first model including a first representation of a plurality of semantics of the executed field service operation to a second model including a second representation of a plurality of semantics of a workflow to execute the field service operation to obtain a first similarity metric. The method further includes determining an indication that indicates whether the first similarity metric satisfies a criteria for the executed field service operation; and transmitting the indication to a network node. [0006] In other embodiments, a computing device is provided. The computing device is configured to provide feedback on an executed field service operation for a communications network. The computing device includes processing circuitry; and at least one memory coupled with the processing circuitry. The memory includes instructions that when executed by the processing circuitry causes the computing device to perform operations. The operations include to compare a first model including a first representation of a plurality of semantics of the executed field service operation to a second model including a second representation of a plurality of semantics of a workflow to execute the field service operation to obtain a first similarity metric. The operations further include to determine an indication that indicates whether the first similarity metric satisfies a criteria for the executed field service operation; and to transmit the indication to a network node.

[0007] In other embodiments, a computing device is provided that is configured to provide feedback on an executed field service operation for a communications network. The computing device is adapted to perform operations. The operations include to compare a first model including a first representation of a plurality of semantics of the executed field service operation to a second model including a second representation of a plurality of semantics of a workflow to execute the field service operation to obtain a first similarity metric. The operations further include to determine an indication that indicates whether the first similarity metric satisfies a criteria for the executed field service operation; and to transmit the indication to a network node.

[0008] In other embodiments, a computer program comprising program code is provided to be executed by processing circuitry of a computing device configured to provide feedback on an executed field service operation for a communication network. Execution of the program code causes the computing device to perform operations. The operations include to compare a first model including a first representation of a plurality of semantics of the executed field service operation to a second model including a second representation of a plurality of semantics of a workflow to execute the field service operation to obtain a first similarity metric. The operations further include to determine an indication that indicates whether the first similarity metric satisfies a criteria for the executed field service operation; and to transmit the indication to a network node.

[0009] In other embodiments, a computer program product is provided comprising a non-transitory storage medium including program code to be executed by processing circuitry of a computing device configured to provide feedback on an executed field service operation for a communication network. Execution of the program code causes the computing device to perform operations. The operations include to compare a first model including a first representation of a plurality of semantics of the executed field service operation to a second model including a second representation of a plurality of semantics of a workflow to execute the field service operation to obtain a first similarity metric. The operations further include to determine an indication that indicates whether the first similarity metric satisfies a criteria for the executed field service operation; and to transmit the indication to a network node.

[0010] Certain embodiments may provide one or more of the following technical advantages. Based on the inclusion of the first and second models, and comparison of the models, the method may scale to provide feedback on an executed field service operation (such as installation, maintenance, and/or troubleshooting) for multiple pieces of equipment or components and, thus, performance issues may be avoided in the future. BRIEF DESCRIPTION OF DRAWINGS

[0011] The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this application, illustrate certain non-limiting embodiments of inventive concepts. In the drawings:

[0012] Figure 1 is a block diagram of components of a system in accordance with some embodiments of the present disclosure;

[0013] Figure 2 is a sequence diagram illustrating operations of an example embodiment in accordance with the present disclosure;

[0014] Figure 3 is a schematic diagram illustrating an example of extraction of a semantic representation of modalities and an instruction keyword extraction in accordance with some embodiments of the present disclosure;

[0015] Figures 4A, 4B are a flow chart of operations of a computing device in accordance with some embodiments of the present disclosure;

[0016] Figure 5 is a block diagram of a communication network in accordance with some embodiments;

[0017] Figure 6 is a block diagram of a computing device in accordance with some embodiments of the present disclosure;

[0018] Figure 7 is a block diagram of a network node in accordance with some embodiments of the present disclosure;

[0019] Figure 8 is a block diagram of a host computer communicating with a user equipment in accordance with some embodiments;

[0020] Figure 9 is a block diagram of a virtualization environment in accordance with some embodiments of the present disclosure; and

[0021] Figure 10 is a block diagram of a host computer communicating via a base station with a user equipment over a partially wireless connection in accordance with some embodiments in accordance with some embodiments. DETAILED DESCRIPTION

[0022] Inventive concepts will now be described more fully hereinafter with reference to the accompanying drawings, in which examples of embodiments of inventive concepts are shown. Inventive concepts may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of present inventive concepts to those skilled in the art. It should also be noted that these embodiments are not mutually exclusive. Components from one embodiment may be tacitly assumed to be present/used in another embodiment.

[0023] The following description presents various embodiments of the disclosed subject matter. These embodiments are presented as teaching examples and are not to be construed as limiting the scope of the disclosed subject matter. For example, certain details of the described embodiments may be modified, omitted, or expanded upon without departing from the scope of the described subject matter.

[0024] As used herein, the term "computing device" refers to equipment capable, configured, arranged, and/or operable to provide feedback on an executed field service operation for a communication network. As discussed further herein, examples of computing devices include, but are not limited to, a computer and a User Equipment (UE). The UE may include, e.g., a smart phone, mobile phone, cell phone, Voice over Internet Protocol (VoIP) phone, wireless local loop phone, desktop computer, personal digital assistant (PDA), wireless cameras, gaming console or device, music storage device, playback appliance, wearable terminal device, wireless endpoint, mobile station, tablet, laptop, laptop-embedded equipment (LEE), laptop-mounted equipment (LME), smart device, wireless customer-premise equipment (CPE), vehicle-mounted or vehicle embedded/integrated wireless device, etc. Other examples include any UE identified by the 3rd Generation Partnership Project (3GPP), including a narrow band internet of things (NB-loT) UE, a machine type communication (MTC) UE, and/or an enhanced MTC (eMTC) UE. For example, in distributed multiple-input and multiple-output (D-MIMO), the computing device may include a distributed collection of access points (APs) that cooperate via a central processing unit (CPU), and channel estimation can be done at an AP or centrally at the CPU where channel estimates from all APs in the distributed collection can be combined for precoding or receive combining. Further, a machine learning (ML) model may be distributed at different ones of the APs and one central entity can combine them.

[0025] As used herein, the term "sensory modalities" refers to capturing sensory data (e.g., with a communication device as discussed further herein) including, without limitation, an image, a video, audio data, environmental stimuli data, force feedback data, haptic feedback data, heat map data, subsonic/supersonic audio data, smell data (e.g., the smell of gas from a gas leak), taste data, etc.

[0026] In a communication network, hardware faults may occur. Sometimes, a hardware fault may not be due to an alarm or other internal warning or fault, but rather may be due to an improper hardware instal lation/fau It. Such an improper hardware installation and/or fault may be coupled with environmental parameters that are beyond control of Al or ML models.

[0027] A radio access network (RAN) includes a geographically distributed set of radio base stations, and may be susceptible to these types of faults. In one non-limiting example, radio frequency (RF) weatherproofing used to shield coaxial cables connecting radio units (RUs) with antennas on a tower mast may become loose, torn (e.g., when duct tape is used), misaligned, or broken/unlocked (e.g., an unlocked clamshell type of weatherproofing) due to rain, wind, condensation, etc. which may be combined with improper initial installation. Such improper or faulty installation may lead to, e.g., high bit error rates or complete loss of traffic in one or more RF cables.

[0028] In another non-limiting example, coaxial cables may be bent excessively during installation or over time. Bent coaxial cables may lead to a breakdown of copper inside the cable breaking, which may result in high packet drops or loss of signal.

[0029] In such examples, an initial improper installation may lead to later problems for which the root cause may not be directly avoided/prevented by an automated network operations center (NOC). Time-series predictors, such as recurrent neural networks (RNN)) may predict loss of signal for example, but it can be difficult to prevent such errors in the absence of a verification during installation of the hardware components.

[0030] While instructions may be available on how to properly install hardware, the instructions may be in the form of a workflow, e.g., a series of steps that include one or more of text (e.g., natural language or controlled language), pictures, and/or video. In some approaches, the outcome of such workflow instructions is not verified in a formal way, but instead a manual verification process may rely on knowledge of a person in the field. In another approach, computer vision may be used to detect defects of hardware equipment in manufacturing, which may replace manual verification by a person. See e.g., https://cloud.google.com/blog/products/ai-machine-learning/improve-manufacturing- quality-control-with-visual-inspection-ai (accessed on 28 October 2022); "PCB Component Detection using Computer Vision for Hardware Assurance", Wenwei Zhao, Suprith Gurudu, Shayan Taheri, Shajib Ghosh, Mukhil Azhagan Mallaiyan Sathiaseelan, Navid Asadizanjani, arXiv:2202.08452 (17 February 2022).

[0031] There currently exist certain challenges. While some approaches may include multiple sensory modalities (e.g., images, video, audio, haptic feedback, etc.) to diagnostically evaluate whether an installed piece of equipment works, such approaches may lack a scalable method to supervise, based on inclusion of a ML model(s), that manual installation of a component(s) is properly implemented (e.g., via feedback on a step(s) of an installation process). A scalable feedback/verification operation(s) may be applicable across varied environments (e.g., varied equipment and installation defects/faults). For example, installing a weatherproofing adaptor or bending a cable can include the application of force by engineers. Applying excessive or little force, and/or holding a piece of equipment the wrong way, may result in misplacement and/or breaking of the equipment which may affect customer traffic during operation. Thus, haptic feedback may provide feedback/verification of an appropriate installation of such equipment. In other examples, sound can be another form of feedback/verification. For example, some weatherproofing connectors are "clammed" into place and may make a characteristic sound when appropriately placed (hence the name "clamshell"). Other sounds such as a piece of equipment breaking also may indicate an issue. An installation can include all of these examples (among other examples) and, thus, multiple sensory modalities may be needed to obtain feedback/verify such an installation. Existing approaches, however, may lack a scalable feedback/verification process that can scale across different pieces of equipment using multiple sensory modalities.

[0032] For example, some approaches may include targeting a specific piece of equipment/verification because an operation identifying whether something is faulty with the operation of the equipment is built into a model that is either rule-based or trained using a deep learning type of technique. As a consequence, such approaches may not scale beyond the use case(s) that the model is originally trained/designed for use.

[0033] Certain aspects of the disclosure and their embodiments may provide solutions to these or other challenges. In some embodiments, operations are provided to verify workflow instructions from multimodal observations from a sensory modality using a neuro-symbolic approach. The operations, in some embodiments, include combining a classification capability of a ML model (e.g., a neural network) with a semantic representation capability of symbolic techniques. In one example, the operations include verification of completion of a sequence of operations in a graph of operations of a workflow. In a respective operation of the workflow, a second model (e.g., a "reference belief model") that includes a formalized representation of an expected result is compared to a first model (e.g., an "observed belief model") that represents an observed result. In an example embodiment, the first model includes a neural network, and the neural network converts the observations from different sensory modalities to semantic representations using classification. Subsequently, the semantically annotated observations are compared to reference observations from a second model using a set of procedural conditions and instructions. A deviation of the first model (e.g., the observed belief model) from the second model (e.g., the reference belief model) above a threshold can signify that a person(s) (e.g., an engineer) on-site needs to repeat the operation in the workflow or stop the workflow process altogether. [0034] As used herein, the term "semantics" refers to a description of an executed field service operation(s) and/or a description of a workflow to execute the field service operation(s). For example, the description can be, without limitation, video frames that are converted to a list of detected objects (e.g., using a convolutional neural network), haptic feedback that is converted to a hand pose and a qualitative force indicator (e.g., using a computer vision based approach), and/or audio that is converted into a recognized attribution, such as "plastic breaking" or "clicking" (e.g., using a recurrent neural network and an audio fingerprinting approach).

[0035] Operations of some example embodiments include neuro-symbolic verification of successful completion of operations in a workflow by comparison of a first model including a first representation of a plurality of semantics of an executed field service operation to a second ML model including a second representation of a plurality of semantics of a workflow to execute the field service operation.

[0036] Certain embodiments may provide one or more of the following technical advantages. Inclusion of feedback/verification of successful completion of an operations(s) in a workflow by comparison of a first model (e.g., an observed belief model) to a second model (e.g., a reference belief model) may contribute towards verifying installation of hardware equipment in accordance with the workflow operation(s). As a consequence, technical performance related issues related to a faulty installation of the hardware equipment/component at a future time may be avoided. A further technical advantage may include preservation of privacy as any sensitive data used for reconstructing a scene on-site for the comparison may not be forwarded to a centralized location (e.g., instead processing may be performed within a computing device (e.g., a user equipment (UE)). Moreover, for telecommunication equipment installation, for example, the operations of some embodiments may contribute towards occupational health and safety (OHS) as some dangerous activities may be eliminated or reduced (e.g., tower climbs may be eliminated or reduced).

[0037] Figure 1 is a block diagram of components of a system in accordance with some embodiments of the present disclosure. As illustrated, the components include one or more communication devices 100a - lOOn (any one of which is referred to herein as a communication device 100); a computing device 102; a subscription management node 108; a mobility management node 110; and a network operation center node 112.

[0038] The one or more communication devices 100a - lOOn can capture a plurality of different sensory modalities in an environment proximate a location in which the field service operation is executed. The sensory modalities can include, without limitation, an image, a video, audio data, environmental stimuli data, biomarkers (e.g., pulse, blood pressure, etc.), and/or touch-based input (e.g., force feedback data and/or haptic feedback data), etc. The communication device 100 can be or include, for example, a camera, a microphone, and/or a piezo-electric glove(s). A piezo-electric glove(s), for example, may indicate if the user is applying some force, the volume of the force, and its exact location (e.g., fingers, palm, etc.).

[0039] As illustrated in the example of Figure 1, subscription management (SM) node 108 provides authorization to a requesting computing device 102 regarding whether the computing device 102 is eligible to access workflow instructions (e.g., verify eligibility for workflow resolution). In a third-generation partnership project (3GPP) network, for example, the SM node 108 can be a home subscriber server (HSS) or unified data management (UDM) node, etc. SM node 108 also can be a non-3GPP node and can have different authentication options. For example, SM node 108 can be a server where the communication device 102 is authenticated via basic or digest authentication, Windows NT Lan Manager (NTLM) authentication, Kerberos/Negotiate, etc. While the example embodiment of Figure 1 illustrates a SM node 108, the present disclosure is not so limited and this node may be any network node that includes authentication and authorization operations to authorize the requesting computing 102 regarding whether the computing device 102 is eligible to access workflow instructions.

[0040] As illustrated in the example of Figure 1, the mobility management node

110 triggers the computing device 102 to begin collecting information from one or more of communication devices 100a . . . lOOn. The trigger can occur when the mobility management node 110 senses the computing device 102 is proximate the area. For example, mobility management node 110 can be a mobility management entity (MME) or an access and mobility management function (AMF) node(s) in a 3GPP network.

[0041] In some embodiments, the network itself senses that the computing device 102 is in place, for example when the computing device 102 is handed over to a particular cell that is located close to the area where the hardware-related activity is to take place. In other embodiments, the location of the computing device 102 is determined with at least one of a fifth generation (5G) positioning technique that uses beamforming and multipleuser (MU) multiple input multiple output (MIMO) (MU-MIMO); a short-range technology such as radio-frequency identification (RFID) and Near Field Communications (NFC), etc.

[0042] In another embodiment, the computing device 102 itself can sense whether it is in a location for a hardware related activity, and notify mobility management node 110 by using a satellite positioning technology such as Global Positioning System (GPS).

[0043] In yet another embodiment, the computing device 102 and a communications network collaborate to provide accurate positioning via assisted GPS (A- GPS), in which case the computing device 102 also reports position to the mobility management node 110.

[0044] As illustrated in the example of Figure 1, the network operations center node 112 provides an authorized (e.g., from SM node 108) computing device 102 with operation-by-operation workflow information and receives notifications on whether a workflow operation has been completed. In another embodiment, the workflow also exists in the memory of the computing device 102 (e.g., via a preload by the NOC node 112 beforehand). In this example embodiment, the NOC node 112 is a logical entity, and the logical entity can be within the computing device 102 or in another network node.

[0045] Still referring to Figure 1, data that includes raw values (e.g., raw video frames, audio waves, sensory glove readings, etc.) from one or more communication devices 100 is forwarded to one or more computing devices 102. A computing device 102 can include cellular connectivity and can process this information further. In some embodiments, the computing device(s) 100 and/or the computing device(s) 102 are logical nodes. For example, a communication device 100 can also be a computing device 102 (e.g., a UE that includes a camera and is capturing video) or they can be separate devices (e.g., a smartwatch capturing audio and sending it to the UE via Bluetooth or a "smart" glove doing the same).

[0046] The computing device(s) 102 can include a modality processor 104 and/or a processor 106 that can access/ receive the data including raw values from the communication device(s) 100 and transform the data to a semantic representation. In an example embodiment, video frames are converted to a list of detected objects via use of a first model comprising a convolutional neural network; haptic feedback is converted to a hand pose and a qualitative force indicator (see e.g., Chen W, Yu C, Tu C, et al. "A Survey on Hand Pose Estimation with Wearable Sensors and Computer-Vision-Based Methods". Sensors (Basel). 2020;20(4):1074. Published 2020 Feb 16, doi:10.3390/s20041074); and audio is converted into a recognized attribution, such as "plastic breaking" or "clicking" (e.g., via a first model using a RNN and an audio fingerprinting approach (see e.g., G. Deepsheka, R. Kheerthana, M. Mourina and B.

Bharathi, "Recurrent neural network based Music Recognition using Audio Fingerprinting," 2020 Third International Conference on Smart Systems and Inventive Technology (ICSSIT), 2020, pp. 1-6, doi: 10.1109/ICSSIT48917.2020.9214302). In this example, the results are fed into a processor component (e.g., processor 106). Upon/after mobility management node 110 identifying that computing device 102 is in a position to begin the hardware installation process, hardware installation operations can proceed.

[0047] Hardware installation operations can include, without limitation, requesting eligibility for a workflow resolution (e.g., requesting verification eligibility for workflow resolution from SM node 108); and executing a process (e.g., an algorithm) that compares a first model and a second model (as discussed further herein).

[0048] The workflow W can include a number of N steps: W = {wi, . . . WN). In some embodiments, the N steps are executed in sequence; but the present disclosure is not so limited. For example, in other embodiments, the workflow includes a graph, where a step can branch out to a plurality of subsequent steps depending on different conditions. [0049] In some embodiments, for a respective operation w_x in the workflow w, there is a second model b_wx (e.g., a reference belief model) such that b_wx = {r_wx, i_wx, c_wx}, where r_wx is a reference scene. In some embodiments, the reference scene r_wx is governed by one or more conditions over facts. For example, for a loose clamshell, one condition can be: "Scene has one or more clamshells", and another condition can be "Hexagonal bolt is not directly above clamshell". These example conditions can be represented as:

• fi=has_one_or_more(scene, clamshell)

• f2=directly_above(hexagonal_bolt, clamshell)

[0050] A reference scene can also include one or more rules. In the above example, a rule can be: success(w_x) :- fl(w_x), \+(f2(w_x))

[0051] In this example rule, the \+ is a negation operator meaning that the scene is interpreted successfully if it has one or more clamshells in the frame and that a hexagonal bolt is not directly above a clamshell. It is noted that while the above example rule is expressed using Prolog, the present disclosure is not so limited and includes other expressions of rules.

[0052] Therefore, in the above example, r_wx={fi, fz, success}, where w_x can be the last step in the process where one checks if the weatherproofing(s) is installed correctly. Other reference scenes preceding this step can be, for example, installing the actual weatherproofing, where there can be conditions on how much force to apply and where to weatherproof (e.g., an engineer uses piezo-electric gloves and tactile maps (see e.g., http://stag.csail.mit.edu)).

[0053] Further, in the above example, i_wx are instructions on how to extract objects for the first model (e.g., the observed belief model). In some embodiments, objects are entities comprising conditions, excluding the scene. In this example, other objects to be extracted include the clamshell and hexagonal_bolt. In some embodiments, the objects are linked to output of a sensor(s) of a sensory modality (and to any postprocessing of this output) from the sensory modality of a person (e.g., an engineer) inspecting the scene. In an example, for a person using a camera headset, instructions for extracting the clamshells can be:

• extract_clamshells():

• scene_picture = get_video_frame(IMSI) // IMSI is a unique International Mobile Subscriber Identity that the network can use to obtain the raw video frame

• list_objects = get_objects(scene, "http://192.168.0.2/tower_detector.h5)

• return = filter_objects(list_objects, 90, "clamshell") // where "90" is a probability/confidence of the object detector

[0054] While Prolog is used in the above example instructions for extracting, a procedural language may be used instead. It is noted, however, that these instructions can still be semantically linked in Prolog. Similar instructions can be provided for extracting the hexagonal_bolt object.

[0055] In addition to the extraction of the objects, instructions can further include instructions on how to interpret the conditions. In the above example, there are two conditions: has_one_or_more, and directly_above.

For has_one_or_more:

• clamshells=extract_clamshells()

• if clamshells. size >= 1: return true; else false;

For directly_above:

• clamshells=extract_clamshells()

• hexagonal_bolts=extract_hexagonal_bolts()

• for hex_bolt in hexagonal_bolts:

■ for clamshell in clamshells: o if(hex_bolt.bottomLeft.y - clamshell.y <= 0) // where 0 is a pixel number and can also be greater than 0, e.g., 5 pixels to account for noise/error.

• return true; return false; [0056] The first model (e.g., current belief model) can now be constructed, and the success rule can be checked.

[0057] In the example above, c_wx is a criteria(s) to be met in order for the workflow step to be verified successfully and the process can move to next step. In the above example, the criteria is as follows: success(r_wx) == success(current_scene), where current_scene is the scene extracted according to information contained in i_wx.

[0058] Figure 2 is a sequence diagram illustrating operations of an example embodiment. As illustrated in the example of Figure 2, operations are shown for a computing device 102, a mobility management node 110, a SM node 108, and a NOC node 112. Computing device 102 can include a modality processor 104, a processor 106, and a radio 200. It is noted that modality process or 102 and/or processorl304 can be a logical entity and can be placed either in a core network, or can be part of the computing device 102 as shown in Figure 2, depending on privacy requirements for example.

[0059] In the example of Figure 2, computing device 102 can be a UE providing data, e.g., audio and/or video data. Modality processor 104 receives raw data from a communication device 100 (e.g., audio, video, sound) and classifies the raw data into semantic representations. Processor 106 decides whether a workflow step has been completed or not based on comparison of a synthesized first model (e.g., an observed belief model) to a second model (e.g., to a reference belief model). Radio 200 in this example includes a cellular transmitter/receiver of the computing device 102 (e.g., a network stack processor).

[0060] Referring to Figure 2, the process begins by a trigger 202, e.g., a condition that indicates to a person on site to do a hardware installation. For example, the person carries a computing device 102; and the computing device 102 is identified by the mobility management node 110 as being proximate to an area of service. Various embodiments can include a trigger. In one embodiment, detection of computing device 102 is performed using the mobility management node 110 (e.g., a MME or AMF) based on attachment on the cell that is close to the equipment to be serviced (or on a neighboring cell). Figure 2 illustrates this example embodiment which may be suited, for example, for dense deployments where cells have limited range and therefore the location of the computing device 102 may be better identified. In this example embodiment, computing device 102 is detected based on a handover/initial attach 204 that includes an IMSI or a Temporary IMSI (T-IMSI) between radio 200 of computing device 102 and mobility management node 110. The detection triggers mobility management node 110 to communicate with SM node 108, in operation 206, to check eligibility of computing device 102 having the IMSI or T-IMSI for a workflow resolution. In operation 208, SM node 108 acknowledges to mobility management node 110 the eligibility. Mobility management node 110, in operation 210, transmits to the radio 200 of computing device 102 an indication that the computing device 102 is in position (e.g., a CelllD, and the IMSI or T-IMSI). Radio 200 forwards 212 to processor 106 of computing device 102 the indication that the computing device 102 is in position.

[0061] In another embodiment, detection of computing device 102 is determined by a mobile network using a positioning technique, such as 5G positioning. See e.g., https://www.ericsson.com/en/blog/2020/5g-positioning-what-you-need-to-know. In yet another embodiment, detection of computing device 102 is done by computing device 102 itself, for example, using a satellite positioning technology such a global positioning system (GPS).

[0062] Referring again to Figure 2, once computing device 102 has been verified to be close to the area where the hardware is to be installed/troubleshot, in operations 214- 220, computing device 102 retrieves workflow W from NOC node 112.

[0063] In loops 224 and 226, of a field services session 222, for a respective current step(s) w in that workflow W, computing device 102 creates the first model (e.g., the observed belief model). That is, as previously discussed, in operation 228, processor 106 of computing device 102 constructs a reference scene current w.r; requests 230 information from the modality processor 104 for the current scene (e.g., current w.current_scene); receives 232 the information from modality processor 104; and constructs 234 objects and conditions for the current. current scene. [0064] In operation 236, the first model is compared to the second model (e.g., the reference belief model) based on the success criteria. In alternative operations 238, the results of the comparison are sent to the NOC node 112 in operations 244 and 246 when the field service operation step succeeded; and the inner loop 226 breaks when the comparison is successful. Thus, when the result is that the field service operation succeeded, the network and/or a person in the field is notified and the field service operation is completed. In some embodiments, in alternative 238, if the comparison for a given step w indicates a mismatch in operations 240 and 242, then the NOC node 112 can order the computing device 102 to repeat the workflow step (that is, recreate the first model and compare it to the second model as shown in operations 228-236), or may cancel the work order, e.g., when damage already exists.

[0065] It is noted that conditions for the second model can be either hardcoded (e.g., by a workflow author) and sent via the NOC node 112 to the computing device 102, as shown in Figure 2; or the second model can be learned using, for example, a neuro- symbolic approach.

[0066] When the second model is learned, written instructions (e.g., in a natural or controlled language) may already exist as instructions (e.g., in a knowledge base). For example, documents including customer product information (CPI) can include information on proper product operation and installation written in natural language.

[0067] In some embodiments, learning the second model includes: (1) the natural language workflows are decomposed into semantic representations using, for example, natural language processing (NLP) and keyword extraction; (2) modality processor 104 provides a list of semantic representations observed in a scene (e.g., audio source, classified objects in a video frame, hand pose, etc., depending on the sensors used); and (3) the semantic representations from operations 1 and 2 are compared. Comparison can be explicit (e.g., using lexical matching); and/or can be implicit, for example through synonym analysis and/or hierarchical clustering (e.g., semantic matching). Figure 3 is a schematic diagram illustrating an example of extraction 300 of a semantic representation of modalities in an observation vector, and instruction keyword extraction 308. [0068] In the example of Figure 3, modality transformation 300 results in an observation vector 306, and instruction keyword extraction 308 results in an instruction vector 312 is shown. The vectors 306, 312 are compared using, e.g., a similarity measure either in pure form or after post processing (e.g., synonym analysis or semantic abstraction).

[0069] Modality transformation 300 includes transformation of data from one or more sensory modalities into observation vector 306. In the example of Figure 3, the data includes audio wave data 302a, video frames/images 302b, and force sensor readings 302c. Audio wave data 302a is input to a first model 304a comprising a long short-term memory (LSTM)/RNN. The first model 304a outputs a semantic representation of an observation: plastic clicking. Video frame(s)/image(s) 302b is input to a first model 304b comprising a convolutional neural network (CNN). The first model 304b outputs sematic representations of observations: radio unit, clamshell, antenna, connector nut. Force sensor reading(s) 302c is input to a first model 304c comprising a deep neural network (DNN). The first model 304c outputs a semantic representation of an observation: hand pose. The extracted semantic representations are included in observation vector 306.

[0070] Instruction keyword extraction 308 includes keyword extraction 310 from a workflow (e.g., from a document that includes images and natural language as shown in the example of Figure 3). The extracted keywords are included in instruction vector 312 (e.g., "connector nut", "squeeze holder shut", and "secure bale" as shown in the example of Figure 3).

[0071] Before proceeding further, the comparison can be repeated for additional data observed by the modality transformation 300 over a time period, and multiple comparisons can be made. In some embodiments, the closest matching workflow instruction to these comparisons is the one most likely to represent a correct instruction for the scene observed.

[0072] In some embodiments, if a workflow is found among those in the documentation database, then it is further analyzed to identify what the relationships between the different semantic representations of the scene are (e.g., relative position of the detected objects). Natural language processing (NLP) can be used to construct entity relationship diagrams between parts of speech in the workflow (e.g., subject - predicate - object), and these can be later attributed to objects from a semantic representation. In this way, conditions can be learned rather than be hardcoded. The conditions can include transformations from a graph or subgraph to a new graph or a single fact (e.g., summarizing an entire structure with a single inferred fact (such as, clamshell is closed) when sufficiently similar.

[0073] Figures 4A and 4B are a flowchart of operations of a computing device 102, 6200 (implemented using the structure of the block diagram of Figure 6) in accordance with some embodiments of the present disclosure. For example, modules may be stored in memory 6210 of Figure 6, and these modules may provide instructions so that when the instructions of a module are executed by respective computing device processing circuitry 6202a and/or 6202b, processing circuitry 6202a and/or 6202b performs respective operations of the flow chart.

[0074] Referring to Figure 4A, a computer-implemented method performed by the computing device to provide feedback on an executed field service operation for a communications network is provided. The method includes comparing (410) a first model including a first representation of a plurality of semantics of the executed field service operation to a second model including a second representation of a plurality of semantics of a workflow to execute the field service operation to obtain a first similarity metric. The method further includes determining (412) an indication that indicates whether the first similarity metric satisfies a criteria for the executed field service operation; and transmitting (414) the indication to a network node. The first similarity metric can include, without limitation, a distance function (e.g., cosine similarity of two vectors), a Euclidean distance, a correlation similarity, a mean squared difference, etc.

[0075] The first model and the second model, respectively, can include at least one of a ML model, a symbolic model, an ontology-based model, and a statistical model.

[0076] In some embodiments, the method further includes accessing (402) data associated with the executed field service operation. The data is obtained from at least one of a plurality of sensory modalities via a communication device proximate a location in which the field service operation is executed. The method of some embodiments further includes processing (404) the accessed data associated with the executed field service operation to transform the accessed data into the first representation of the plurality of the semantics of the executed field service operation; and providing (406) the first representation to the first model.

[0077] The data obtained from at least one of a plurality of sensory modalities can include an image, a video, audio data, environmental stimuli data from an environment proximate the location, force feedback data, and haptic feedback data.

[0078] In some embodiments, the method further includes processing (400) a trigger that indicates to the computing device to access the workflow to execute the field service operation. The trigger can include an indication that the computing device is in a location to execute the field service operation. The indication that the computing device is in the location can be determined by the computing device.

[0079] The communications network can include a telecommunication network and the indication that the computing device is in the location can be received from a network node and can be based on an attachment of the computing device to a cell in the telecommunication network. The indication that the computing device is in the location that can be received from a network node, can be based on at least one of (i) a 5G positioning technique that determined the location of the computing device and (ii) a satellite positioning service that that determined the location of the computing device. [0080] In some embodiments, the second model comprises respective operations in the workflow to execute the field service operation, and a respective operation comprises (i) a reference scene governed by at least one condition over a fact for the field service operation, (ii) an instruction on how to extract an object and a first scene for the first model, and (iii) the criteria comprising at least one rule to be satisfied in the respective operation for the first scene, as previously discussed. The comparing can include a comparison for the respective operation of (i) a first vector from the first model including the extracted object and first scene, with (ii) a second vector from the second model including the reference scene, and (iii) obtaining the first similarity metric based on the comparison. The similarity metric can include a measure of the similarity of the first vector to the second vector based on the rule.

[0081] Referring to Figure 4B, in some embodiments the method further includes accessing (416) additional data associated with the executed field service operation. The additional data can be obtained from at least one of a plurality of the sensory modalities. The method of some embodiments further includes processing (418) the accessed additional data associated with the executed field service operation to transform the accessed additional data into an addition to the first representation; providing (420) the addition to the first representation to the first model; comparing (422) the first model comprising the addition to the first representation to the second model to obtain a second similarity metric; and determining (424) the indication that indicates whether one of the first and the second similarity metrics satisfies the criteria for the executed field service operation.

[0082] The procedure for execution of the field service operation can include a series of respective operations in at least one of a natural language, an artificial language, and a notation; and the method of some embodiments further includes constructing (408) the second model based on extraction of at least one of the natural language, the artificial language, and the notation into the second representation of the plurality of semantics of the workflow to execute of the field service operation. The executed field service operation can include at least one of an installation, an upgrade, a maintenance operation, and a troubleshooting operation of a component. The component can be at least one of a RF weatherproofing component, a coaxial cable, a non-coaxial cable, a cable nut, a bolt, a radio unit, and an antenna.

[0083] In some embodiments, the feedback includes the indication and the indication includes at least one of a verification that the executed field service operation satisfies the workflow and information about why the workflow is not satisfied.

[0084] In some embodiments, the at least one of a plurality of sensory modalities includes the communication device configured to collect the data, an artificial intelligence (Al) based headset or glasses configured to collect the data, and a sensory glove configured to collect the data.

[0085] Various operations from the flow chart of Figures 4A and 4B may be optional with respect to some embodiments of computing devices and related methods. For example, operations of blocks 400-408 and 416-424 may be optional.

[0086] Example embodiments of the methods of the present disclosure may be implemented in a communication network that includes, without limitation a telecommunication network, as illustrated on Figure 5. The telecommunications network 5102 may include an access network 5104, such as a RAN, and a core network 5106, which includes one or more core network nodes 5108. The access network 5104 may include one or more access nodes 5110A. 5110B, such as network nodes (e.g., base stations), or any other similar 3GPP access node or non-3GPP access point. The network nodes 5110 facilitate direct or indirect connection of computing devices 5112A-D (e.g., a UE), such as by and/or other computing devices to the core network 5106 over one or more wireless connections.

[0087] Example wireless communications over a wireless connection include transmitting and/or receiving wireless signals using electromagnetic waves, radio waves, infrared waves, and/or other types of signals suitable for conveying information without the use of wires, cables, or other material conductors. Moreover, in different embodiments, the network may include any number of wired or wireless networks, network nodes, UEs, computing devices, and/or any other components or systems that may facilitate or participate in the communication of data and/or signals whether via wired or wireless connections. The network may include and/or interface with any type of communication, telecommunication, data, cellular, radio network, and/or other similar type of system.

[0088] As a whole, the network 5100 enables connectivity between the computing devices 5112 and other devices. In that sense, the network 5100 may be configured to operate according to predefined rules or procedures, such as specific standards that include, but are not limited to: Global System for Mobile Communications (GSM); Universal Mobile Telecommunications System (UMTS); Long Term Evolution (LTE), and/or other suitable 2G, 3G, 4G, 5G standards, or any applicable future generation standard (e.g., 6G); wireless local area network (WLAN) standards, such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standards (WiFi); and/or any other appropriate wireless communication standard, such as the Worldwide Interoperability for Microwave Access (WiMax), Bluetooth, Z-Wave, Near Field Communication (NFC) ZigBee, LiFi, and/or any low-power wide-area network (LPWAN) standards such as LoRa and Sigfox.

[0089] In some examples, the telecommunication network 5102 is a cellular network that implements 3GPP standardized features. Accordingly, the telecommunications network 5102 may support network slicing to provide different logical networks to different devices that are connected to the telecommunication network. For example, the telecommunications network 5102 may provide Ultra Reliable Low Latency Communication (URLLC) services to some computing devices (e.g., UEs), while providing Enhanced Mobile Broadband (eMBB) services to other computing devices, and/or Massive Machine Type Communication (mMTC)/Massive loT services to yet further computing devices.

[0090] In some examples, the network 5100 is not limited to including a RAN, and rather includes any that includes any programmable/configurable decentralized access point or network element that also records data from performance measurement points in the network 5100.

[0091] In some examples, computing devices are configured as a computer without radio/baseband, etc. attached.

[0092] The method of the present disclosure is amenable to distributed node and cloud implementation. Various distributed processing options may be used that suit data source, storage, compute, and coordination. For example, data sampling may be done at a communication device, with data analysis, model creation, model comparison, etc. performed at the computing device or at a cloud server/node.

[0093] Methods of the present disclosure may be performed by a computing device (e.g., any computing device 5112A-D of Figure 5 (one or more of which may be generally referred to as computing device 5112) implemented using the structure of computing device 6200 of Figure 6).

[0094] Referring to Figure 6, as previously discussed, a computing device can include computing device 102 of Figure 1, computing device 5112 of Figure 5, or computing device 6200 of Figure 6. The computing device includes equipment capable, configured, arranged, and/or operable to provide feedback on an executed field service operation for a communication network. Examples of computing devices include, but are not limited to, a computer and a UE. In some embodiments, the computing device 6200 includes processing circuitry 6202a, 6202b that is operatively coupled to a memory 6210, and/or any other component, or any combination thereof. Certain computing devices may utilize all or a subset of the components shown in Figure 6. The level of integration between the components may vary from one computing device to another computing device. Further, certain computing devices may contain multiple instances of a component, such as multiple processors, memories, transceivers, transmitters, receivers, etc.

[0095] The processing circuitry 6202a, 6202b is configured to process instructions and data and may be configured to implement any sequential state machine operative to execute instructions stored as machine-readable computer programs in the memory. The processing circuitry 6202a, 6202b may be implemented as one or more hardware- implemented state machines (e.g., in discrete logic, field-programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc.); programmable logic together with appropriate firmware; one or more stored computer programs, general-purpose processors, such as a microprocessor or digital signal processor (DSP), together with appropriate software; or any combination of the above. For example, the processing circuitry 6202a, 6202b may include multiple central processing units (CPUs).

[0096] The memory 6210 may be or be configured to include memory such as random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic disks, optical disks, hard disks, removable cartridges, flash drives, and so forth. In one example, the memory 6210 includes one or more application programs, such as an operating system, web browser application, a widget, gadget engine, or other application, and corresponding data. The memory 6210 may store, for use by the computing device 6200, any of a variety of various operating systems or combinations of operating systems.

[0097] The memory 6210 may be configured to include a number of physical drive units, such as redundant array of independent disks (RAID), flash memory, USB flash drive, external hard disk drive, thumb drive, pen drive, key drive, high-density digital versatile disc (HD-DVD) optical disc drive, internal hard disk drive, Blu-Ray optical disc drive, holographic digital data storage (HDDS) optical disc drive, external mini-dual in-line memory module (DIMM), synchronous dynamic random access memory (SDRAM), external micro-DIMM SDRAM, smartcard memory such as tamper resistant module in the form of a universal integrated circuit card (UICC) including one or more subscriber identity modules (SIMs), such as a USIM and/or ISIM, other memory, or any combination thereof. The UICC may for example be an embedded UICC (eUlCC), integrated UICC (iUICC) or a removable UICC commonly known as 'SIM card.' The memory 6210 may allow the computing device 6200 to access instructions, application programs and the like, stored on transitory or non-transitory memory media, to off-load data, or to upload data. An article of manufacture, such as one utilizing a network may be tangibly embodied as or in the memory 6210, which may be or comprise a device-readable storage medium.

[0098] The processing circuitry 6202a, 6202b may be configured to communicate with an access network or other network using a communication interface 6212. The communication interface may comprise one or more communication subsystems and may include or be communicatively coupled to an optional antenna. The communication interface 6212 may include one or more transceivers used to communicate, such as by communicating with one or more remote transceivers of another device capable of wireless communication (e.g., another computing device or a network node). Each transceiver may include a transmitter and/or a receiver appropriate to provide network communications (e.g., optical, electrical, frequency allocations, and so forth). Moreover, the optional transmitter and receiver may be coupled to one or more optional antennas and may share circuit components, software or firmware, or alternatively be implemented separately.

[0099] In the illustrated embodiment, communication functions of the communication interface 6212 may include cellular communication, Wi-Fi communication, LPWAN communication, data communication, voice communication, multimedia communication, short-range communications such as Bluetooth, near-field communication, location-based communication such as the use of GPS to determine a location, another like communication function, or any combination thereof.

Communications may be implemented in according to one or more communication protocols and/or standards, such as IEEE 802.11, Code Division Multiplexing Access (CDMA), Wideband Code Division Multiple Access (WCDMA), GSM, LTE, New Radio (NR), UMTS, WiMax, Ethernet, transmission control protocol/internet protocol (TCP/IP), synchronous optical networking (SONET), Asynchronous Transfer Mode (ATM), QUIC, Hypertext Transfer Protocol (HTTP), and so forth.

[00100] Figure 7 shows a network node 7300 in accordance with some embodiments. As used herein, network node refers to equipment capable, configured, arranged and/or operable to communicate directly or indirectly with a computing device and/or with other network nodes or equipment, in a communication network. Examples of network nodes include, but are not limited to, subscription

[00101] management nodes, mobility management nodes, network operations center nodes, access points (APs) (e.g., radio access points), base stations (BSs) (e.g., radio base stations, Node Bs, evolved Node Bs (eNBs) and NR NodeBs (gNBs)).

[00102] Base stations may be categorized based on the amount of coverage they provide (or, stated differently, their transmit power level) and so, depending on the provided amount of coverage, may be referred to as femto base stations, pico base stations, micro base stations, or macro base stations. A base station may be a relay node or a relay donor node controlling a relay. A network node may also include one or more (or all) parts of a distributed radio base station such as centralized digital units and/or remote radio units (RRUs), sometimes referred to as Remote Radio Heads (RRHs). Such remote radio units may or may not be integrated with an antenna as an antenna integrated radio. Parts of a distributed radio base station may also be referred to as nodes in a distributed antenna system (DAS).

[00103] Other examples of network nodes include multiple transmission point (multi-TRP) 5G access nodes, multi-standard radio (MSR) equipment such as MSR BSs, network controllers such as radio network controllers (RNCs) or base station controllers (BSCs), base transceiver stations (BTSs), transmission points, transmission nodes, multi- cell/multicast coordination entities (MCEs), Operation and Maintenance (O&M) nodes, Operations Support System (OSS) nodes, Self-Organizing Network (SON) nodes, positioning nodes (e.g., Evolved Serving Mobile Location Centers (E-SMLCs)), and/or Minimization of Drive Tests (MDTs).

[00104] The network node 7300 includes a processing circuitry 7302, a memory 7304, a communication interface 7306, and a power source 7308. The network node 7300 may be composed of multiple physically separate components (e.g., a NodeB component and a RNC component, or a BTS component and a BSC component, etc.), which may each have their own respective components. In certain scenarios in which the network node 7300 comprises multiple separate components (e.g., BTS and BSC components), one or more of the separate components may be shared among several network nodes. For example, a single RNC may control multiple NodeBs. In such a scenario, each unique NodeB and RNC pair, may in some instances be considered a single separate network node. In some embodiments, the network node 7300 may be configured to support multiple radio access technologies (RATs). In such embodiments, some components may be duplicated (e.g., separate memory 7304 for different RATs) and some components may be reused (e.g., a same antenna 7310 may be shared by different RATs). The network node 7300 may also include multiple sets of the various illustrated components for different wireless technologies integrated into network node 7300, for example GSM, WCDMA, LTE, NR, WiFi, Zigbee, Z-wave, LoRaWAN, Radio Frequency Identification (RFID) or Bluetooth wireless technologies. These wireless technologies may be integrated into the same or different chip or set of chips and other components within network node 7300.

[00105] The processing circuitry 7302 may comprise a combination of one or more of a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application-specific integrated circuit, field programmable gate array, or any other suitable computing device, resource, or combination of hardware, software and/or encoded logic operable to provide, either alone or in conjunction with other network node 7300 components, such as the memory 7304, to provide network node 7300 functionality. [00106] In some embodiments, the processing circuitry 7302 includes a system on a chip (SOC). In some embodiments, the processing circuitry 7302 includes one or more of radio frequency (RF) transceiver circuitry 7312 and baseband processing circuitry 7314. In some embodiments, the radio frequency (RF) transceiver circuitry 7312 and the baseband processing circuitry 7314 may be on separate chips (or sets of chips), boards, or units, such as radio units and digital units. In alternative embodiments, part or all of RF transceiver circuitry 7312 and baseband processing circuitry 7314 may be on the same chip or set of chips, boards, or units.

[00107] The memory 7304 may comprise any form of volatile or non-volatile computer-readable memory including, without limitation, persistent storage, solid-state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), mass storage media (for example, a hard disk), removable storage media (for example, a flash drive, a Compact Disk (CD) or a Digital Video Disk (DVD)), and/or any other volatile or non-volatile, non-transitory device- readable and/or computer-executable memory devices that store information, data, and/or instructions that may be used by the processing circuitry 7302. The memory 7304 may store any suitable instructions, data, or information, including a computer program, software, an application including one or more of logic, rules, code, tables, and/or other instructions capable of being executed by the processing circuitry 7302 and utilized by the network node 7300. The memory 7304 may be used to store any calculations made by the processing circuitry 7302 and/or any data received via the communication interface 7306. In some embodiments, the processing circuitry 7302 and memory 7304 is integrated.

[00108] The communication interface 7306 is used in wired or wireless communication of signaling and/or data between a network node, access network, and/or UE. As illustrated, the communication interface 7306 comprises port(s)/terminal(s) 7316 to send and receive data, for example to and from a network over a wired connection. The communication interface 7306 also includes radio front-end circuitry 7318 that may be coupled to, or in certain embodiments a part of, the antenna 7310. Radio front-end circuitry 7318 comprises filters 7320 and amplifiers 7322. The radio front-end circuitry 7318 may be connected to an antenna 7310 and processing circuitry 7302. The radio frontend circuitry may be configured to condition signals communicated between antenna 7310 and processing circuitry 7302. The radio front-end circuitry 7318 may receive digital data that is to be sent out to other network nodes or UEs via a wireless connection. The radio front-end circuitry 7318 may convert the digital data into a radio signal having the appropriate channel and bandwidth parameters using a combination of filters 7320 and/or amplifiers 7322. The radio signal may then be transmitted via the antenna 7310. Similarly, when receiving data, the antenna 7310 may collect radio signals which are then converted into digital data by the radio front-end circuitry 7318. The digital data may be passed to the processing circuitry 7302. In other embodiments, the communication interface may comprise different components and/or different combinations of components.

[00109] In certain alternative embodiments, the network node 7300 does not include separate radio front-end circuitry 7318, instead, the processing circuitry 7302 includes radio front-end circuitry and is connected to the antenna 7310. Similarly, in some embodiments, all or some of the RF transceiver circuitry 7312 is part of the communication interface 7306. In still other embodiments, the communication interface 7306 includes one or more ports or terminals 7316, the radio front-end circuitry 7318, and the RF transceiver circuitry 7312, as part of a radio unit (not shown), and the communication interface 7306 communicates with the baseband processing circuitry 7314, which is part of a digital unit (not shown). [00110] The antenna 7310 may include one or more antennas, or antenna arrays, configured to send and/or receive wireless signals. The antenna 7310 may be coupled to the radio front-end circuitry 7318 and may be any type of antenna capable of transmitting and receiving data and/or signals wirelessly. In certain embodiments, the antenna 7310 is separate from the network node 7300 and connectable to the network node 7300 through an interface or port.

[00111] The antenna 7310, communication interface 7306, and/or the processing circuitry 7302 may be configured to perform any receiving operations and/or certain obtaining operations described herein as being performed by the network node. Any information, data and/or signals may be received from a UE, another network node and/or any other network equipment. Similarly, the antenna 7310, the communication interface 7306, and/or the processing circuitry 7302 may be configured to perform any transmitting operations described herein as being performed by the network node. Any information, data and/or signals may be transmitted to a UE, another network node and/or any other network equipment.

[00112] The power source 7308 provides power to the various components of network node 7300 in a form suitable for the respective components (e.g., at a voltage and current level needed for each respective component). The power source 7308 may further comprise, or be coupled to, power management circuitry to supply the components of the network node 7300 with power for performing the functionality described herein. For example, the network node 7300 may be connectable to an external power source (e.g., the power grid, an electricity outlet) via an input circuitry or interface such as an electrical cable, whereby the external power source supplies power to power circuitry of the power source 7308. As a further example, the power source 7308 may comprise a source of power in the form of a battery or battery pack which is connected to, or integrated in, power circuitry. The battery may provide backup power should the external power source fail.

[00113] Embodiments of the network node 7300 may include additional components beyond those shown in Figure 7 for providing certain aspects of the network node's functionality, including any of the functionality described herein and/or any functionality necessary to support the subject matter described herein. For example, the network node 7300 may include user interface equipment to allow input of information into the network node 7300 and to allow output of information from the network node 7300. This may allow a user to perform diagnostic, maintenance, repair, and other administrative functions for the network node 7300.

[00114] Figure 8 is a block diagram of a host 8400, which may be an embodiment of the host 5116 of Figure 5, in accordance with various aspects described herein. As used herein, the host 8400 may be or comprise various combinations hardware and/or software, including a standalone server, a blade server, a cloud-implemented server, a distributed server, a virtual machine, container, or processing resources in a server farm. The host 8400 may provide one or more services to one or more UEs.

[00115] The host 8400 includes processing circuitry 8402 that is operatively coupled via a bus 8404 to an input/output interface 8406, a network interface 8408, a power source 8410, and a memory 8412. Other components may be included in other embodiments. Features of these components may be substantially similar to those described with respect to the devices of previous figures, such as Figures 8 and 9, such that the descriptions thereof are generally applicable to the corresponding components of host 8400.

[00116] The memory 8412 may include one or more computer programs including one or more host application programs 8414 and data 8416, which may include user data, e.g., data generated by a UE for the host 8400 or data generated by the host 8400 for a UE. Embodiments of the host 8400 may utilize only a subset or all of the components shown. The host application programs 8414 may be implemented in a container-based architecture and may provide support for video codecs (e.g., Versatile Video Coding (VVC), High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC), MPEG, VP9) and audio codecs (e.g., FLAC, Advanced Audio Coding (AAC), MPEG, G.711), including transcoding for multiple different classes, types, or implementations of UEs (e.g., handsets, desktop computers, wearable display systems, heads-up display systems). The host application programs 8414 may also provide for user authentication and licensing checks and may periodically report health, routes, and content availability to a central node, such as a device in or on the edge of a core network. Accordingly, the host 8400 may select and/or indicate a different host for over-the-top services for a UE. The host application programs 8414 may support various protocols, such as the HTTP Live Streaming (HLS) protocol, Real- Time Messaging Protocol (RTMP), Real-Time Streaming Protocol (RTSP), Dynamic Adaptive Streaming over HTTP (MPEG-DASH), etc.

[00117] Figure 9 is a block diagram illustrating a virtualization environment 9500 in which functions implemented by some embodiments may be virtualized. In the present context, virtualizing means creating virtual versions of apparatuses or devices which may include virtualizing hardware platforms, storage devices and networking resources. As used herein, virtualization can be applied to any device described herein, or components thereof, and relates to an implementation in which at least a portion of the functionality is implemented as one or more virtual components. Some or all of the functions described herein may be implemented as virtual components executed by one or more virtual machines (VMs) implemented in one or more virtual environments 9500 hosted by one or more of hardware nodes, such as a hardware computing device that operates as a second computing device (e.g., a network node), a computing device (e.g., a UE), core network node, or host. Further, in embodiments in which the virtual node does not require radio connectivity (e.g., a core network node or host), then the node may be entirely virtualized. [00118] Applications 9502 (which may alternatively be called software instances, virtual appliances, network functions, virtual nodes, virtual network functions, etc.) are run in the virtualization environment 9500 to implement some of the features, functions, and/or benefits of some of the embodiments disclosed herein.

[00119] Hardware 9504 includes processing circuitry, memory that stores software and/or instructions executable by hardware processing circuitry, and/or other hardware devices as described herein, such as a network interface, input/output interface, and so forth. Software may be executed by the processing circuitry to instantiate one or more virtualization layers 9506 (also referred to as hypervisors or virtual machine monitors (VMMs)), provide VMs 9508a and 9508b (one or more of which may be generally referred to as VMs 9508), and/or perform any of the functions, features and/or benefits described in relation with some embodiments described herein. The virtualization layer 9506 may present a virtual operating platform that appears like networking hardware to the VMs 9508.

[00120] The VMs 9508 comprise virtual processing, virtual memory, virtual networking or interface and virtual storage, and may be run by a corresponding virtualization layer 9506. Different embodiments of the instance of a virtual appliance 9502 may be implemented on one or more of VMs 9508, and the implementations may be made in different ways. Virtualization of the hardware is in some contexts referred to as network function virtualization (NFV or VNF). NFV may be used to consolidate many network equipment types onto industry standard high volume server hardware, physical switches, and physical storage, which can be located in data centers, and customer premise equipment.

[00121] In the context of NFV, a VM 9508 may be a software implementation of a physical machine that runs programs as if they were executing on a physical, nonvirtualized machine. Each of the VMs 9508, and that part of hardware 9504 that executes that VM, be it hardware dedicated to that VM and/or hardware shared by that VM with others of the VMs, forms separate virtual network elements. Still in the context of NFV, a virtual network function is responsible for handling specific network functions that run in one or more VMs 9508 on top of the hardware 9504 and corresponds to the application 9502.

[00122] Hardware 9504 may be implemented in a standalone network node with generic or specific components. Hardware 9504 may implement some functions via virtualization. Alternatively, hardware 9504 may be part of a larger cluster of hardware (e.g. such as in a data center or CPE) where many hardware nodes work together and are managed via management and orchestration 9510, which, among others, oversees lifecycle management of applications 9502. In some embodiments, hardware 9504 is coupled to one or more radio units that each include one or more transmitters and one or more receivers that may be coupled to one or more antennas. Radio units may communicate directly with other hardware nodes via one or more appropriate network interfaces and may be used in combination with the virtual components to provide a virtual node with radio capabilities, such as a radio access node or a base station. In some embodiments, some signaling can be provided with the use of a control system 9512 which may alternatively be used for communication between hardware nodes and radio units.

[00123] Figure 10 shows a communication diagram of a host 10602 communicating via a network node 10604 with a UE 10606 over a partially wireless connection in accordance with some embodiments. Example implementations, in accordance with various embodiments, of the UE (such as a computing device 5112a of Figure 5 and/or computing device 6200 of Figure 6), network node (such as network node 5110a of Figure 5 and/or network node 7300 of Figure 7), and host (such as host 5116 of Figure 5 and/or host 8400 of Figure 8) discussed in the preceding paragraphs will now be described with reference to Figure 10.

[00124] Like host 8400, embodiments of host 10602 include hardware, such as a communication interface, processing circuitry, and memory. The host 10602 also includes software, which is stored in or accessible by the host 10602 and executable by the processing circuitry. The software includes a host application that may be operable to provide a service to a remote user, such as the UE 10606 connecting via an over-the-top (OTT) connection 10650 extending between the UE 10606 and host 10602. In providing the service to the remote user, a host application may provide user data which is transmitted using the OTT connection 10650.

[00125] The network node 10604 includes hardware enabling it to communicate with the host 10602 and UE 10606. The connection 10660 may be direct or pass through a core network (like core network 5106 of Figure 5) and/or one or more other intermediate networks, such as one or more public, private, or hosted networks. For example, an intermediate network may be a backbone network or the Internet. [00126] The UE 10606 includes hardware and software, which is stored in or accessible by UE 10606 and executable by the UE's processing circuitry. The software includes a client application, such as a web browser or operator-specific "app" that may be operable to provide a service to a human or non-human user via UE 10606 with the support of the host 10602. In the host 10602, an executing host application may communicate with the executing client application via the OTT connection 10650 terminating at the UE 10606 and host 10602. In providing the service to the user, the UE’s client application may receive request data from the host's host application and provide user data in response to the request data. The OTT connection 10650 may transfer both the request data and the user data. The UE's client application may interact with the user to generate the user data that it provides to the host application through the OTT connection 10650.

[00127] The OTT connection 10650 may extend via a connection 10660 between the host 10602 and the network node 10604 and via a wireless connection 10670 between the network node 10604 and the UE 10606 to provide the connection between the host 10602 and the UE 10606. The connection 10660 and wireless connection 10670, over which the OTT connection 10650 may be provided, have been drawn abstractly to illustrate the communication between the host 10602 and the UE 10606 via the network node 10604, without explicit reference to any intermediary devices and the precise routing of messages via these devices.

[00128] As an example of transmitting data via the OTT connection 10650, in step 10608, the host 10602 provides user data, which may be performed by executing a host application. In some embodiments, the user data is associated with a particular human user interacting with the UE 10606. In other embodiments, the user data is associated with a UE 10606 that shares data with the host 10602 without explicit human interaction. In step 10610, the host 10602 initiates a transmission carrying the user data towards the UE 10606. The host 10602 may initiate the transmission responsive to a request transmitted by the UE 10606. The request may be caused by human interaction with the UE 10606 or by operation of the client application executing on the UE 10606. The transmission may pass via the network node 10604, in accordance with the teachings of the embodiments described throughout this disclosure. Accordingly, in step 10612, the network node 10604 transmits to the UE 10606 the user data that was carried in the transmission that the host 10602 initiated, in accordance with the teachings of the embodiments described throughout this disclosure. In step 10614, the UE 10606 receives the user data carried in the transmission, which may be performed by a client application executed on the UE 10606 associated with the host application executed by the host 10602.

[00129] In some examples, the UE 10606 executes a client application which provides user data to the host 10602. The user data may be provided in reaction or response to the data received from the host 10602. Accordingly, in step 10616, the UE 10606 may provide user data, which may be performed by executing the client application. In providing the user data, the client application may further consider user input received from the user via an input/output interface of the UE 10606. Regardless of the specific manner in which the user data was provided, the UE 10606 initiates, in step 10618, transmission of the user data towards the host 10602 via the network node 10604. In step 10620, in accordance with the teachings of the embodiments described throughout this disclosure, the network node 10604 receives user data from the UE 10606 and initiates transmission of the received user data towards the host 10602. In step 10622, the host 10602 receives the user data carried in the transmission initiated by the UE 10606.

[00130] In an example scenario, factory status information may be collected and analyzed by the host 10602. As another example, the host 10602 may process audio and video data which may have been retrieved from a UE for use in creating maps. As another example, the host 10602 may collect and analyze real-time data to assist in controlling vehicle congestion (e.g., controlling traffic lights). As another example, the host 10602 may store surveillance video uploaded by a UE. As another example, the host 10602 may store or control access to media content such as video, audio, VR or AR which it can broadcast, multicast or unicast to UEs. As other examples, the host 10602 may be used for energy pricing, remote control of non-time critical electrical load to balance power generation needs, location services, presentation services (such as compiling diagrams etc. from data collected from remote devices), or any other function of collecting, retrieving, storing, analyzing and/or transmitting data.

[00131] In some examples, a measurement procedure may be provided for the purpose of monitoring data rate, latency and other factors on which the one or more embodiments improve. There may further be an optional network functionality for reconfiguring the OTT connection 10650 between the host 10602 and UE 10606, in response to variations in the measurement results. The measurement procedure and/or the network functionality for reconfiguring the OTT connection may be implemented in software and hardware of the host 10602 and/or UE 10606. In some embodiments, sensors (not shown) may be deployed in or in association with other devices through which the OTT connection 10650 passes; the sensors may participate in the measurement procedure by supplying values of the monitored quantities exemplified above, or supplying values of other physical quantities from which software may compute or estimate the monitored quantities. The reconfiguring of the OTT connection 10650 may include message format, retransmission settings, preferred routing etc.; the reconfiguring need not directly alter the operation of the network node 10604. Such procedures and functionalities may be known and practiced in the art. In certain embodiments, measurements may involve proprietary UE signaling that facilitates measurements of throughput, propagation times, latency and the like, by the host 10602. The measurements may be implemented in that software causes messages to be transmitted, in particular empty or 'dummy' messages, using the OTT connection 10650 while monitoring propagation times, errors, etc.

[00132] Although the computing device described herein may include the illustrated combination of hardware components, other embodiments may comprise computing devices with different combinations of components. It is to be understood that these computing devices may comprise any suitable combination of hardware and/or software needed to perform the tasks, features, functions and methods disclosed herein. Determining, calculating, obtaining or similar operations described herein may be performed by processing circuitry, which may process information by, for example, converting the obtained information into other information, comparing the obtained information or converted information to information stored in the computing device, and/or performing one or more operations based on the obtained information or converted information, and as a result of said processing making a determination. Moreover, while components are depicted as single boxes located within a larger box, or nested within multiple boxes, in practice, first and/or second computing devices may comprise multiple different physical components that make up a single illustrated component, and functionality may be partitioned between separate components. For example, a communication interface may be configured to include any of the components described herein, and/or the functionality of the components may be partitioned between the processing circuitry and the communication interface. In another example, non- computationally intensive functions of any of such components may be implemented in software or firmware and computationally intensive functions may be implemented in hardware.

[00133] In certain embodiments, some or all of the functionality described herein may be provided by processing circuitry executing instructions stored on in memory, which in certain embodiments may be a computer program product in the form of a non- transitory computer-readable storage medium. In alternative embodiments, some or all of the functionality may be provided by the processing circuitry without executing instructions stored on a separate or discrete device-readable storage medium, such as in a hard-wired manner. In any of those particular embodiments, whether executing instructions stored on a non-transitory computer-readable storage medium or not, the processing circuitry can be configured to perform the described functionality. The benefits provided by such functionality are not limited to the processing circuitry alone or to other components of the computing device, but are enjoyed by the computing device as a whole, and/or by end users and a wireless network generally.

[00134] In the above description of various embodiments of the present disclosure, it is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of present inventive concepts. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which present inventive concepts belong. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[00135] When an element is referred to as being "connected", "coupled", "responsive", or variants thereof to another element, it can be directly connected, coupled, or responsive to the other element or intervening elements may be present. In contrast, when an element is referred to as being "directly connected", "directly coupled", "directly responsive", or variants thereof to another element, there are no intervening elements present. Like numbers refer to like elements throughout. Furthermore, "coupled", "connected", "responsive", or variants thereof as used herein may include wirelessly coupled, connected, or responsive. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Well-known functions or constructions may not be described in detail for brevity and/or clarity. The term "and/or" includes any and all combinations of one or more of the associated listed items.

[00136] It will be understood that although the terms first, second, third, etc. may be used herein to describe various elements/operations, these elements/operations should not be limited by these terms. These terms are only used to distinguish one element/operation from another element/operation. Thus, a first element/operation in some embodiments could be termed a second element/operation in other embodiments without departing from the teachings of present inventive concepts. The same reference numerals or the same reference designators denote the same or similar elements throughout the specification.

[00137] As used herein, the terms "comprise", "comprising", "comprises", "include", "including", "includes", "have", "has", "having", or variants thereof are open-ended, and include one or more stated features, integers, elements, steps, components or functions but does not preclude the presence or addition of one or more other features, integers, elements, steps, components, functions or groups thereof. Furthermore, as used herein, the common abbreviation "e.g.", which derives from the Latin phrase "exempli gratia," may be used to introduce or specify a general example or examples of a previously mentioned item, and is not intended to be limiting of such item. The common abbreviation "i.e.", which derives from the Latin phrase "id est," may be used to specify a particular item from a more general recitation.

[00138] Example embodiments are described herein with reference to block diagrams and/or flowchart illustrations of computer-implemented methods, apparatus (systems and/or devices) and/or computer program products. It is understood that a block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by computer program instructions that are performed by one or more computer circuits. These computer program instructions may be provided to a processor circuit of a general purpose computer circuit, special purpose computer circuit, and/or other programmable data processing circuit to produce a machine, such that the instructions, which execute via the processor of the computer and/or other programmable data processing apparatus, transform and control transistors, values stored in memory locations, and other hardware components within such circuitry to implement the functions/acts specified in the block diagrams and/or flowchart block or blocks, and thereby create means (functionality) and/or structure for implementing the functions/acts specified in the block diagrams and/or flowchart block(s).

[00139] These computer program instructions may also be stored in a tangible computer-readable medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including instructions which implement the functions/acts specified in the block diagrams and/or flowchart block or blocks. Accordingly, embodiments of present inventive concepts may be embodied in hardware and/or in software (including firmware, resident software, microcode, etc.) that runs on a processor such as a digital signal processor, which may collectively be referred to as "circuitry," "a module" or variants thereof.

[00140] It should also be noted that in some alternate implementations, the functions/acts noted in the blocks may occur out of the order noted in the flowcharts. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality/acts involved. Moreover, the functionality of a given block of the flowcharts and/or block diagrams may be separated into multiple blocks and/or the functionality of two or more blocks of the flowcharts and/or block diagrams may be at least partially integrated. Finally, other blocks may be added/inserted between the blocks that are illustrated, and/or blocks/operations may be omitted without departing from the scope of inventive concepts. Moreover, although some of the diagrams include arrows on communication paths to show a primary direction of communication, it is to be understood that communication may occur in the opposite direction to the depicted arrows.

[00141] Many variations and modifications can be made to the embodiments without substantially departing from the principles of the present inventive concepts. All such variations and modifications are intended to be included herein within the scope of present inventive concepts. Accordingly, the above disclosed subject matter is to be considered illustrative, and not restrictive, and the examples of embodiments are intended to cover all such modifications, enhancements, and other embodiments, which fall within the spirit and scope of present inventive concepts. Thus, to the maximum extent allowed by law, the scope of present inventive concepts is to be determined by the broadest permissible interpretation of the present disclosure including the examples of embodiments and their equivalents, and shall not be restricted or limited by the foregoing detailed description.

Claims

CLAIMS:

1. A computer-implemented method performed by a computing device to provide feedback on an executed field service operation for a communication network, the method comprising: comparing (410) a first model comprising a first representation of a plurality of semantics of the executed field service operation to a second model comprising a second representation of a plurality of semantics of a workflow to execute the field service operation to obtain a first similarity metric; determining (412) an indication that indicates whether the first similarity metric satisfies a criteria for the executed field service operation; and transmitting (414) the indication to a network node.

2. The method of Claim 1, wherein the first model and the second model, respectively, comprise at least one of a machine learning, ML, model, a symbolic model, an ontology-based model, and a statistical model.

3. The method of Claim 1 further comprising: accessing (402) data associated with the executed field service operation, the data obtained from at least one of a plurality of sensory modalities via a communication device proximate a location in which the field service operation is executed; processing (404) the accessed data associated with the executed field service operation to transform the accessed data into the first representation of the plurality of the semantics of the executed field service operation; and providing (406) the first representation to the first model.

4. The method of Claim 3, wherein the data obtained from at least one of a plurality of sensory modalities comprises an image, a video, audio data, environmental stimuli data from an environment proximate the location, force feedback data, and haptic feedback data.

5. The method of any one of Claims 1 to 4, further comprising: processing (400) a trigger that indicates to the computing device to access the workflow to execute the field service operation.

6. The method of Claim 5, wherein the trigger comprises an indication that the computing device is in a location to execute the field service operation.

7. The method of Claim 6, wherein the indication that the computing device is in the location is determined by the computing device.

8. The method of Claim 6, wherein the communications network comprises a telecommunication network and the indication that the computing device is in the location is received from a network node and is based on an attachment of the computing device to a cell in the telecommunication network.

9. The method of Claim 6, wherein the indication that the computing device is in the location is received from a network node and is based on at least one of (i) a fifth generation, 5G, positioning technique that determined the location of the computing device and (ii) a satellite positioning service that that determined the location of the computing device.

10. The method of any one of Claims 1 to 9, wherein the second model comprises respective operations in the workflow to execute the field service operation, and a respective operation comprises (i) a reference scene governed by at least one condition over a fact for the field service operation, (ii) an instruction on how to extract an object and a first scene for the first model, and (iii) the criteria comprising at least one rule to be satisfied in the respective operation for the first scene.

11. The method of Claim 10, wherein the comparing comprises a comparison for the respective operation of (i) a first vector from the first model comprising the extracted object and first scene, with (ii) a second vector from the second model comprising the reference scene, and (iii) obtaining the first similarity metric based on the comparison.

12. The method of Claim 11, wherein the similarity metric comprises a measure of the similarity of the first vector to the second vector based on the rule.

13. The method of any one of Claims 1 to 12, further comprising: accessing (416) additional data associated with the executed field service operation, the additional data obtained from at least one of a plurality of the sensory modalities; processing (418) the accessed additional data associated with the executed field service operation to transform the accessed additional data into an addition to the first representation; providing (420) the addition to the first representation to the first model; comparing (422) the first model comprising the addition to the first representation to the second model to obtain a second similarity metric; and determining (424) the indication that indicates whether one of the first and the second similarity metrics satisfies the criteria for the executed field service operation.

14. The method of any one of Claims 1 to 13, wherein the procedure for execution of the field service operation comprises a series of respective operations in at least one of a natural language, an artificial language, and a notation, and further comprising: constructing (408) the second model based on extraction of at least one of the natural language, the artificial language, and the notation into the second representation of the plurality of semantics of the workflow to execute of the field service operation.

15. The method of Claim 13, wherein the executed field service operation comprises at least one of an installation, an upgrade, a maintenance operation, and a troubleshooting operation of a component.

16. The method of Claim 15, wherein the component is at least one of a radio frequency, RF, weatherproofing component, a coaxial cable, a non-coaxial cable, a cable nut, a bolt, a radio unit, and an antenna.

17. The method of any one of Claims 1 to 16, wherein the feedback comprises the indication and the indication comprises at least one of a verification that the executed field service operation satisfies the workflow and information about why the workflow is not satisfied.

18. The method of any one of Claims 3 to 17, wherein the at least one of a plurality of sensory modalities comprises the communication device configured to collect the data, an artificial intelligence (Al) based headset or glasses configured to collect the data, and a sensory glove configured to collect the data.

19. A computing device (102, 6200) configured to provide feedback on an executed field service operation for a communications network, the computing device comprising: processing circuitry (6202a, 6202b); memory (6210) coupled with the processing circuitry, wherein the memory includes instructions that when executed by the processing circuitry causes the computing device to perform operations comprising: compare a first model comprising a first representation of a plurality of semantics of the executed field service operation to a second model comprising a second representation of a plurality of semantics of a workflow to execute the field service operation to obtain a first similarity metric; determine an indication that indicates whether the first similarity metric satisfies a criteria for the executed field service operation; and transmit the indication to a network node.

20. The computing device of Claim 19, wherein the memory includes instructions that when executed by the processing circuitry causes the computing device to perform further operations comprising any of the operations of any one of Claims 2 to 18.

21. A computing device (102, 6200) configured to provide feedback on an executed field service operation for a communications network, the computing device adapted to perform operations comprising: compare a first ML model comprising a first representation of a plurality of semantics of the executed field service operation to a second model comprising a second representation of a plurality of semantics of a workflow execute the field service operation to obtain a first similarity metric; determine an indication that indicates whether the first similarity metric satisfies a criteria for the executed field service operation; and transmit the indication to a network node.

22. The computing device of Claim 21 adapted to perform further operations according to any one of Claims 2 to 18.

23. A computer program comprising program code to be executed by processing circuitry (6202a, 6202b) of a computing device (102, 6200) configured to provide feedback on an executed field service operation for a communication network, whereby execution of the program code causes the computing device to perform operations comprising: compare a first model comprising a first representation of a plurality of semantics of the executed field service operation to a second comprising a second representation of a plurality of semantics of a workflow to execute the field service operation to obtain a first similarity metric; determine an indication that indicates whether the first similarity metric satisfies a criteria for the executed field service operation; and transmit the indication to a network node.

24. The computer program of Claim 23, whereby execution of the program code causes the computing device to perform operations according to any one of Claims 2 to 18.

25. A computer program product comprising a non-transitory storage medium (6210) including program code to be executed by processing circuitry (6202a, 6202b) of a computing device (102, 6200) configured, whereby execution of the program code causes the computing device to perform operations comprising: compare a first model comprising a first representation of a plurality of semantics of the executed field service operation to a second model comprising a second representation of a plurality of semantics of a workflow to execute the field service operation to obtain a first similarity metric; determine an indication that indicates whether the first similarity metric satisfies a criteria for the executed field service operation; and transmit the indication to a network node.

26. The computer program product of Claim 25, whereby execution of the program code causes the computing device to perform operations according to any one of Claims 2 to 18.