WO2021221635A1

WO2021221635A1 - Feedback insight recommendation

Info

Publication number: WO2021221635A1
Application number: PCT/US2020/030516
Authority: WO
Inventors: Rafael Dal ZOTTO
Original assignee: Hewlett-Packard Development Company, L.P.
Priority date: 2020-04-29
Filing date: 2020-04-29
Publication date: 2021-11-04

Abstract

In an example implementation according to aspects of the present disclosure, a system, method, and storage medium for feedback insight recommendations are disclosed. A system comprising a storage medium and a processor receive feedback data from a feedback agent. The processor extracts a sentiment and a topic from the feedback. The processor receives telemetry data from a telemetry agent. The processor correlates the telemetry data to the topic. The processor correlates the topic to insight data and recommends a change to the computing device based on the insight data.

Description

FEEDBACK INSIGHT RECOMMENDATION

BACKGROUND

[0001]A telemetry agent may execute on a computing device collecting metrics about the health of a computing device. The telemetry agent in conjunction with an analytical service may detect situations within the computing device that may result in an undesirable system state.

BRIEF DESCRIPTION OF THE DRAWINGS

[0002] FIG. 1 illustrates a feedback insight recommendation system, according to an example;

[0003] FIG. 2 is a block diagram corresponding to creating feedback insight recommendations, according to an example;

[0004] FIG. 3A and FIG. 3B are graphical illustrations visualizing the creation of feedback insight recommendations according to an example;

[0005] FIG. 4 is an illustration of a knowledge graph of the relationships in the feedback data and the analytical service, according to one example; and

[0006] FIG. 5 is a computing device for supporting instructions for creating feedback insight recommendations, according to an example.

DETAILED DESCRIPTION

[0007] When utilizing a computing device, such as a personal computer, often a perceived negative user experience corresponds with an issue of the computing device. The issue may be software or hardware related, and often the issue is not self-evident or user correctable. As describe herein is a system for the receiving of user feedback, the correlation of the topic of the feedback, along with the correlation of the topic to insight data, to a recommendation to change the system based on the insight data. [0008] In one example, a processor within the system receives feedback from a feedback agent. The processor extracts a sentiment and a topic from the feedback. The processor receives telemetry data from a telemetry agent. The processor correlates the telemetry data to the extracted topic. The processor utilizes the correlation to identify insight data corresponding to the topic. The processor recommends a change to the system to correct the original issue based on the insight data.

[0009] FIG. 1 illustrates a feedback insight recommendation system 100, according to an example. The system 100 may include a processor 102, instruction 104, a telemetry agent 106, and a feedback agent 108.

[0010] The processor 102 may be implemented as a general-purpose processor such as a central processing unit (CPU). The processor 102 may also be implemented as a virtual processor. A virtual processor may be abstracted from a specific piece of hardware and may be defined by the workload it processes. A virtual processor may be a cloud processing instance, or a virtual machine instance. The processor 102 may be connected to supporting electronics including a host system and a network (not shown) to facilitate the operation of the processor.

[0011] The processor 102 may be able to process instructions 104. The instructions 104 may be firmware or software to change or control the behavior of the processor 102. For example, instructions 104 for the processor 102 may including receiving feedback, extracting a sentiment and topic from the feedback, receiving telemetry data, correlating the telemetry data to the topic , correlating the topic to insight data, and recommending a change to the computing device based on the insight. The instructions 104 may be stored in a non-transitory storage medium.

[0012] A telemetry agent 106 may reside on a plurality of additional computing devices. The telemetry agent 106 may be utilized by an organization as a way of pushing updates (e.g. security fixes, software features) to targeted computing devices within the organization. Additionally, a telemetry agent 106 may collect information about the computing device and its usage. The telemetry agent 106 may collect information relating to the hardware of the computing device. The hardware may include but is not limited to information about processor model, installed memory, graphics adapter, display resolution and type, peripherals, and network connection. The information relating to the hardware may capture aspects of the hardware in operation that may be relevant to a certain use case for the user of the computing device. The information relating to the hardware may capture operating aspects of the hardware in operation to determine device health with software application usage pattern.

[0013] The telemetry agent 106 may also collect information relating to the software of the computing device. The software may include but is not limited to information about the operating system, drivers installed, applications installed, and applications used (including frequency and duration). Like the hardware information, the information relating to the software may capture aspects of the software in operation that may be relevant to a certain use case for the user of the computing device.

[0014]The feedback agent 108 may collect information relating to a user experience on the computing device. The feedback agent 108 serves as an entry point for a user input into the system. When a user experiences an issue with the computing device, the user may utilize the feedback agent 108 to input prose natural language into the system 100. The feedback agent 108 may receive typed input as delivered from a physical or virtual keyboard. The feedback agent 108 may receive transcribed textual input delivered from a voice processing or dictation agent. In another implementation, the feedback agent 108 may receive input as a predetermined response illustrated as a lexicon. For example, the feedback agent 108 may present the user with three indications of various levels of satisfaction corresponding to an emotional facial expression, where a smiling face is a high level of satisfaction, and a frowning face is a low level of satisfaction. The feedback agent 108 may time date stamp the entry of any feedback within the system as to identify the time that the user expressed an emotion directed at the computing device.

[0015]The feedback agent 108 may be an application installed on a computing device that executes in the background. The feedback agent 108 may consume minimal computing resources until utilized by a user. In one implementation, the feedback agent 108 may be packaged along with the telemetry agent 106 as part of a standardized installation. In a personal computer implementation, the feedback agent 108 may be a widget that integrates with a taskbar or desktop that allows a user quick access with minimal keystrokes or mouse actions.

[0016] The telemetry agent 106 and feedback agent 108 may be connected to the processor 102 by underlying networking structure (not shown). The underlying networking structure may include wired or wireless networking infrastructure including a networking adapter, as well as available LAN, WAN and Internet connectivity. In another implementation, the telemetry agent 106, the feedback agent 108 and the processor 102 to execute the instructions may co-exist on the same physical computing device.

[0017] FIG. 2 is a block diagram 200 corresponding to creating feedback insight recommendations, according to an example. For purposes describing portions of FIG. 2, references may be made to portions of FIG. 1.

[0018] At 202, the processor 102 receives feedback from a feedback agent, wherein the feedback corresponds to a user experience on a computing device. A user may be interfacing with a computing device when something unexpected happens. For example, an application may become unresponsive or may crash. A user may proceed to close or re-execute the application without issue, outside of the frustration of the unexpected problem. In an implementation, the user may provide feedback to the feedback agent 108 upon the unexpected problem. Using the previous example, an application may have an unexpected crash. The user may use the feedback agent to articulate the problem, including the application and a description of what the user experienced. Some experiences may be system freezes, incomplete experiences (e.g. audio or video net working), application crashes, and non responsiveness. System crashes may be input into the feedback agent when the system reboots.

[0019] At 204, the processor 102 extracts a sentiment and a topic from the feedback. Within each feedback, a sentiment and a topic may be included. In one implementation, a sentiment may be positive, negative or neutral. In another implementation there may be additional sentiments correlating with human emotions such as happy, sad, angry, confused, disgusted, surprised and calm. In the case of utilizing only textual feedback, a sentiment may be extracted from the text. In other cases, utilizing the predetermined lexicon approach described previously, the sentiment may be extracted by an icon the user indicates (e.g. smiling face). The sentiment may be used to prioritize the feedback. In one implementation negative feedback may be processes or prioritized over neutral or mixed feedbacks. Rules based sentiment analysis techniques or machine learning techniques may be utilized for this purpose. The topic may be extracted through natural language processing (NLP). The topic may correspond to an operational aspect of the computing device and may include one or more aspects or topics corresponding to the computing device. The topic may be used to detect to which area the feedback relates, in order to help correlating with existent telemetry data. The received feedback may be used as input to a set of machine learning algorithms in order to detect and extract topics and key phrases from it, using automatic data summarization techniques. The feedback may be analyzed for multiple topics and sentiments corresponding to more than one operational aspect. The purpose of the topic extraction may be to find a subset of text which contains the "information" of the entire set. A representative summary or abstract list of main topics may be extracted out of the feedback.

[0020] At 206, the processor 102 receives telemetry data from a telemetry agent 106, wherein the telemetry data comprises information about the computing device. Telemetry data may be the collection of analytical data points corresponding to the operation of the computing device starting at prior to the feedback reception. As described previously, the telemetry agent 106 may execute on the computing device. The telemetry agent 106 may not have a front end or graphical user interface. The telemetry agent 106 may be a service or kernel level application that periodically queries subsystems for state changes and statuses. The collection of the state changes and status of the various components may be made in effort to collect details relevant to root cause analysis, or proactive management of the device. The telemetry agent 106 may be an edge component of a larger analytical service.

[0021] At 208, the processor 102 responsive to the sentiment, correlates telemetry data to the topic from the feedback. In one implementation, the processor 102 may determine a feedback with a negative feedback sentiment. A negative sentiment may correspond with a problem with the computing device, whereas a positive sentiment may not. A negative sentiment indicates a priority for processing. The extracted topic may be correlated to the telemetry. In one implementation, the telemetry and the topic may be correlated based on a common time date stamp. In another implementation the correlation may be based upon a predefined period prior to the receipt of the feedback including the topic. The feedback may be received immediately or a relatively small time period after the problem.

[0022] At 210, the processor 102 correlates the topic to insight data from an analytical service. An analytical service may independently utilize information from the telemetry agent 106 to provide insights and insight data to an information technology fleet manager. The analytical service may correlate telemetry data from a plurality of computing devices. For example, a company may utilize an analytical service with a telemetry agent installed and executing on each computing device within the company to monitor and ensure the smooth operations of the company. As such, correlated feedback topics to telemetry data may be correlated to the insights derived from telemetry data. Using this relationship, the processor 102 may be able to correlate the feedback through the telemetry from the time of the feedback to the insight data of the analytical system. In an analytical service which hosts the insight data, solutions to each of the insights may be stored. As such, the topic may be linked to a solution to the root cause which generated the feedback.

[0023] At 212, the processor 102 identifies a plurality of computing devices associated to the analytical service with characteristics in common with the computing device. As the analytical service may receive telemetry data from a plurality of computing devices, the identified solution corresponding to the topic, may be proactively deployed to similar computing devices. Similar computing devices may be computing devices connected to the analytical service which have specification similar to or the same as the reporting computing device, where an applicable solution may affect it in the same manner. For example, a user may report feedback on their computing device with eight gigabytes of random-access memory (RAM). The solution correlated to that feedback corresponds to a driver issue for a sound device. The plurality of computing devices attached to the analytical service have thirty-two gigabytes of RAM but the same sound device. The solution correlated to the feedback may be applicable to the plurality of computing devices as they are similar enough that the solution affects each in the same manner. [0024] At 214, the processor 102 transmits a change to the plurality of computing devices based on the insight data. The change may correspond to a plurality of operational aspects of the computing devices. Responsive to the processor 102 identifying the plurality of computing devices for with the insight data solution may affect, the processor 102 may proactively implement the solution. The solution may be packaged as a system change to update each of the plurality of computing devices. In some implementations, the change may be a new driver package, a configuration change, or a removal of a misbehaving application. The change may be accessed by the telemetry agent on receipt and executed. In another implementation, the change may be transmitted through an endpoint management system designed to effectuate policies within fleet systems.

[0025] In another implementation, the processor 102 may recommend a change to the computing device based on insight data. In this implementation, a change may not be able to be programmatically implemented. The processor 102 may utilize the telemetry agent on the computing device to notify the user of a recommendation as to a change the user may need to effectuate.

[0026] In another implementation, the processor 102 may recommend a change to a plurality of computing devices based on the insight data. When attached to an analytical service, a plurality of computing devices may be afflicted with the same problem that generated the feedback. The recommendation may be a proactive response to resolve issues before the users of the plurality of computing devices experience the event that spawned the feedback.

[0027] FIG. 3A and FIG. 3B are graphical illustrations visualizing the creation of feedback insight recommendations according to an example.

[0028] Referring to FIG. 3A, a user 302 may interface with a computing device 314.

A computing device 314 is represented in FIG. 3A as a laptop computer, however, the computing device 314 may also be but is not limited to a desktop computer, a virtualized computing system, a mobile phone, a tablet, or a wearable. The computing device 314 may be able to receive feedback 304 as described previously. In some implementations, that may include the utilization of physical or virtualized keyboards, and touch screens. Additionally, a computing device 314 may also be “headless” in that it has no display, however, the computing device 314 may include an input device such as a microphone.

[0029]As discussed previously, the feedback 304 may include natural language. Natural language may include the user’s spoken or written language. The feedback 304 may take a form consistent with the computing device’s input devices. Inherent to the feedback 304 may be a sentiment 306 and a topic 308. The sentiment 306 and the topic 308 may be extracted through programmatic means relying on inferences based on natural language. In one example, a feedback 304 may be divided into more than one topics and a corresponding sentiment for each of the topics. For example, feedback 304 may include “the keyboard is non-responsive, but the trackpad works well.” In this example, there may be two topics with a corresponding sentiment each.

[0030] FIG. 3B further illustrates the feedback insight recommendation system. FIG.3B illustrates the analytical service 300B. The analytical service 300B, utilizes an analytics engine 314. The analytics engine 314 receives telemetry data from a plurality of telemetry agents deployed in the analytical service. Independent of the feedback processing, the analytics engine 314 correlates the received telemetry data into insights illustrated in FIG.3B as insight A 312A and insight B 312B. In at least one implementation, insights A 312A and insight B 312B may be correlated to a subsystem. In many instances, the subsystems correlate to the topic 308. For example, if the topic 308 is “battery”, insight A 312A and insight B 312B are identified as insights corresponding to batteries. In some implementations, the mapping of insights to subsystems may be manual and provided by a IT fleet manager.

[0031] Likewise, as set of solutions 316, exist in the analytical system 300B independent of the feedback processing. The solutions illustrated in FIG. 3B include solution A 310A, solution B 310B, and solution C 310C, where solution A 310A, solution B 310B, and solution C 310C are distinct from each other and may be used to resolve issued identified by one or more insights.

[0032]As described in reference to FIG. 2, a topic 308 may be correlated with one or more insights (as illustrated insight A 312A, and insight B 312B). The topic 308 may be correlated to insight A 312A and insight B 312B. In this example, both insights may correspond to the same topic or subsystem such as a “battery” issue. By linking, the topic 308 to insight A 312A and insight B 312B, solutions to the feedback may be retrieved. As illustrated, solution A 310A and solution B 31 OB may be solutions to insight A 312A. Additionally, solution C 310C corresponds to insight B 312B. In this illustration as the topic 308 correlates to both insight a 312A and insight B 312B, solution A 310A, solution B 310B, and solution C 310C may be applied to resolve an issue which spawned the feedback.

[0033] FIG. 4 is an illustration of a knowledge graph 400 of the relationships in the feedback data and the analytical service, according to one example. A knowledge graph may be a schema used to describe an ontology between objects within a data structure. The knowledge graph demonstrates the relationships between the objects. The knowledge graph 400 depicted in FIG. 4 corresponds to many concepts previously discussed.

[0034] A computing device 404 may be the central component of this knowledge graph 400. The computing device 404 may be the point of instantiation of the feedback based on an issue with the computing device. The computing device 404 may be owned by a user 402. The user 402 utilizes their senses to detect if the computing device 404 may be having an unusual event. Based on their senses the user 402 writes the feedback 406. Likewise, the computing device 404 generates the feedback 406. The writing of the feedback 406 is the natural language description provided by the user 402. The generation of the feedback 406 may be the digital representation of the feedback 406 within the system. The feedback 406 may have a sentiment 408, and a topic 410 which were described in detail previously.

[0035] The computing device 404 within the analytical service, may also be built upon an architecture 414. Likewise, the computing device 404, is a model 416 with a manufacturer 418. The relationships between the nodes in the knowledge graph allow the feedback to be processed in the analytical service and then computing device 404 to be properly identified for remediation. As knowledge graphs 400 may be flexible representations of ontologies, any of the nodes may be extendable to include other relationships. For example, the computing device 404 node may include configuration “HAS_A” nodes for configuration specifications such as CPU, memory, etc.

[0036] FIG. 5 is a computing device for supporting instructions for feedback insight recommendations, according to an example. The computing device 500 depicts a processor 102 and a storage medium 504 and, as an example of the computing device 500 performing its operations, the storage medium 504 may include instructions 506-518 that are executable by the processor 102. The processor 102 may be synonymous with the processor 102 referenced in FIG. 1. Additionally, the processor 102 may include but is not limited to central processing units (CPUs). The storage medium 504 can be said to store program instructions that, when executed by processor 102, implement the components of the computing device 500.

[0037] The executable program instructions stored in the storage medium 504 include, as an example, instructions to receive feedback from a feedback agent 506, instructions to extract a sentiment and a topic from the feedback 508, instructions to receive telemetry data from a telemetry agent 510, responsive to the sentiment, correlate telemetry data to the topic from the feedback 512, instructions to correlate the topic to insight data from an analytical service 514, instructions to identify a plurality of computing devices associated to the analytical service with characteristics in common with the computing device 516, and instructions to recommend a change to the plurality of computing devices 518.

[0038] Storage medium 504 represents generally any number of memory components capable of storing instructions that can be executed by processor 102. Storage medium 504 is non-transitory in the sense that it does not encompass a transitory signal but instead is made up of at least one memory component configured to store the relevant instructions. As a result, the storage medium 504 may be a non-transitory computer-readable storage medium. Storage medium 504 may be implemented in a single device or distributed across devices. Likewise, processor 102 represents any number of processors capable of executing instructions stored by storage medium 504. Processor 102 may be integrated in a single device or distributed across devices. Further, storage medium 504 may be fully or partially integrated in the same device as processor 102, or it may be separate but accessible to that computing device 500 and the processor 102.

[0039] In one example, the program instructions 506-518 may be part of an installation package that, when installed, can be executed by processor 102 to implement the components of the computing device 500. In this case, storage medium 504 may be a portable medium such as a CD, DVD, or flash drive, or a memory maintained by a server from which the installation package can be downloaded and installed. In another example, the program instructions may be part of an application or applications already installed. Here, storage medium 504 can include integrated memory such as a hard drive, solid state drive, or the like.

[0040] It is appreciated that examples described may include various components and features. It is also appreciated that numerous specific details are set forth to provide a thorough understanding of the examples. However, it is appreciated that the examples may be practiced without limitations to these specific details. In other instances, well known methods and structures may not be described in detail to avoid unnecessarily obscuring the description of the examples. Also, the examples may be used in combination with each other.

[0041] Reference in the specification to “an example” or similar language means that a particular feature, structure, or characteristic described in connection with the example is included in at least one example, but not necessarily in other examples. The various instances of the phrase “in one example” or similar phrases in various places in the specification are not necessarily all referring to the same example.

[0042] It is appreciated that the previous description of the disclosed examples is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these examples will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other examples without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the examples shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

CLAIMS WHAT IS CLAIMED IS:

1. A system comprising: a memory; a processor communicatively coupled to the memory, wherein the processor retrieves and executes instructions from the memory, the instructions comprising: receive feedback from a feedback agent, wherein the feedback corresponds to a user experience on a computing device; extract a sentiment and a topic from the feedback; receive telemetry data from a telemetry agent, wherein the telemetry data comprises information about the computing device; responsive to the sentiment, correlate telemetry data to the topic from the feedback; correlate the topic to insight data from an analytical service; and recommend a change to the computing device based on insight data.

2. The system of claim 1 wherein the extracted sentiment comprises a positive, negative or neutral sentiment.

3. The system of claim 1 wherein the telemetry data corresponds to a collection of analytical data points corresponding to an operation of the computing device starting at prior to the feedback reception.

4. The system of claim 1 wherein the topic corresponds to an operational aspect of the computing device.

5. The system of claim ,1 the instructions further comprise transmit instructions comprising a change to the computing device based on the insight data.

6. A method comprising: receiving feedback from a feedback agent, wherein the feedback corresponds to a user experience on a computing device; extracting a sentiment and a topic from the feedback; receiving telemetry data from a telemetry agent, wherein the telemetry data comprises information about the computing device; responsive to the sentiment, correlating telemetry data to the topic from the feedback; correlating the topic to insight data from an analytical service; identifying a plurality of computing devices associated to the analytical service with characteristics in common with the computing device; transmitting a change to the plurality of computing devices based on the insight data.

7. The method of claim 6 wherein the extracted sentiment comprises a positive, negative or neutral sentiment.

8. The method of claim 6 wherein the telemetry data corresponds to a collection of analytical data points corresponding to an operation of the computing device starting at prior to the feedback reception.

9. The method of claim 6 wherein the topic corresponds to an operational aspect of the computing device.

10. The method of claim 6 wherein the change corresponds to a plurality of operational aspects of the plurality of computing devices.

11. A non-transitory computer readable medium comprising instructions executable by a processor to: receive feedback from a feedback agent, wherein the feedback corresponds to a user experience on a computing device; extract a sentiment and a topic from the feedback; receive telemetry data from a telemetry agent, wherein the telemetry data comprises information about the computing device; responsive to the sentiment, correlate telemetry data to the topic from the feedback; correlate the topic to insight data from an analytical service; identify a plurality of computing devices associated to the analytical service with characteristics in common with the computing device; recommend a change to the plurality of computing devices based on insight data.

12. The storage medium of claim 11 wherein the extracted sentiment comprises a positive, negative or neutral sentiment.

13. The storage medium of claim 11 wherein the telemetry data corresponds to a collection of analytical data points corresponding to an operation of the computing device starting at prior to the feedback reception.

14. The storage medium of claim 11 wherein the topic corresponds to an operational aspect of the computing device.

15. The storage medium of claim 11 wherein the change corresponds to a plurality of operational aspects of the plurality of computing devices.