KR20220148981A - A natural language processing and analyzing method in interactive computing system - Google Patents

Abstract

According to the present invention, a software agent used to assist in providing a service receives communication from a set of users attempting to use the software agent, wherein the communication includes communication interacting with the software agent and communication not interacting with the software agent. In addition, the software agent performs natural language processing on all communications to identify things such as user emotion, user interest, or other items in message contents, while identifying action taken by users to obtain a measure of user satisfaction with the software agent. Accordingly, one or more action signals are generated based on the identified user satisfaction with the software agent.

Description

A natural language processing and analyzing method in interactive computing system

It is about natural language processing and analysis techniques in an interactive computing system.

Computing systems are now widely used. Some computing systems include such as electronic mail (email) computing systems, calendaring or scheduling computing systems, among a wide variety of others.

Also, some computing systems include digital agents, or personal digital assistants. This type of agent or assistant includes, for example, a software agent or other bot, which is a computer program capable of performing a variety of tasks and may provide an interface for human interaction using natural language. can do. They illustratively use natural language understanding logic to identify the linguistic meaning of natural language input, and based on that natural language input, automatically perform a specific task for the user. For example, an agent or assistant may set reminders, answer questions by performing searches, and interact with other services, among other things, based on natural language input.

In performing such a task, such an agent or bot is often involved in performing a conversation with a user in order to perform a service. For example, some bots or agents may be used when a user is talking to other users. As an example, suppose a first user sends an e-mail message (e-mail message) asking a group of other users whether they want to watch a movie or attend another entertainment on a specific date. To find available activities, one of the users or a group may send an email message (including all other users in the group in reference) to a software agent configured to identify entertainment taking place at different locations on different days. Thereafter, the software agent may indicate to the group the series of entertainment activities that are taking place on that day, and may help the group obtain a ticket. As an example, the software agent may send an email to the entire group indicating a series of entertainment activities and asking if the software agent should obtain a ticket. When instructed to do so, the software agent may conduct a conversation with one or more of its users to identify, among other things, the number of tickets, the amount to be spent on them. The software agent may then interact with the ticketing site to obtain a ticket.

It may be difficult to improve such a computing system because it may be difficult to know which functions various users are satisfied with, which functions may be dissatisfied, and why. Some current systems ask users to provide feedback by taking part in a survey. However, users often find surveys to be cumbersome and time-consuming. Therefore, feedback can be quite limited as only a relatively small number of users actually participate in the survey.

The above discussion is provided for general background information only and is not intended to help determine the scope of the claimed subject matter.

1 is a block diagram of one embodiment of a computing system architecture.
2A and 2B (collectively referred to herein as FIG. 2 ) show operation of the architecture illustrated in FIG. 1 in analyzing different signals to identify user satisfaction and generate control signals based on the identified user satisfaction; shows a flow diagram illustrating one embodiment of
3 is a block diagram illustrating an embodiment of the architecture illustrated in FIG. 1 , deployed in a cloud computing architecture.
4-6 show embodiments of a mobile device that may be used in the architectures shown in the previous figures.
7 is a block diagram illustrating one embodiment of a computing environment that may be used in the architectures shown in the previous figures.

1 is a block diagram of one embodiment of a computing system architecture 100 . Architecture 100 illustratively includes a computing system 102 that creates a user interface 104 having a user input mechanism 106 for interaction by a user 108 . The user 108 interacts with the user input mechanism 106 to control and manipulate portions of the computing system 102 , illustratively.

1 , computing system 102 illustratively includes one or more processors or servers 110 , electronic mail (email) system 112 , calendar system 114 , communication system 116 , User interface logic 118 , scheduling assistance system 120 , while may also include a wide variety of other items 122 as well. The calendar system 114 itself, illustratively includes scheduling logic 124, a calendar data store 126 (which may be a calendar data store for user 108 and/or other users), while very Various other calendar functions 128 may be included.

Scheduling assistance system 120 is illustratively a computing system that assists user 108 in interacting with calendar system 114 and other users' calendar systems to perform scheduling operations. In one embodiment, user 108 interacts with scheduling assistance system 120 by sending a message to an inbox of scheduling assistance system 120 , illustratively using email system 112 . works However, in other embodiments, the user 108 may communicate with the scheduling assistance system 120 using another type of communication, such as natural language voice input, another messaging system, or the like.

Accordingly, scheduling assistance system 120 is illustratively shown as part of natural language understanding system 130 (scheduling assistance system 120 ), but may be separate from it or remote accessed by system 120 . service), scheduling assistant workflow processing system 132 , user satisfaction signal analyzer and control system 134 , while other items 136 . Before describing the operation of architecture 100 in more detail, a brief overview of some of the items in architecture 100 and their operation is first provided.

Email system 112 illustratively provides email functionality to user 108 . Thus, in one embodiment, user 108 may create, send, receive, open, and delete email messages. User 108 may illustratively use e-mail system 112 to configure his or her mailbox and e-mail account, set filters, attach various attachments to different e-mail messages, and provide a wide variety of other e-mail features. To do this, you can perform a setting operation.

Scheduling logic 124 within calendar system 114 illustratively allows user 108 to schedule items on his or her calendar. Calendar data representing user 108's calendar is stored as calendar data, illustratively in calendar data store 126 . The calendar system 114 provides a wide variety of other calendar functions 128, such as the ability to send and receive meeting requests, schedule tasks and appointments, schedule meetings with a variety of different users, and check the availability of other users. can provide

Communication system 116 illustratively enables computing system 102 to communicate with a wide variety of other computing systems. For example, it may be a communication system that enables communication over a wide area network, a local area network, a cellular communication network, a short-range communication network, or a wide variety of other networks or combinations of networks.

User interface logic 118 creates a user interface 104 , illustratively with a user input mechanism 106 . It also detects user interaction, illustratively with user input mechanism 106 or with any of a wide variety of other user input mechanisms. This may create an indication of that user interaction and provide it to other parts of the computing system 102 as well.

Scheduling assistance system 120 illustratively uses scheduling assistant workflow processing system 132 to describe when a user 108 requests system 120 to assist in scheduling an event, such as a meeting, phone call, or the like. detect In one embodiment, user 108 may do this by composing a natural language email message and sending it to a mailbox of email system 112 corresponding to scheduling assistance system 120 . When system 120 receives a message in its mailbox, exemplarily using natural language understanding system 130 , system 120 parses the message and identifies the linguistic content (eg, meaning) of the content within the email message. Thereafter, the message may be identified as a message seeking interaction with the system 120 or as a message that does not seek interaction, but is copied into the message.

Scheduling assistant workflow processing when system 120 identifies the email message as a message seeking interaction with system 120 (eg, a request to assist user 108 in performing a scheduling operation). System 132 may illustratively perform a workflow that may involve a conversation with user 108 (eg, via email system 112 or otherwise) so that user 108 asks what system 120 does. You can identify what you want to do. This may include a scheduling assistant workflow processing system 132 that queries the user 108 to identify a task to be performed.

Thereafter, the system 132 may perform operations such as performing information retrieval, retrieving various users' calendar systems, retrieving other computing systems or databases, and returning results. It may then perform other actions, such as interacting with scheduling logic 124 for user 108 and other users, to schedule meetings or other events, or interact with other computing systems to perform other functions ( for example, purchasing tickets, booking, among a wide variety of others.

With a system such as the scheduling assistance system 120 , it can be difficult to determine the level of satisfaction the user 108 feels for the system. Accordingly, the user satisfaction signal analyzer and control system 134 illustratively monitors various signals and actions from the user 108 to determine the user's satisfaction. Thereafter, an action may be taken by generating a control signal based on the corresponding satisfaction level.

It will be appreciated that, for the purposes of this embodiment, the scheduling assistance system 120 need not be a scheduling assistance system, but may instead be another software agent or bot that engages in a conversation with one or more users when rendering its service. . For example, the system may not be a system that supports a user in performing a scheduling operation, but a system that supports a user in performing a translation task in an email discussion among multiple users who speak different languages. A system may be a system that assists a user to order lunch, schedule one or more different locations, purchase an air ticket or event ticket, or may be a system that supports a wide variety of users participating in a conversation with a set of users to perform a desired service. It could be another system. For purposes of example only, the present description is described in the context of scheduling assistance system 120 , a system that supports one or more users to schedule items. However, the same treatment may apply to other types of auxiliary systems or other software agents or software programs participating in discussions with users who are using the services provided by the system.

User satisfaction signal analyzer and control system 134 exemplarily includes user interest detection logic 138 , sentiment analysis logic 140 , user recommendation detection logic 142 , calendar mining logic 144 , and satisfaction level detection logic 145 . ), feature discovery involvement logic 146 , issue identification logic 147 , control signal generator logic 148 , while may include a wide variety of other items 150 . Calendar mining logic 144 itself illustratively includes meeting classification logic 152 , service coverage determination logic 154 , trend identification logic 156 , survey logic 158 , while other items 160 . may include Control signal generator logic 148 itself, illustratively includes shared prompting logic 162 , while may include a wide variety of other items 164 .

In one embodiment, the scheduling assistance system 120 not only requires messages that are directly related to the scheduling assistance service provided by the system 120 and that explicitly seek interaction with the system 120 , the system 120 . It should be noted first that it may be included or copied to other messages in the communication thread that do not need to seek interaction with 120 . For example, the user 108 may be sending an email message to the group of other users and also to the system 120 specifically requesting the system 120 to assist the group in scheduling a meeting. As an example, the user may enter "Scheduling Assistant, please help us schedule a meeting." The scheduling assistant workflow processing system 132 then executes the workflow to identify details of the meeting and various meeting opportunities for which the meeting may be scheduled. By doing so, the system can generate an email replying to the group of users being chatted with, and receive responses from those users.

However, it may be that the users participating in the conversation copy the system 120 to the message but create a message to each other that does not seek direct interaction with the system 120 . For example, one of the users may be sending the question "What is your favorite restaurant near the meeting location?" to the entire group of users. Although this communication may be copied to the system 120 , it does not require any interaction with the system 120 . Instead, it is seeking interaction with other users. Similarly, one of the users may enter and copy to the system 120 a message for all of the other users providing an indication of the user's level of satisfaction with the system 120 . For example, assuming the name of the system 120 is “The ACME Scheduling Assistant,” one of the users will say “This ACME Scheduling Assistant is very slow.” Or you can send a message like "This ACME Scheduling Assistant is really helpful." Thus, even if the originator of the message is not seeking interaction with the system 120 , the content of the message itself can be analyzed to identify the user's level of satisfaction with the system 120 .

Similarly, actions of users in a conversation may provide an indication of their level of satisfaction with the system 120 . For example, if one of the users sends an email recommending other users to use the “ACME Scheduling Assistant”, this may indicate that the level of satisfaction of the creator of the message is relatively high. However, when a user sends a message such as "Do you know of any other scheduling assistants that we might user?", the user's satisfaction is relatively low. can indicate that

Also, if the user is using the system 120 to schedule most of his or her appointments in his or her calendar, this may indicate that the user is relatively satisfied with the system 120 . However, if the user is using the system 120 to schedule only a small number of appointments in his or her calendar compared to all appointments in the user's schedule, this would indicate that the user is relatively dissatisfied with the system 120 . can

Additionally, while a user is using system 120 to schedule a certain percentage of his or her meetings in one category (eg, business meetings), the system 120 may be used to schedule meetings in another category (eg, personal meetings). Or they may be scheduling their meetings at a significantly different rate (such as phone calls). This may indicate that the user is relatively satisfied when using the system 120 to perform an action in a particular action category, but relatively dissatisfied when using the system 120 to perform an action in one or more other action categories. .

Similarly, a user's usage tendencies in using the system 120 may also be helpful. If a user tends to use the system 120 frequently, this may be a positive trend indicating that the user is relatively satisfied with the system 120 . However, if the trend is negative in that the user is using the system 120 less frequently, this may indicate that the user is not sufficiently satisfied with the system 120 .

Also, the user 108 may be engaging with the system 120 in another way. For example, while a user is engaged in system 120 to perform one set of functions (eg, to schedule a meeting), what other types of functions may be performed using system 120 ? It may also be engaging the system 120 in an attempt to find By way of example, whether user 108 can “find entertainment options” or “schedule phone conferences” using system 120 (eg, For example, it may be a request to the system 120 (by composing an email and sending it to a mailbox for the system 120 ). If the user 108 engages the system 120 to discover how the user 108 could otherwise use the system 120 , this would indicate that the user 108 is relatively satisfied with the system 120 . or may indicate that additional or other functionality is enabled or should be added to the system 120 .

Thus, it can be seen that it is not only direct feedback on the performance or satisfaction level of the system 120 that is helpful in determining the user's satisfaction level. Instead, the content of all messages received by system 120 may be analyzed to determine whether the messages represent a level of user satisfaction. Communications received by the system 120 but not seeking involvement, as well as communications with the system 120 directly seeking involvement may be helpful in this regard. In addition, not only communication content from various users may be used to determine user satisfaction, but also user actions may be used. In one embodiment, the user satisfaction signal analyzer and control system 134 captures signals indicative of communication content and user actions, and identifies one or more user satisfaction levels. A control signal may be generated to take an action or control step based on the identified user satisfaction.

For example, the user interest detection logic 138 may detect when the content of the user's message indicates that the user is interested in a particular function or feature or portion of the system 120 . As an example, the user may input "This ACME Scheduling Assistant is very complicated to use." This may indicate that the user is interested in an interface provided by system 120 , or other parts of an interactive process. The emotion analysis logic 140 may be used to detect the user's emotion from the user's message content. For example, there are many different types of sentiment analyzers that can be used to analyze sentiment in text. Sentiment analysis logic 140 exemplarily determines a correlation between user emotion identified in the user's message and system 120 . If emotions are closely correlated to system 120 , the message may represent the user's feelings about the performance of system 120 . The user recommendation detection logic 142 exemplarily detects when a user recommends other users to use the system 120 . It may be possible to detect instances where a user is attempting to influence other users not to use the system 120 .

Calendar mining logic 144 illustratively mines calendar data in calendar data store 126 for various users using system 120 . The meeting classification logic 152 may identify different meeting categories in the user's calendar (eg, face-to-face meetings, conference calls, business lunch meetings, personal meetings, etc.). The service coverage determination logic 154 may indicate, in different categories, how often the user 108 used the system 120 to schedule a corresponding meeting, relative to how often the user was not using the system 120 . scale can be determined. Trend identification logic 156 exemplarily identifies usage trends by user 108 (or other users of system 120 ) over time.

The survey logic 158 may be used to control the scheduling assistant workflow processing system 132 to request the user to complete a survey that asks more detailed questions regarding the user's level of satisfaction with the system 120 . . Users may be identified to participate in a survey based on a level of satisfaction determined for the user, based on the user's use of the system 120 , or otherwise.

Feature discovery engagement logic 146 identifies communications engaging system 120 , illustratively in such a way that user 108 seeks to identify additional features that may be provided by system 120 . This category of engagement with system 120 may indicate user satisfaction.

Satisfaction level detection logic 145 may include, for example, user interest detection logic 138 , sentiment analysis logic 140 , user recommendation detection logic 142 , calendar mining logic 144 , survey logic 158 , features Given the information detected by one or more of the discovery engagement logic 146 , and any other items of information, detect a level of satisfaction for the user 108 . The issue identification logic 147 may use the same or different information to identify any particular issue that the user may be interested in interacting with and using the system 120 .

The control signal generator logic 148 may then generate various control signals based on a level of satisfaction for a given user, based on issues identified for a given user, or other items. For example, the shared prompting logic 162 controls the scheduling assistant workflow processing system 132 to create a communication with the user 108 prompting or suggesting that the user recommends the system 120 to other users. can do. For example, if a user is determined to be very satisfied with the system 120 , it may be possible for the user to recommend the system 120 to other users in accordance with that suggestion or prompt. A wide variety of other control signals may be generated based on the identified level of user satisfaction and any issues identified for a particular user.

FIG. 2 is a flow diagram illustrating in more detail one embodiment of the operation of the architecture shown in FIG. 1 ; 1 and 2 will now be described in conjunction with each other.

First, assume that scheduling assistance system 120 is running and available for interaction by user 108 . This is indicated by block 180 in the flowchart of FIG. As noted above, system 120 may be a software agent, or an intelligent digital assistant or web bot or other such computing system. Exemplary is a system that receives communications from a group of users attempting to use a service provided by the system. The system receives communications from the users if they have requested the system 120 to assist in providing a service, even if the users are not directly attempting to interact with the system 120 .

Thereafter, the natural language understanding system 130 detects a request message from a user requesting to use the scheduling assistance system 120 , illustratively. This is indicated by block 182 . As an example, the system 120 may detect an email message sent to its inbox as indicated by block 184 . The system may detect a voice command 186 or other type of communication 188 . The natural language understanding system 130 then identifies the linguistic meaning of the request as a request for assistance by the system 120 .

The scheduling assistant workflow processing system 130 then conducts a conversation (eg, by participating in a discussion with a group of users) to assist in scheduling something for users based on the request. Of course, if system 120 is another type of assistant, bot, computer agent, etc., it conducts a conversation to assist in providing the other type of service. Conversation performance is indicated by block 190 in the flowchart of FIG. 2 .

In one embodiment, the scheduling assistant workflow processing system 132 sends and receives scheduling-specific messages (or messages that users are attempting to interact with the system 120 ). This is indicated by block 192 . Also, in one embodiment, system 120 receives a non-scheduling-specific message (or a message stating that users may be communicating with each other but are not attempting to interact with system 120 ). This is indicated by block 194 . Again, as noted above, this may be a message that users are communicating with each other, but not necessarily the system 120 intends to act on, or interact with these messages.

The natural language understanding system 130 then performs natural language understanding to obtain the linguistic meaning of the message content of all received messages (not just the messages for which interaction with the system 120 is intended or sought). Obtaining the linguistic meaning of the message content is indicated by block 196 . Conversation may of course be conducted in a wide variety of other ways, as indicated by block 198 .

User interest detection logic 138 receives the linguistic meaning of the content and performs linguistic processing on all incoming messages to detect any user interests for system 120 . This is indicated by block 200 . For example, it may be that certain words (or other linguistic units) trigger a mapping or rule that maps to issues of interest. When the user interest detection logic 138 finds the word in the linguistic meaning of the content of the message, it accesses the corresponding mapping or rule to identify the interest expressed by the user. User interests may be identified by a dynamic model or in other manners as well.

In addition, the emotion analysis logic 140 performs emotion analysis on the content of the received message. For example, based on the content of the received message, different emotion categories may be identified, such as preference, strong preference, non-preference, strong non-preference, negative, positive, angry, or a wide variety of other emotions. Performing sentiment analysis to detect emotions in all incoming messages is indicated by block 202 . User interest detection logic 138 and sentiment analysis logic 140 both provide output signals representative of, illustratively, any detected sentiments and user interests. In one embodiment, they correlate information to a particular user, while correlating it to any particular action category or characteristic of the assistance system 120 . As an example, a user may be commenting that a particular feature of the system 120 is very useful, not very useful, and the like. In that case, the user interest detection logic 138 may identify a particular characteristic of interest to the user, as described above, and the emotion analysis logic 140 may generate a strong positive or strong negative ( or otherwise).

In addition, the user recommendation detection logic 142 detects any message that the user is recommending the system 120 for use by other users. It also generates an output signal indicative of the fact that the user has recommended the system 120 . Other information may also be provided, such as, among other things, the number of users who have been recommended to system 120 . Detecting any message that the user recommends a service (eg, system 120 ) to another user is indicated by block 204 .

The calendar mining logic 144 then mines the calendar information in the calendar data store 126 to generate a measure of how often the service is being used by users, as compared to other methods of scheduling. This is indicated by block 206 . In one embodiment, system 120 may have access to the user's calendar data and thus count the number of scheduled meetings or other calendar items. It also tracks which of those items have, for example, been scheduled by the user 108 using the system 120 . A metric can then be created that indicates how much time the user 108 is using the system 120 to schedule different calendar items.

In one embodiment, the meeting classification logic 152 detects different usage scenarios, such as different categories of meetings that the user is scheduling. This may include business meetings, personal meetings, phone calls, face-to-face meetings, and remote meetings. Detecting different usage scenarios or categories is indicated by block 208 .

The service coverage determination logic 154 then determines a rate or other usage measure in each of the identified categories. For example, user 108 spends relatively more time (compared to a user scheduling without using system 120) to schedule a personal meeting and relatively little time to schedule a business meeting. may be that you are using Determining the percentage of usage in each category or scenario is indicated by block 210 .

Trend identification logic 156 is illustratively configured to determine whether usage of user 108 in different categories is increasing or decreasing as a percentage of the number of meetings or calendar items scheduled in those different categories. , aggregate usage rates (or coverage information) over time. This is indicated by block 212 . Detecting or generating a measure of how often a service is being used as compared to scheduling in other ways may be performed in a wide variety of different ways, in addition to or in lieu of those described above. This is indicated by block 214 .

The calendar mining logic 144 generates output signals representative of the various examples detected. For example, it may output a signal representing different meeting categories or scenarios identified by logic 152 . It may output a signal representing the service coverage determined by logic 124 , and various trends identified by logic 156 . A wide variety of other information can also be output.

In addition, the feature discovery involvement logic 146 detects when the user engages the service 120 to discover additional functionality, for example. This is indicated by block 216 . For example, this logic may identify different categories or types of engagement that user 108 uses to engage with system 120 . This type of engagement may include engaging the system 120 to assist in scheduling a business meeting, for example. It may also include involvement to inquire about what other types of services or features the system 120 may provide. Identification of an engagement as having no scheduling intent (or an engagement in which the user does not attempt to schedule a meeting using the system 120 ) is indicated by block 218 . Various types of involvement may be classified into different categories as indicated by block 220 . A specific engagement may then be identified that inquires what various features may be provided by the system 120 . This is indicated by block 222 . The number of engagements (or the level of that engagement as measured in some other way) may indicate user satisfaction. Detecting involvement in a service to discover additional functionality may also be performed in other ways. This is indicated by block 224 .

Additionally, the feature discovery involvement logic 146 may generate a variety of different output signals. For example, this logic may generate an output signal indicative of the different engagement categories being used by a particular user. This logic may output a signal indicating the level of involvement for which the user is seeking additional features. Other signals can also be output.

Based on the different logic portions described above or various signals output by the system, the satisfaction level detection logic 145 determines, illustratively, a user satisfaction level for the scheduling assistant system 120 . This is indicated by block 226 in the flowchart of FIG. Again, this may be based on detected interests, user sentiment, recommendations, measures of coverage, non-scheduling engagements, and a wide variety of other items. This is indicated by block 228 .

Further, satisfaction level detection logic 145 may continue to determine additional detailed information. For example, this logic may output an output indicative of a given user's satisfaction with each of the different categories or features of the system 120 , or using the system 120 in each of a plurality of different ways or under different scenarios. can create This is indicated by block 230 . User satisfaction may also be determined in a wide variety of other ways, as indicated by block 232 .

Thereafter, the satisfaction level detection logic 145 outputs one or more signals representative of the exemplary determined user satisfaction. For example, this logic may output a signal indicative of the overall user satisfaction of the user 108 with the system 120 . This logic may output a signal indicative of user satisfaction corresponding to different features, a tendency to correspond to different features or categories or usage scenarios across user satisfaction. In addition, a signal indicating how much trust is in the identified user satisfaction level may be output. For example, if the user satisfaction level is determined based on a relatively large amount of data, the system will be more reliable. In addition, if the content or action of the message is less ambiguous, it goes without saying that the reliability of the determined level of satisfaction may be higher.

Based on the information output by satisfaction level detection logic 145 , and any other information, control signal generator logic 148 generates one or more control signals. This is indicated by block 234 . For example, logic 148 may control survey logic 158 , or user interface logic 118 , or both, to perform a natural language survey of one or more users based on user satisfaction. This is indicated by block 236 . Share prompting logic 162 may control user interface logic 118 to suggest a particular user to recommend a service to other users. This is indicated by block 238 . For example, this logic may use the user interface logic 118 to generate a message (eg, an email message or other message) and display the message to the user 108 so that the user 108 can use one or more other messages. A user may be offered recommendations for use of the system 120 . This logic may simply generate the message as a predefined suggestion that the user 108 needs to pass on to other users. The message may have links through which other users may easily configure system 120 themselves, or otherwise.

In addition, the control signal generator logic 148 may illustratively control the communication system 116 to generate an intervening communication guaranteed by a user satisfaction level. For example, if the user is highly dissatisfied, the logic 148 may control the communication system 116 to generate a communication to the dissatisfied user, or to a support person who may contact the dissatisfied user. Controlling the communication system 116 to generate intervening communications to users, support personnel, etc. is indicated by block 242 .

In addition, the control signal generator logic 148 may, illustratively, aggregate the results of various logical items and systems for multiple users, and control provided to a remote system or other portion of the computing system to display the aggregated results. It can also generate a signal. This is indicated by block 244 . For example, aggregation of results may reveal that a relatively large number of users have issues with the same feature or set of functions on system 120 . It may also indicate that a relatively large number of users are seeking one or more additional functional items from the system 120 . It may indicate that a relatively large number of users have the same type of issue or interest with respect to the system 120 . This, and a wide variety of other information, can be obtained from the aggregated satisfaction results.

Control signal generator logic 148 may generate control signals in a wide variety of other ways. This is indicated by block 246 .

Accordingly, it can be seen that the present discussion enhances the computing system itself. The computing system is configured to capture a series of signals that can be used or combined to identify user satisfaction with different features or different portions of the computing system. This can be obtained from communications that users are attempting to engage with the computing system, as well as when users are not attempting to engage directly with the system. Also, a wide variety of different types of control signals may be generated based on the captured signal.

Note that the above discussion has described a variety of different systems, components, and/or logic. Hardware items in which such systems, components, and/or logic perform functions associated with these systems, components, and/or logic (eg, a processor and associated memory, or other processing components, some of which are described below It will be understood that it may consist of Further, systems, components, and/or logic may consist of software loaded into memory and then executed by a processor or server, or other computing component, as described below. In addition, systems, components, and/or logic may be configured in different combinations of hardware, software, firmware, and the like, some embodiments of which are described below. These are just some embodiments of different structures that may be used to form the systems, components and/or logic described above. Other structures may also be used.

This discussion referred to processors and servers. In one embodiment, the processor and server include a computer processor with associated memory and timing circuitry not separately shown. These are the functional parts of a system or device to which they belong and which are activated by it, and enable the functionality of other components or items in such a system.

A number of user interface displays have also been discussed. They can take a wide variety of different forms, and a wide variety of different user-actuated input mechanisms can be placed therein. For example, the user-actuated input mechanism may be a text box, check box, icon, link, drop-down menu, search box, and the like. They may be operated in a wide variety of different ways. For example, they may be operated using a point and click device (eg, a track ball or mouse). These may be operated using hardware buttons, switches, joysticks or keyboards, thumb switches or thumb pads, and the like. They may also be operated using a virtual keyboard or other virtual actuator. Also, if the screen on which they are displayed is a touch-sensitive screen, they can be actuated using touch gestures. Also, if the device displaying them has a voice recognition component, they can be operated using voice commands.

A number of data stores have also been discussed. Note that each of these can be divided into multiple data stores. All may be local to the system accessing them, all may be remote, or some may be local and others may be remote. All of these configurations are contemplated herein.

Also, the figure shows a number of blocks in which a function is assigned to each block. Note that functions are performed by fewer components as fewer blocks can be used. Also, more blocks can be used if functionality is distributed among more components.

FIG. 3 is a block diagram of the architecture 100 shown in FIG. 1 , except that elements are placed in a cloud computing architecture 500 . Cloud computing provides compute, software, data access, and storage services that do not require the end user to know the physical location or configuration of the system providing the service. In various embodiments, cloud computing provides services over a wide area network, such as the Internet, using appropriate protocols. For example, a cloud computing provider may provide the application via a wide area network, and may be accessed via a web browser or any other computing component. The software or components of architecture 100 as well as corresponding data may be stored on a server at a remote location. Computing resources in a cloud computing environment may be consolidated at remote data center locations or they may be distributed. Cloud computing infrastructure can provide services through a shared data center, even if it appears to the user as a single point of access. Accordingly, the components and functions described herein may be provided from a service provider at a remote location using a cloud computing architecture. Alternatively, they may be provided from a conventional server, or may be installed directly or otherwise on the client device.

This description is intended to encompass both public cloud computing and private cloud computing. Cloud computing (both public and private) not only provides substantially seamless pooling of resources, but also reduces the need to manage and configure the underlying hardware infrastructure.

Public clouds are managed by a vendor and typically serve multiple consumers using the same infrastructure. Also, unlike private clouds, public clouds can free end users from hardware management. The private cloud can be managed by the grid itself, and the infrastructure is usually not shared with other grids. The system still maintains the hardware to some extent, such as installation and repair.

3 , some items are similar to those shown in FIG. 1 and are numbered similarly. 3 specifically shows that computing system 102 may be located in cloud 502 (which may be a public cloud, a private cloud, or a combination of some public clouds and others private clouds). Accordingly, the user 108 uses the user device 504 to access such a system via the cloud 502 .

3 also depicts another embodiment of a cloud architecture. 3 illustrates that it is contemplated that some elements of computing system 102 may be located in cloud 502 , while other elements may not. By way of example, data store 126 may be located external to cloud 502 and accessed through cloud 502 . In other embodiments, system 134 (or other item) may be external to cloud 502 . Regardless of where they are located, they may be accessed directly by the device 504 over a network (wide area network or local area network), they may be hosted at a remote location by a service, or they may be via the cloud It may be provided as a service or accessed by a connection service residing in the cloud. Both of these architectures are contemplated herein.

It is also noted that architecture 100, or portions thereof, may be deployed in a wide variety of different devices. Some of these devices include servers, desktop computers, laptop computers, tablet computers, or other mobile devices such as palmtop computers, cell phones, smartphones, multimedia players, personal digital assistants, and the like.

4 is a simplified block diagram of one exemplary embodiment of a handheld or mobile computing device that may be used as the handheld device 16 of a user or client on which the present system (or portions thereof) may be deployed. 5 and 6 are embodiments of a handheld or mobile device.

4 is a general block diagram of components of a component computing system 102 or a client device 16 capable of executing a user device 504 or system 134 or interacting with the architecture 100, or both. provides a diagram In device 16, there is a communication link 13 that enables the handheld device to communicate with other computing devices and, in some embodiments, provides a channel for automatically receiving information, for example, by scanning. is provided Embodiments of the communication link 13 include an infrared port, a serial/USB port, a cable network port such as an Ethernet port, and General Packet Radio Service (GPRS), LTE, a wireless service used to provide cellular access to the network; Allows communication over one or more communication protocols, including HSPA, HSPA+ and other 3G and 4G wireless protocols, 1Xrtt, and short message services, as well as the Wi-Fi protocol that provides local wireless connectivity to networks, and the Bluetooth protocol. Includes a wireless network port.

In other embodiments, the application or system is received on a removable SD card coupled to a secure digital (SD) card interface 15 . SD card interface 15 and communication link 13 connect bus 19 to memory 21 and input/output (I/O) components 23 as well as clock 25 and positioning system 27 and a processor 17 (which may also embodied a processor or server from other figures).

In one embodiment, an I/O component 23 is provided to enable input and output operations. I/O components 23 for various embodiments of device 16 include buttons, touch sensors, multi-touch sensors, optical or video sensors, voice sensors, touch screens, proximity sensors, microphones, tilt sensors, and It may include input components such as gravity switches, and output components such as display devices, speakers, and/or printer ports. Other I/O components 23 may also be used.

Clock 25 illustratively includes a real-time clock component that outputs time and date. In addition, a timing function may be provided to the processor 17 by way of example.

The positioning system 27 illustratively includes a component that outputs the current geographic location of the device 16 . This may include, for example, a global positioning system (GPS) receiver, a LORAN system, a dead reckoning system, a cellular triangulation system, or other positioning system. It may include, for example, mapping software or navigation software that generates desired maps, navigation routes and other geographic functions.

Memory 21 stores operating system 29 , network settings 31 , applications 33 , application configuration settings 35 , data store 37 , communication drivers 39 , and communication configuration settings 41 . do. Memory 21 may include all types of tangible volatile and non-volatile computer-readable memory devices. It may also include computer storage media (discussed below). Memory 21 stores computer readable instructions that, when executed by processor 17 , cause the processor to perform computer-implemented steps or functions according to the instructions. Similarly, device 16 may have a client system 24 that may execute various applications or may embody some or all of architecture 100 . The processor 17 may also be activated by other components to enable its function.

Embodiments of network settings 31 include such things as proxy information, Internet connection information, and mapping. Application configuration settings 35 include settings for tailoring an application to a specific enterprise or user. Communication configuration setting 41 provides parameters for communication with other computers and includes items such as GPRS parameters, SMS parameters, connection user name and password.

Applications 33 may be applications previously stored on device 16 or applications installed during use, but they may be part of operating system 29 or may be hosted external to device 16 .

5 shows an embodiment in which the device 16 is a tablet computer 600 . In FIG. 5 , a computer 600 is shown with a user interface display screen 602 . Screen 602 may be a touch screen (thus, touch gestures from the user's finger may be used to interact with the application), or may be a pen-enabled interface that receives input from a pen or stylus. An on-screen virtual keyboard can also be used. Of course, it may also be attached to a keyboard or other user input device via an appropriate attachment mechanism, such as, for example, a wireless link or USB port. Also, the computer 600 may, for example, receive a voice input.

6 shows that the device may be a smartphone 71 . The smartphone 71 has a touch-sensitive display 73 that displays an icon or tile or other user input mechanism 75 . Mechanism 75 may be used by a user to launch an application, make a phone call, perform a data transfer operation, and the like. In general, a smartphone 71 is based on a mobile operating system and provides more advanced computing power and connectivity than a feature phone.

Note that other types of devices 16 are possible.

7 is an embodiment of a computing environment in which architecture 100, or a portion thereof, may be (eg) deployed. Referring to FIG. 7 , an exemplary system for implementing some embodiments includes a general purpose computing device in the form of a computer 810 . The components of computer 810 include processing unit 820 (which may include a processor or server from the previous figures), system memory 830, and various system components, including system memory, processing unit 820 . may include, but is not limited to, a system bus 821 that couples to The system bus 821 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures are also referred to as Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Mezzanine bus. Includes the known Peripheral Component Interconnect (PCI) bus. The memory and programs described with respect to FIG. 1 may be located in corresponding portions of FIG. 7 .

Computer 810 generally includes a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by computer 810 and includes both volatile and nonvolatile media, and removable and non-removable media. By way of example, and not limitation, computer-readable media may include computer storage media and communication media. Computer storage media is different from, and does not include, a modulated data signal or carrier wave. Computer storage media includes hardware storage including both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. includes media. A computer storage medium may include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device, or including, but not limited to, any other medium that can be used to store desired information and that can be accessed by computer 810 . Communication media generally embodies computer readable instructions, data structures, program modules, or other data as a transport mechanism and includes any information delivery media. The term "modulated data signal" means a signal having one or more characteristics that are set or changed in such a way as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of any of the above should also be included within the scope of computer-readable media.

System memory 830 includes computer storage media in the form of volatile and/or non-volatile memory, such as read only memory (ROM) 831 and random access memory (RAM) 832 . For example, during startup, basic input/output system 833 (BIOS) containing basic routines that help transfer information between elements inside computer 810 are typically stored in ROM 831 . to be. RAM 832 generally contains data and/or program modules that are immediately accessible and/or currently being operated on by processing unit 820 . By way of example and not limitation, FIG. 7 illustrates an operating system 834 , application programs 835 , other program modules 836 , and program data 837 .

In addition, computer 810 may include other removable/removable, volatile/non-volatile computer storage media. By way of example only, FIG. 7 shows a hard disk drive 841 that reads from or writes to a fixed nonvolatile magnetic medium, and a hard disk drive 841 that reads or writes to or from a removable nonvolatile optical disk 856 such as a CD ROM or other optical medium. An optical disk drive 855 is illustrated. Other removable/removable volatile/nonvolatile computer storage media that may be used in the exemplary operating environment include, but are not limited to, magnetic tape cassettes, flash memory cards, digital versatile disks, digital video tapes, solid state RAM, solid state ROM, and the like. is not limited to Hard disk drive 841 is typically connected to system bus 821 via a fixed memory interface, such as interface 840 , and optical disk drive 855 is typically connected to the system via a removable memory interface, such as interface 850 . connected to the bus 821 .

Alternatively, or additionally, the functions described herein may be performed, at least in part, by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that may be used include Field-Programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Program-specific Standard Products (ASSPs), System-on-SOCs (SOCs). a-chip systems), and Complex Programmable Logic Devices (CPLD).

The drives discussed above and illustrated in FIG. 7 and their associated computer storage media provide storage of computer readable instructions, data structures, program modules, and other data to the computer 810 . In FIG. 7 , for example, a hard disk drive 841 is illustrated as storing an operating system 844 , application programs 845 , other program modules 846 , and program data 847 . Note that these components may be the same as or different from the operating system 834 , application programs 835 , other program modules 836 , and program data 837 . Operating system 844 , application programs 845 , other program modules 846 , and program data 847 are given different numbers herein to illustrate, at a minimum, that they are different copies.

A user may enter commands and information into the computer 810 through input devices such as a keyboard 862 , a microphone 863 , and a pointing device 861 , such as a mouse, trackball, or touch pad. Other input devices (not shown) may include joysticks, gamepads, satellite antennas, scanners, and the like. These and other input devices are connected to the processing unit 820 through a user input interface 860, which is often coupled to a system bus, but other interface and bus structures such as a parallel port, game port, or Universal Serial Bus (USB). can be connected by A visual display 891 or other type of display device is also coupled to the system bus 821 via an interface, such as a video interface 890 . In addition to the monitor, the computer may also include other peripheral output devices, such as speakers 897 and printer 896 , which may be connected via an output peripheral interface 895 .

Computer 810 operates in a networked environment using logical connections to one or more remote computers, such as remote computer 880 . Remote computer 880 may be a personal computer, handheld device, server, router, network PC, peer device, or other common network node, and may generally include many or all of the elements described above with respect to computer 810 . may include The logical connections depicted in FIG. 7 include a LAN 871 and a WAN 873, but may include other networks. Such networking environments are common in offices, enterprise computer networks, intranets, and the Internet.

When used in a LAN networking environment, computer 810 is connected to LAN 871 through a network interface or adapter 870 . When used in a WAN networking environment, computer 810 typically includes a modem 872 or other means for establishing communications over a WAN 873, such as the Internet. Modem 872 , which may be internal or external, may be coupled to system bus 821 via user input interface 860 , or other suitable mechanism. In a networked environment, program modules depicted relative to computer 810 or portions thereof may be stored in a remote memory storage device. By way of example and not limitation, FIG. 7 illustrates a remote application program 885 residing on a remote computer 880 . It will be appreciated that the network connections shown are exemplary and other means of establishing a communication link between the computers may be used.

It should also be noted that the different embodiments described herein may be combined in different ways. That is, portions of one or more embodiments may be combined with portions of one or more other embodiments. All of these are contemplated herein.

Embodiment 1 is a computing system comprising:

a messaging system for sending and receiving group messages among a plurality of different users within a group of users;

Receive from a messaging system a natural language request message representing a user request from a requesting user within a group of users to interact with the assistance system, and in response to the natural language request message, perform a conversation with the user using the messaging system to provide a service a rendering assistance computing system;

Detect, from a plurality of different users in a user group, in addition to the natural language request message and messages in the conversation, other natural language messages, and provide assistance based on the content of the other natural language messages, the natural language request message and messages in the conversation an analyzer system that generates a satisfaction indication indicative of user satisfaction corresponding to the computing system; and

and control signal generator logic to generate a control signal for controlling the computing system based on the indication of satisfaction.

Embodiment 2 is the computing system of any or all previous embodiments, wherein the control signal generator logic is configured to generate a control signal for controlling the messaging system to prompt the requesting user to recommend the assistance computing system to another user.

Embodiment 3 is the computing system of any or all previous embodiments, wherein the analyzer system comprises:

feature discovery engagement logic configured to identify a feature discovery natural language message that inquires about additional functionality provided by the assistance computing system and generate a feature discovery engagement indicator indicative of the feature discovery natural language message; and

and satisfaction level detection logic configured to generate an indication of satisfaction based on the feature discovery involvement indicator.

Embodiment 4 is the computing system of any or all previous embodiments, wherein the assistance computing system comprises a scheduling assistant configured to render a scheduling assistance service, the analyzer system comprising:

calendar mining logic configured to access calendar information corresponding to the requesting user to identify a usage level indicative of a level of usage of the scheduling assistant by the requesting user; and

and satisfaction level detection logic configured to generate the satisfaction indication based on the usage level.

Embodiment 5 is the computing system of any or all previous embodiments, wherein the calendar mining logic comprises:

access calendar information, and identify how often the requesting user used the scheduling assistant in performing the scheduling action relative to how often the user performed the scheduling action without using the scheduling assistant to obtain a coverage indicator and service coverage determination logic, wherein the satisfaction level detection logic generates an indication of satisfaction based on the coverage indicator.

Embodiment 6 is the computing system of any or all previous embodiments, wherein the calendar mining logic comprises:

and meeting classification logic configured to access calendar information and identify different categories of scheduling items for which the requesting user performed a scheduling operation.

Embodiment 7 is the computing system of any or all previous embodiments, wherein the service coverage determining logic is configured to obtain a coverage indicator corresponding to each category of scheduling items.

Embodiment 8 is the computing system of any or all previous embodiments, wherein the calendar mining logic comprises:

and trend identification logic configured to identify a trend of how often the requesting user used the scheduling assistant in performing the scheduling operation relative to how often the user performed the scheduling operation without using the scheduling assistant.

Embodiment 9 is the computing system of any or all previous embodiments, wherein the analyzer system comprises:

user recommendation detection logic configured to detect whether the requesting user has recommended the assistance computing system to other users to generate a recommendation detection metric, wherein the satisfaction level detection logic generates a satisfaction indication based on the recommendation detection metric; configured to do

Embodiment 10 is the computing system of any or all previous embodiments, wherein the analyzer system comprises:

and sentiment analysis logic configured to identify an emotion of the requesting user in the content of the messages and to identify a correlation between the identified emotion and the assistance computing system, wherein the satisfaction level detection logic is configured to: A satisfaction indication is generated based on the correlation.

Embodiment 11 is the computing system of any or all previous embodiments, wherein the sentiment analysis logic is configured to identify an emotion of the requesting user in the content of the messages and to identify a correlation between the identified emotion and a respective characteristic of the assistance computing system. and the satisfaction level detection logic generates an indication of satisfaction based on the identified emotions and correlations to individual characteristics of the assistance computing system.

Embodiment 12 is a computer implemented method comprising:

In the assistance computing system, receive a natural language request message from a messaging system that sends and receives group messages among a plurality of different users within the group of users, wherein the natural language request message is a requesting user within a group of users for interaction with the assistance system. indicating a user request from;

controlling the messaging system to render a service by performing a conversation with the user in response to the natural language request message;

detecting, from a plurality of different users within the user group, in addition to the natural language request message and messages in the conversation, other natural language messages;

generating a satisfaction indication indicative of user satisfaction corresponding to the assistance computing system based on content of other natural language messages, natural language request messages, and messages in the conversation; and

and generating a control signal for controlling the computing system based on the indication of satisfaction.

Embodiment 13 is the computer-implemented method of any or all previous embodiments, wherein generating a control signal comprises:

generating a control signal for controlling the messaging system to prompt the requesting user to recommend the assistance computing system to another user.

Embodiment 14 is the computer-implemented method of any or all previous embodiments, wherein generating the satisfaction indication comprises:

identifying a feature discovery natural language message that inquires about additional functionality provided by the assistance computing system;

generating a feature discovery involvement indicator representing a feature discovery natural language message; and

and generating an indication of satisfaction based on the feature discovery engagement index.

Embodiment 15 is the computer-implemented method of any or all previous embodiments, wherein the assistance computing system comprises a scheduling assistant configured to render a scheduling assistance service, wherein generating the satisfaction indication comprises:

accessing calendar information corresponding to the requesting user;

identifying a usage level indicative of a level of usage of the scheduling assistant by the requesting user; and

and generating an indication of satisfaction based on the level of usage.

Embodiment 16 is the computer-implemented method of any or all previous embodiments, wherein identifying the usage level comprises:

identifying, from the calendar information, a level of usage, by identifying how often the requesting user used the scheduling assistant in performing the scheduling operation relative to how often the user performed the scheduling operation without using the scheduling assistant; and

and identifying a coverage metric based on the level of usage.

Embodiment 17 is the computer-implemented method of any or all previous embodiments, wherein identifying the usage level comprises:

identifying, from the calendar information, different categories of scheduling items for which the requesting user performed a scheduling operation; and

and identifying a coverage indicator corresponding to each category of scheduling items.

Embodiment 18 is the computer-implemented method of any or all previous embodiments, comprising:

In each of the categories of scheduling items, the categories of scheduling items, a trend of how often the requesting user used the scheduling assistant in performing the scheduling operation, relative to how often the user performed the scheduling operation without using the scheduling assistant. The method further includes identifying based on a set of coverage indicators corresponding to each.

Embodiment 19 is the computer-implemented method of any or all previous embodiments, wherein generating the satisfaction indication comprises:

detecting whether the requesting user has recommended the assistance computing system to other users;

generating a recommended detection indicator; and

and generating an indication of satisfaction based on the recommended detection indicator.

Embodiment 20 is the computer-implemented method of any or all previous embodiments, wherein generating the satisfaction indication comprises:

identifying the emotion of the requesting user in the content of the messages;

identifying a correlation between the identified emotion and the assistance computing system; and

generating an indication of satisfaction based on the identified emotion and the correlation to the assistance computing system.

Embodiment 21 is a computing system comprising:

a software agent that receives a natural language request message indicating the interaction request and provides a service based on the natural language interaction request;

a messaging system for conducting a conversation having a series of messages sent from a user to a software agent, wherein at least one of the series of messages comprises a natural language request message;

a signal analyzer that analyzes contents of the natural language request message and other messages in the series of messages and generates a satisfaction indication indicating user satisfaction corresponding to the assistance computing system based on contents of the natural language request message and other messages in the series of messages; and

and control signal generator logic to control the messaging system based on the satisfaction indication.

Although claimed subject matter has been described in language specific to structural features and/or methodological acts, it should be understood that the subject matter defined in the claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims (15)

A computing system comprising:
a messaging system for sending and receiving group messages among a plurality of different users within a group of users;
Receive from the messaging system a natural language request message representing a user request from a requesting user within the group of users to interact with an assistance computing system, and in response to the natural language request message, interact with a user using the messaging system the assistance computing system to render a service by performing ;
Detect, from a plurality of different users in the user group, in addition to the natural language request message and messages in the conversation, other natural language messages, and content of the other natural language messages, the natural language request message and messages in the conversation an analyzer system that generates a satisfaction indication indicative of user satisfaction corresponding to the assistance computing system based on ; and
and control signal generator logic to generate a control signal for controlling the computing system based on the satisfaction indication.
computing system.
According to claim 1,
wherein the control signal generator logic is configured to generate a control signal for controlling the messaging system to prompt the requesting user to recommend the assistance computing system to another user.
computing system.
According to claim 1,
The analyzer system comprises:
feature discovery engagement logic configured to identify a feature discovery natural language message that inquires about additional functionality provided by the assistance computing system and generate a feature discovery engagement indicator indicative of the feature discovery natural language message; and
satisfaction level detection logic configured to generate the satisfaction indication based on the feature discovery involvement indicator;
computing system.
According to claim 1,
The assistance computing system comprises a scheduling assistant configured to render a scheduling assistance service, the analyzer system comprising:
calendar mining logic configured to access calendar information corresponding to the requesting user to identify a usage level indicative of a level of usage of the scheduling assistant by the requesting user; and
satisfaction level detection logic configured to generate the satisfaction indication based on the usage level;
computing system.
5. The method of claim 4,
The calendar mining logic is:
how often the requesting user used the scheduling assistant in performing a scheduling operation to access the calendar information and how often the user performed a scheduling operation without using the scheduling assistant to obtain a coverage indicator service coverage determination logic configured to identify, wherein the satisfaction level detection logic generates the satisfaction indication based on the coverage metric.
computing system.
6. The method of claim 5,
The calendar mining logic is:
and meeting classification logic configured to access the calendar information and identify different categories of scheduling items for which the requesting user performed the scheduling operation.
computing system.
7. The method of claim 6,
wherein the service coverage determination logic is configured to obtain the coverage indicator corresponding to each category of scheduling items.
computing system.
6. The method of claim 5,
The calendar mining logic is:
and trend identification logic configured to identify a trend of how often the requesting user used the scheduling assistant in performing the scheduling operation with respect to how often the user performed the scheduling operation without using the scheduling assistant. doing
computing system.
4. The method of claim 3,
The analyzer system comprises:
and user recommendation detection logic, configured to detect whether the requesting user has recommended the assistance computing system to other users to generate a recommendation detection indicator, wherein the satisfaction level detection logic is configured to: configured to generate the satisfaction indication.
computing system.
4. The method of claim 3,
The analyzer system comprises:
and sentiment analysis logic configured to identify an emotion of the requesting user in the content of the messages and to identify a correlation between the identified emotion and the assistance computing system, wherein the satisfaction level detection logic is configured to: generating the satisfaction indication based on the correlation to the assistance computing system;
computing system.
11. The method of claim 10,
wherein the sentiment analysis logic is configured to identify an emotion of the requesting user in the content of the messages and to identify a correlation between the identified emotion and a respective characteristic of the assistance computing system, and wherein the satisfaction level detection logic is configured to: generating the satisfaction indication based on emotion and correlation to individual characteristics of the assistance computing system;
computing system.
A computer implemented method comprising:
In an assistance computing system, receive a natural language request message from a messaging system that sends and receives group messages among a plurality of different users within a group of users, wherein the natural language request message is configured for interaction with the assistance computing system. indicating a user request from a requesting user in the group;
controlling the messaging system to render a service by performing a conversation with a user in response to the natural language request message;
detecting, in addition to the natural language request message and messages in the conversation, other natural language messages from a plurality of different users in the user group;
generating a satisfaction indication indicative of user satisfaction corresponding to the assistance computing system based on content of the other natural language messages, the natural language request message, and messages in the conversation; and
generating a control signal for controlling the computing system based on the satisfaction indication;
How to implement a computer.
13. The method of claim 12,
The step of generating the control signal comprises:
generating a control signal for controlling the messaging system to prompt the requesting user to recommend the assistance computing system to another user;
How to implement a computer.
13. The method of claim 12,
The step of generating the satisfaction indication comprises:
identifying a feature discovery natural language message that inquires about additional functionality provided by the assistance computing system;
generating a feature discovery involvement indicator representing the feature discovery natural language message; and
generating the satisfaction indication based on the feature discovery involvement index
How to implement a computer.
A computing system comprising:
a software agent that receives a natural language request message indicating an interaction request and provides a service based on the natural language interaction request;
a messaging system for conducting a conversation having a series of messages sent from a user to the software agent, wherein at least one of the series of messages includes the natural language request message;
Analyze the contents of the natural language request message and other messages of the series of messages, and generate a satisfaction indication indicating user satisfaction corresponding to the assistance computing system based on the contents of the natural language request message and other messages of the series of messages signal analyzer; and
and control signal generator logic to control the messaging system based on the satisfaction indication.
computing system.

Images