CN107923762B

CN107923762B - Detecting and correcting potential errors in user behavior

Info

Publication number: CN107923762B
Application number: CN201680024572.0A
Authority: CN
Inventors: 雅各布·尼古劳斯·弗尔斯特; 弗雷德里克·彼得·布鲁因
Original assignee: Google LLC
Current assignee: Google LLC
Priority date: 2015-09-21
Filing date: 2016-06-27
Publication date: 2024-09-10
Anticipated expiration: 2036-06-27
Also published as: WO2017052708A1; US10650317B2; RU2694747C1; US20200234168A1; US20170083821A1; CN107923762A; EP3274659A1

Abstract

A computing system is described that predicts future actions to be taken by a user of a computing device and determines current actions to be taken by the user based on contextual information associated with the computing device. The computing system determines, based on the current action, a degree of likelihood of whether the user is able to take the future action and predicts, based on the degree of likelihood, that the user will not be able to take the future action. The computing system sends information to the computing device indicating that the current action taken by the user will result in the user failing to take the future action.

Description

Detecting and correcting potential errors in user behavior

Background

Some computing devices (e.g., wearable devices or mobile phones) may act as personal assistants configured to perform search queries for upcoming flights, nearby attractions, and other information that may be of interest to the user or to notify the user of upcoming flights, nearby attractions, and other information that may be of interest to the user. For example, the computing device may be able to access the user's digital calendar, and the computing device may alert the user when to begin traveling from the current location and arrive in time at a future meeting or event. Or in another example, the computing device may be able to access the user's shopping history, and the computing device may suggest certain products or services that are compatible with the products that the user purchased in the past, but not other products and services that are incompatible with the products purchased in the past. However, even with all useful reminder and access information provided by some computing devices, the reminder and access information does not always prevent individual users from making decisions that lead to mistakes in their daily lives and taking actions that lead to mistakes in their daily lives.

Disclosure of Invention

In one example, the present disclosure relates to a method comprising: predicting, by the computing system, future actions to be taken by a user of the computing device; determining, by the computing system, a current action to take by the user based on context information associated with the computing device; determining, by the computing system, a degree of likelihood of whether the user is able to take the future action based on the current action; and predicting, by the computing system, that the user will not be able to take future actions based on the degree of likelihood. The method further comprises the steps of: information is sent from the computing system to the computing device indicating that the current action taken by the user will result in the user being unable to take future actions.

In another example, the present disclosure is directed to a computing system comprising at least one processor; and at least one module operable by the at least one processor to: the method includes predicting future actions to be taken by a user of a computing device, determining a current action to be taken by the user based on context information associated with the computing device, and determining a degree of likelihood of whether the user is able to take the future action based on the current action. The at least one module is further operable by the at least one processor to: predicting that the user will not be able to take future actions based on the degree of likelihood, and sending information to the computing device indicating that the current action taken by the user will result in the user not being able to take future actions.

In another example, the disclosure relates to a computer-readable storage medium comprising instructions that, when executed, cause at least one processor of a computing system to: the method includes predicting future actions to be taken by a user of a computing device, determining a current action to be taken by the user based on context information associated with the computing device, and determining a degree of likelihood of whether the user is able to take the future action based on the current action. The instructions, when executed, cause the at least one processor of the computing system to further: predicting that the user will not be able to take future actions based on the degree of likelihood, and sending information to the computing device indicating that the current action taken by the user will result in the user not being able to take future actions.

The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the disclosure will be apparent from the description and drawings, and from the claims.

Drawings

FIG. 1 is a conceptual diagram illustrating an example system for predicting whether a user of a computing device is able to take future actions in accordance with one or more aspects of the present disclosure.

FIG. 2 is a block diagram illustrating an example computing system configured to predict whether a user of a computing device is able to take future actions in accordance with one or more aspects of the present disclosure.

FIG. 3 is a conceptual diagram illustrating an example navigation scenario in which an example computing system predicts whether a user of a computing device is able to take future actions in accordance with one or more aspects of the present disclosure.

FIG. 4 is a flowchart illustrating example operations performed by an example computing system configured to predict whether a user of a computing device is able to take future actions in accordance with one or more aspects of the present disclosure.

FIG. 5 is a conceptual diagram illustrating an example timeline in which an example computing system predicts whether a user of a computing device is able to take future actions in accordance with one or more aspects of the present disclosure.

Detailed Description

In general, the techniques of this disclosure may enable a computing system to infer a current action taken by a user of a computing device based on contextual information about the computing device and predict whether a user is able to take future actions by taking the current action. In response to determining that the user will not be able to take future actions by taking the current action, the computing system may send computing device information indicating such. In other words, by observing the actions of the user inferred from the context of the computing device, the computing system can determine when the user is not doing something that the computing system expects the user to do, and can send information to the computing device informing the user that he is doing something unexpected. In some examples, the computing system may provide information to the computing device indicating corrective actions that the user may employ to increase the likelihood that the user is able to take future actions. In other examples, the computing system may at least provide information to the computing device that informs the user that the user is doing something unexpected so that the user may confirm that he is potentially missing something or making an error.

Throughout this disclosure, examples are described in which a computing device and/or computing system analyzes information (e.g., context, location, speed, search queries, etc.) associated with the computing device and a user of the computing device whenever the computing device receives permission to analyze the information from the user of the computing device. For example, in the case discussed below, before a computing device or computing system can collect or otherwise make use of information associated with a user, the user may be provided with the following opportunities: inputs are provided to control whether a program or feature of a computing device and/or computing system may collect and utilize user information (e.g., information about a user's current location, current speed, etc.) or to instruct the device and/or system whether and/or how to receive content that may be relevant to the user. In addition, the particular data may be processed in one or more ways before it is stored or used by the computing device and/or computing system so that personally identifiable information may be removed. For example, the identity of the user may be processed such that personally identifiable information of the user cannot be determined, or the geographic location of the user may be generalized (such as to a city, zip code, or state level) where location information is available such that a particular location of the user cannot be determined. Thus, the user may control the manner in which information about the user is collected and/or used by the computing device and computing system.

FIG. 1 is a conceptual diagram illustrating a system 100 as an example system for predicting whether a user of a computing device 110 is able to take future actions in accordance with one or more aspects of the present disclosure. The system 100 includes an information server system ("ISS") 160 in communication with the computing device 100 via the network 130. ISS 160 may detect an intended action (e.g., a possible destination) associated with a user of computing device 100, then observe the behavior of the user and computing device 110 (e.g., using a location service), and intervene by alerting computing device 110 when ISS 160 deems a behavior error likely to be observed. ISS 160 may use a model for predicting the destination of a user of computing device 100 and observe the actual behavior of the user by using a location tracking service or a location history service. ISS 160 may compare the behavior to the expected behavior and if ISS 160 determines that the user is not currently taking an action that results in the expected behavior, ISS 160 may notify computing device 100 to alert the user that it may be making an error that may prevent the user from performing the expected action.

Network 130 represents any public or private communication network, such as cellular, wi-Fi, and/or other types of networks, for transmitting data between computing systems, servers, and computing devices. Network 130 may include one or more hubs, network switches, network routers, or any other network device operatively coupled to each other to enable the exchange of information between ISS 160 and computing devices 110. Computing device 110 and ISS 160 may use any suitable communication technology to transmit and receive data in network 130.

ISS 160 and computing device 110 may each be operatively coupled to network 130 using a respective network link. The links coupling computing device 110 and ISS 160 to network 130 may be ethernet, ATM, or other types of network connections, and such connections may be wireless and/or wired connections.

Computing device 110 represents a mobile computing device alone or a non-mobile computing device. Examples of computing device 110 include a mobile phone, a tablet computer, a laptop computer, a desktop computer, a server, a mainframe, a set-top box, a television, a computerized watch, a computerized eyewear, a computerized glove, or any other type of portable computing device, a Personal Digital Assistant (PDA), a portable gaming system, a media player, an electronic book reader, a mobile television platform, an automated navigation and entertainment system, or any other type of mobile, non-mobile, wearable, and non-wearable computing device configured to receive information via a network such as network 130.

Computing device 110 includes a User Interface Device (UID) 112 and a User Interface (UI) module 120. In addition, the computing device 110 includes a query module 122. The modules 120-122 may perform the described operations using software, hardware, firmware, or a mixture of hardware, software, and firmware residing in the respective computing device 110 and/or executing at the respective computing device 110. The computing device 110 may execute the modules 120-122 using multiple processors or multiple devices. The computing device 110 may act as virtual machine execution modules 120 through 122 executing on underlying hardware. Modules 120 through 122 may execute as one or more services of an operating system or computing platform. The modules 120-122 may be executed as one or more executable programs at an application layer of a computing platform.

UID 112 of computing device 110 may serve as an input and/or output device for computing device 110. UID 112 may be implemented using various techniques. For example, UID 112 may function as an input device using a presence-sensitive input screen, such as a resistive touch screen, a surface acoustic wave touch screen, a capacitive touch screen, a projected capacitive touch screen, a pressure-sensitive screen, an acoustic pulse recognition touch screen, or another presence-sensitive display technology. In addition, UID 112 may include microphone technology, infrared sensor technology, or other input device technology for receiving user input.

UID 112 may act as an output (e.g., display) device using any one or more display devices, such as a Liquid Crystal Display (LCD), a dot matrix display, a Light Emitting Diode (LED) display, an Organic Light Emitting Diode (OLED) display, electronic ink, or similar monochrome or color display capable of outputting visual information to a user of computing device 110. In addition, UID 112 may include speaker technology, haptic feedback technology, or other output device technology for outputting information to a user.

UID 112 may each include a respective presence-sensitive display that may receive tactile input from a user of computing device 110. UID 112 may receive indications of tactile input by detecting one or more gestures from a user (e.g., a user touching or pointing to one or more locations of UID 112 with a finger or a stylus pen). UID 112 may present output to the user, e.g., at a corresponding presence-sensitive display. UID 112 may present the output as a graphical user interface (e.g., user interface 114), which may be associated with functionality provided by computing device 110. For example, UID 112 may present various user interfaces (e.g., user interface 114) related to search functionality provided by query module 112 or other features of a computing platform, operating system, application, and/or service (e.g., an electronic messaging application, an internet browser application, a mobile operating system, or a desktop operating system, etc.) executing at computing device 110 or accessible from computing device 110.

UI module 120 may manage user interactions with UID 112 and other components of computing device 110. UI module 120 may cause UID 112 to output a user interface, such as user interface 114 (or other example user interfaces), for display as a user of computing device 110 views the output and/or provides the input at UID 112. UI module 120 and UID 112 may receive one or more indications of input from the user as the user interacts with the user interface at different times and when the user and computing device are in different locations. UI module 120 and UID 112 may interpret the input detected at UID 112 and may relay information regarding the input detected at UID 112 to one or more associated platforms, operating systems, applications, and/or services executing at computing device 110, e.g., to cause computing device 110 to perform functions.

UI module 120 may receive information and instructions from one or more associated platforms, operating systems, applications, and/or services executing at computing device 110 and/or one or more remote computing systems, such as ISS 160. Additionally, UI module 120 may act as middleware between the one or more associated platforms, operating systems, applications, and/or services executing at computing device 110 and various output devices of computing device 100 (e.g., speakers, LED indicators, audio or electrostatic haptic output devices, etc.) to generate output (e.g., graphics, flashing lights, sounds, haptic responses, etc.) with computing device 100.

In the example of fig. 1, user interface 114 is a graphical user interface associated with a predictive service provided by ISS 160 and accessed by computing device 100. As described in more detail below, the user interface 114 includes graphical information (e.g., text) that represents information that the ISS 160 predicts that a user of the computing device 110 may need to perform a future task at a current time or at least avoid making an error by taking a current action that prohibits the user from performing the future task. The user interface 14 may include various other types of graphical indications, such as visual descriptions of predicted information that a user of the computing device 110 may need to perform future tasks or at least avoid making errors by taking current actions that prohibit the user from performing future tasks. UI module 120 may cause UID 112 to output user interface 114 based on data received by UI module 120 from ISS 160 via network 130. UI module 120 may receive graphical information (e.g., text data, graphical data, etc.) for presenting user interface 114 as input from ISS 160 and instructions from ISS 160 for presenting the graphical information within user interface 114 at UID 112.

The notification module 122 performs notification management-related functions of the computing device 110. Notification module 122 may receive information data from applications and services executing at computing device 100 and data related to the notification from ISS 160, and in response, may output an indication of the information contained in the notification data to UI module 120 for presentation at UID 112. Notification module 120 may receive notification data from a predictive service provided by ISS 160 and accessed by computing device 110. The prediction service may send notification data to notification module 122 that includes information regarding corrective actions that a user of computing device 110 may take to correct the user's current actions and ensure that the user takes future actions. Notification module 120 may output information regarding the corrective action to UI module 120 for presentation at UID 112 (e.g., as user interface 114).

As used throughout this disclosure, the term "notification data" is used to describe various types of information that may indicate the occurrence of events associated with various platforms, applications, and services executing within an execution environment at one or more computing devices, such as computing device 110. For example, notification data may include, but is not limited to, information specifying events such as: receiving a communication message (e.g., email, instant message, SMS, etc.) through a messaging account associated with the computing device, receiving information through a social network account associated with the computing device 110, a reminder of a calendar event (meeting, appointment, etc.) associated with a calendar account of the computing device 110, information generated and/or received by a third party application executing at the computing device 100, transmission and/or reception of inter-component communications between two or more components of a platform, application, and/or service executing at the computing device 100, and so forth. In addition to including information about specific events, such as the various events described above, the notification data may include various attributes or parameters embedded within the notification data that specify various characteristics of the notification data. For example, notification data may include a portion (e.g., bits, metadata, fields, etc.) of the source (e.g., platform, application, and/or service that generated the notification data) of the specified notification data of the data.

ISS 160 represents any suitable remote computing system capable of sending information to and receiving information from a network (such as network 130), such as one or more desktop computers, laptop computers, mainframe computers, servers, cloud computing systems, and the like. ISS 160 hosts a search system for searching information and a prediction system for predicting, for a particular context, whether a user of computing device 110 is able to take future actions (or at least provide access to both).

Computing device 110 may communicate with ISS 160 via network 130 to access the search system and the prediction system provided by ISS 160. In some examples, ISS 160 represents a cloud computing system that provides search and prediction services to one or more computing devices 110 that access the search and prediction services via a cloud provided by the ISS through network 130.

In the example of fig. 1, ISS 160 includes a context module 162 and a prediction module 164. Together, modules 162 and 164 provide predictive services that are accessible to computing device 110 and other computing devices connected to network 130 for receiving useful information predicted by ISS 160 to be of interest to a user. Modules 162 and 164 may perform operations for modeling user behavior in order to predict when a user of a computing device may make mistakes (e.g., deviate from expected behavior) that may potentially have a significant impact on their lives.

Modules 162 and 164 may perform operations using software, hardware, firmware, or a mixture of hardware, software, and firmware residing in ISS 160 and/or executing at ISS 160. ISS 160 may execute modules 162 and 164 using multiple processors or multiple devices. ISS 160 may execute modules 162 and 164 as virtual machines executing on the underlying hardware. Modules 162 and 164 may execute as one or more services for an operating system or computing platform. Modules 162 and 164 may execute as one or more executable programs at an application layer of a computing platform.

As used throughout this disclosure, the term "contextual information" is used to describe information that may be used by computing systems and/or computing devices, such as information server system 160 and computing device 110, to define one or more physical environmental characteristics and/or virtual environmental characteristics associated with the computing device and/or user of the computing device, in addition to one or more observable physical or virtual actions taken by the user of the computing device at a particular time. In other words, the context information represents any data that can be used by a computing device and/or computing system to determine a "user context" that forms a circumstance of a virtual and/or physical experience that a user is experiencing at a particular time for a particular location.

Examples of context information include past physical locations, current physical locations, and future physical locations, degrees of movement, magnitudes of changes associated with movement, weather conditions, traffic conditions, travel patterns, movement patterns, application usage, calendar information, purchase history, internet browsing history, and the like. In some examples, the contextual information may include sensor information obtained by one or more sensors (e.g., gyroscopes, accelerometers, proximity sensors) of a computing device such as computing device 110, radio transmission information obtained from one or more communication units and/or radios of the computing device (e.g., global Positioning System (GPS), cellular, wi-Fi), information obtained by one or more input devices of the computing device (e.g., camera, microphone, keyboard, touch pad, mouse), and network/device identifier information (e.g., network name, device internet protocol address). In some examples, the context information may include communication information, such as information derived from an email message, a text message, a voice mail message, or a voice conversation, calendar entries, task lists, social media network related information, and any other information about the user or computing device that may support determining the user context.

The context module 162 may process and analyze information associated with the computing device 110 to define a context of the computing device 110. The context of computing device 110 may specify one or more characteristics associated with the user of computing device 110 and its physical environment and/or virtual environment at various locations and times. For example, the context module 162 may determine a physical location associated with the computing device 110 at a particular time as part of the context of the computing device 110 based on context information associated with the computing device 110 from the particular time. When the context information changes (e.g., based on sensor information indicating movement over time), the context module 162 may update the physical location in the determined context of the computing device 110.

The types of information defining the context of a computing device for a particular location and/or time are not enumerated. As some examples, the context of a computing device may specify: location, movement trajectory, direction, speed, organization name, street address, location type, building, weather conditions, and traffic conditions at various locations and times. The context of the computing device may further include calendar information defining meetings or events associated with various locations and times, web page addresses viewed at various locations and times, text entries made in data fields of web pages viewed at various locations and times (e.g., search or browsing history), and other application usage data associated with various locations and times. The context of the computing device may further include information about audio and/or video streams accessed by the computing device at various locations and times, television or cable/satellite broadcasts accessed by the computing device at various locations and times, and information about other services accessed by the computing device at various locations and times.

The context module 162 may maintain a past context history and a future context history associated with a user of the computing device 110. The context module 162 may catalog and record previous contexts of the computing device 110 at various locations and times in the past, and from the previously recorded contexts may project or infer future contexts of the computing device at various future locations and future times. The context module 162 may associate future dates and times with loop contexts of previous dates and times to build a future context history associated with a user of the computing device 110.

For example, information contained in the past context history of computing device 110 may indicate the user's location during a typical work week of the user as the user travels to and from the work location to the home location along the typical route. Based on the past context history, the context module 162 may generate a future context history including information reflecting the actual locations recorded in the past context history indicating the expected locations of the user during the future work week.

The context module 162 may supplement future context history associated with the user of the computing device 110 with information stored on the electronic calendar or mined from other communication information associated with the computing device 110. For example, when a user is typically at a home location, an electronic calendar may include locations associated with events or appointments that occur at a future time or date. Rather than including the home location as the expected location in the future context history during the future time and date of the event, the context module 162 may include the event location as the expected location during the future time or date of the event.

Context module 162 may share past location history with prediction module 164, and prediction module 164 may use past location history to better predict, infer, or confirm current actions taken by a user at a particular time. The context module 162 may respond to requests from the prediction module 164 of the ISS 160 for the current context associated with the computing device 110 and/or the future context associated with the computing device 110 by outputting data to the prediction module 164 specifying the current or future context of the computing device 110.

The prediction module 164 learns and predicts current and future actions of a user of a computing device, such as the computing device 110, for different contexts. The prediction module 164 may predict what the user is doing for the current context based on the current context received from the context module 162 and may predict what the user will do for the future context based on the future context received from the context module 162.

As part of the predictive service accessed by the computing device 110, the predictive module 164 may automatically output notification data or other information to the notification module 122 of the computing device 110 to alert the user that corrections that the user may take to the current action are a way to ensure that future actions are taken. In other words, the prediction module 164 may determine whether the current action performed by the user in the current context is likely or unlikely to result in the user being able to perform future actions in future contexts.

The prediction module 164 is described in more detail with reference to additional schematic diagrams. In sum, prediction module 164 may perform one or more machine learning techniques to learn and model actions that are typically taken by users of computing device 100 and other computing devices for different contexts. By learning and modeling actions for different contexts, prediction module 164 may generate one or more rules for predicting actions taken by a user of computing device 110 for a particular context.

For example, using rules generated prediction module 164 regarding past observations regarding actions taken by a user of computing device 110 and other computing device for different contexts, prediction module 164 may guess or infer what the user of computing device 110 is doing for a particular context. As one example, prediction module 164 may infer that a user is "driving to work" or "commuting home" when computing device 110 is moving along a particular travel route on a particular date. As another example, prediction module 164 may determine that the user is watching a particular sports broadcast when computing device 110 is located in the user's living room in autumn and winter in sunday afternoon. As yet another example, prediction module 164 may predict that the user is sleeping when computing device 110 is stationary for a longer period of time late at night or early in the morning. And as yet another example, prediction module 164 may predict that a user is moving through a security check gate when computing device 100 is standing in an airport line.

In addition, by learning and modeling actions for different contexts, prediction module 164 may generate one or more additional rules for predicting whether a current action being taken by a user of computing device 110 for a current context will result in the user of computing device 110 successfully performing future actions in future contexts. The one or more additional rules may output a degree of likelihood (e.g., score, probability, value, etc.) indicating whether the user is still able to successfully perform the future action by performing the current action. The prediction module 164 may compare the likelihood output from the one or more additional rules to one or more thresholds and if the likelihood meets a threshold indicating that the user will not be able to perform future actions, the prediction module 164 may output notification data to alert the user to their potential failure or future error.

In accordance with the techniques of this disclosure, ISS 160 may detect and attempt to correct user errors that may have a significant impact on the user's life. For example, ISS 160 may infer an intended future destination for the user of computing device 110, then, upon observing a change in the location and movement trajectory of the user of computing device 110 that indicates that the user is not actually able to reach the intended future destination, intervene and alert the user to their potential error.

Consider an example scenario in which a user of computing device 110 plans to fly from london to zurich at the early morning time of a particular date. The electronic calendar or confirmation email received by the computing device 110 may indicate that the flight is scheduled to take off at 7:10 a.m. In the worst case, the user may inadvertently turn off the alarm on the departure day early morning, and as a result sleep too late to arrive at the airport take-up flight on time. Rather than allowing the user to sleep lazy and potentially miss a flight, the ISS 160 may predict that the user has fallen asleep on the departure day and automatically send notification data to the computing device 110 to cause the computing device 110 to alert the user and wake the user up the flight in time in accordance with the techniques of this disclosure.

In operation, ISS 160 may predict future actions to be taken by a user of computing device 110, for example, in order to alert the user of computing device 110 to wake up a flight on the calendar of computing device 110 in time. For example, the prediction module 164 may query the context module 162 for future contexts of the computing device 110 periodically or occasionally. In response to the query from the prediction module 164, the context module 162 may determine a future context of the computing device 110 based on the context information received from the computing device 110 (e.g., calendar information, communication information, or other context information as described above) and send the future context to the prediction module 164.

Based on the future context received from the context module 162, the prediction module 164 may infer future actions that the user may take given the future context by using machine learning or other rule-based systems. For example, in response to receiving a future context indicating that the user of computing device 110 is to be located in an airport-ride 7:10 am departure before 6:00 am, the machine learning of prediction module 164 or other rule-based system may predict that the user of computing device 110 is to be in am 5:30 to the airport to drive 30 minutes to the airport.

ISS 160 may determine a current action taken by the user based on context information associated with computing device 110 and determine a degree of likelihood of whether the user is able to take future actions based on the current action. For example, considering that the prediction module 164 may expect the user of the computing device 110 to begin traveling to the airport before 5:30 a.m., the prediction module 164 may query the context module 162 about the current context of the computing device 110 in 5:00 a.m. to determine whether the user is on track to begin performing the intended action of traveling to the airport in time to take a flight. The current context determined by the context module 162 may indicate a lack of movement associated with the computing device at the user's home location and may simulate to the prediction module 164 a movement pattern (or lack of movement pattern) in which the user may be sleeping.

ISS 160 may determine a degree of likelihood (e.g., probability, score, binary value, etc.) of whether the user is able to take the future action based on the current action. To facilitate the above example, prediction module 164 may input a current action (e.g., sleep), a future action (e.g., go out of the airport in the morning 5:30), a current context, and a future context into one or more rules for predicting whether the current action being taken by the user of computing device 110 for the current context will result in the user of computing device 110 successfully performing the future action in the future context.

ISS 160 may predict, based on the degree of likelihood, that the user will not be able to take future actions and, in response, send information to computing device 110 indicating that the current action taken by the user will result in the user not being able to take future actions. For example, prediction module 164 may compare the degree of likelihood output from one or more additional rules to one or more thresholds. If the degree of likelihood meets a threshold (e.g., if the user is currently sleeping, if one or more rules give the user a chance to go out of the airport on time greater than 50%), the prediction module 164 may refrain from outputting notification data to the computing device 110. Conversely, if the degree of likelihood does not meet the threshold (e.g., if the user is currently sleeping, if one or more rules give the user less than 50% of the chance to go out of the airport on schedule), the prediction module 164 may output notification data to alert the user to their potential failure or future error. The prediction module 164 may learn and tune thresholds used by machine learning and rule-based system timeouts to improve notifying the user of the likelihood that they may make a mistake.

In some examples, prediction module 164 may determine the degree of likelihood of whether the user is able to take the future action by decreasing the degree of likelihood in response to the current time being within a threshold time of the future time and not decreasing the degree of likelihood or increasing the degree of likelihood in response to determining that the current time is not within the threshold time of the future time. For example, prediction module 164 may determine that the user is sleeping within a threshold amount of time that the user may leave the current location to arrive at half an hour of the latest departure time of the airport before the latest arrival time. In response to determining that the user is sleeping within the half-hour threshold, prediction module 164 may reduce the likelihood that the user of computing device 110 is able to perform future actions and leave on time. In response to determining that the user is sleeping outside of the half-hour threshold, prediction module 164 may avoid reducing or even increasing the likelihood that the user of computing device 110 is able to perform future actions and leave on time.

Notification data output by computing device 110 in response to determining the degree of likelihood that the user will not be able to perform future actions may cause computing device 110 to loudly ring or sound an alarm to wake up the user once received by computing device 110. In some examples, computing device 110 may turn on a light, television, or other form of notification to wake up the user. In some examples, notification data may cause notification module 122 to output a graphical user interface at UID112 through UI module 120 to notify the user to wake up to leave on time. In other words, ISS 160 may send notification data to computing device 110 such that when a user of computing device 110 has an early morning flight and the user fails to begin moving toward an airport, computing device 110 actively alerts the user (e.g., by generating an audible alert) as follows: the user is signaled until the computing device 110 detects that the user has interacted with the computing device 110 (e.g., as an indication that the user heard, watched, or felt an alarm).

Thus, when a user of a computing device receives information from an example computing system that performs the techniques described herein, the user is unlikely to continue to perform actions that may result in the inability to perform future actions that the system expects the user to take in the future context. Instead, the example system may automatically infer that the user is likely to make an error and notify the user of their potential failure.

In this manner, unlike other computing devices and systems that merely provide reminders and access to information, but may still allow a user to enter directions or take actions that may lead to mistakes in their daily lives, example systems take additional steps to ensure that errors are avoided and in fact implement future actions associated with reminders and information. Even if the user does not know that he is making an error, the example system will automatically provide information to guide the user through avoidance to avoid making an error.

Thus, the user need not even know that he is making an error, and the computing system can infer whether the user needs to take corrective action for the current action without user input. The user may be subjected to less stress and take less time to go to future actions along the wrong path and less time to correct the procedure to avoid making a mistake. By correcting potential errors earlier rather than later, the example system may enable the computing device to receive less input from a user searching for information to attempt to correct or correct the errors. With less input to correct for potential failures, example systems may enable a computing device to save energy and use less power than other systems that merely provide users with the ability to receive reminders or search information.

Fig. 2 is a block diagram illustrating an Information Server System (ISS) 260 as an example computing system configured to predict whether a user of computing device 110 of fig. 1 is able to take future actions in accordance with one or more aspects of the present disclosure. ISS 260 is a more detailed example of ISS 160 of fig. 1, and ISS 260 is described below within the context of system 100 of fig. 1. Fig. 2 illustrates only one particular example of ISS 260, and many other examples of ISS 260 may be used in other instances, and may include a subset of the components included in example ISS 260, or may include additional components not shown in fig. 2.

ISS 260 provides computing device 110 with a conduit through which a computing device, such as computing device 110, may access a service for receiving information that ISS 260 automatically predicts that will assist a user of the computing device in correcting current actions in order to ensure that the user takes future actions and avoid making errors. As shown in the example of fig. 2, ISS 260 includes one or more processors 270, one or more communication units 272, and one or more storage devices 274. The storage 274 of the ISS 260 includes a context module 262 and a prediction module 264. Within prediction module 264, storage 74 includes training module 268. Modules 262 and 264 each include at least the same (if not more) capabilities as modules 162 and 164 of fig. 1.

The storage 274 of the ISS 260 further includes a user information data store 270A, a context history data store 270B, and an action rule data store 270C (collectively, "data stores 270"). Communication channel 276 may interconnect each of components 270, 272, and 274 for inter-component communication (physically, communicatively, and/or operatively). In some examples, communication channel 276 may include a system bus, a network connection, an interprocess communication data structure, or any other method for communicating data.

The one or more communication units 272 of the ISS 260 may communicate with external computing devices, such as the computing device 110 of fig. 1, by sending and/or receiving network signals over one or more networks. For example, ISS 260 may use communication unit 272 to send and/or receive radio signals in network 130 to exchange information with computing device 110. Examples of communication unit 272 include a network interface card (e.g., such as an ethernet card), an optical transceiver, a radio frequency transceiver, a GPS receiver, or any other type of device that may send and/or receive information. Other examples of communication unit 272 may include short wave radios, cellular data radios, wireless ethernet radios, and Universal Serial Bus (USB) controllers.

Storage 274 may store information for processing during operation of ISS 260 (e.g., ISS may store data accessed during execution at ISS 260 by modules 262, 264, and 268). In some examples, storage 274 is temporary memory, meaning that the primary purpose of storage 274 is not long-term storage. The storage 274 on the ISS 260 may be configured to store information for a short period of time as volatile memory and, thus, does not retain stored content if the storage is powered down. Examples of volatile memory include Random Access Memory (RAM), dynamic Random Access Memory (DRAM), static Random Access Memory (SRAM), and other forms of volatile memory known in the art.

In some examples, storage 274 further comprises one or more computer-readable storage media. Storage 274 may be configured to store larger amounts of information than volatile memory. The storage 274 may be further configured to store information for long periods of time as non-volatile memory space and retain information after power up/down cycles. Examples of non-volatile memory include magnetic hard disks, optical disks, floppy disks, flash memory, or the form of electrically programmable memory (EPROM) or Electrically Erasable Programmable (EEPROM) memory. Storage 274 may store program instructions and/or data associated with modules 262, 264, and 268.

The one or more processors 270 may implement functions and/or execute instructions within the ISS 260. For example, processor 270 on ISS 260 may receive and execute instructions stored by storage 274 that perform the functions of modules 262, 264, and 268. These instructions, executed by processor 270, may cause ISS 260 to store information within storage 274 during program execution. Processor 270 may execute instructions of modules 262, 264, and 268 to predict whether a user of a computing device is able to take future actions for various contexts given previous actions by other users of other computing devices for the same context, and automatically provide information to other computing devices based on the predictions. That is, modules 262, 264, and 268 may be operated by processor 270 to perform various actions or functions of ISS 270 described herein.

Information stored at the data store 270 may be searched and/or information stored at the data store 270 may be categorized. For example, one or more of modules 262, 264, and 268 may provide for input requesting information from one or more of data store 270 and, in response to the input, receive information stored at data store 270. The information server system 260 may provide access to information stored as a cloud-based data access service at the data store 270 to devices connected to the network 130, such as the computing device 110. When the data store 270 contains information associated with individual users or when information is generalized among multiple users, all personally identifiable information, such as names, addresses, phone numbers, and/or email addresses, linking the information back to the person may be removed prior to storage at the information server system 260. The information service system 260 may further encrypt information stored at the data store 270 to prevent access to any information stored therein. Additionally, if such users positively agree to such information collection, the information server system 260 may store only information associated with the users of the computing devices. The information server system 260 may further provide the user with an opportunity to revoke consent, in which case the information server system 260 may cease to collect or retain information associated with that particular user.

Data store 270A represents any suitable storage medium for storing searchable information associated with a user of a computing device (computing device 100 of FIG. 1). User information data stores 270A and 270C may be maintained by prediction module 264, which may be part of context history data store 270B maintained generally by context module 262 or separate from context history data store 270B maintained generally by context module 262. User information data store 270A may include one or more searchable databases or data structures that organize different types of information associated with individual users of computing devices, such as computing device 110 of FIG. 1. In some examples, user information data store 270A includes information related to user search history, email messages, text-based messages, voice messages, social network information, photographs, application data, application usage information, purchase history, and any and all other information associated with a user and user interaction with a computing device such as computing device 110. The user information stored at data store 270A may be searchable. For example, the context module 262 may provide particular date and/or time of day information as input into the data store 270A and receive as output a plurality of pieces of user information related to the input. For example, the context module 262 may provide a particular date as input and receive information regarding flights indicated in the reservation confirmation email of the user of the computing device 110. Or as another example, the context module 262 may receive an indication of a keyword query (e.g., a string) and output an appointment, purchase associated with the keyword.

Data store 270B represents any suitable storage medium for storing a searchable context history including context information organized by date and time (e.g., location, time of day, weather information, traffic information, navigation information, device status information, user information, etc.). The data store 270B may include past context history and/or future context history. The information server system 260, and in particular the context module 262, may collect context information associated with a computing device, such as the computing device 110, and store the collected context information at the data store 270B. The context module 262 may rely on information stored at the context history data store 270B to determine the context of a user or a computing device such as the computing device 110.

Data store 270C may store rules (e.g., of a machine learning system) for predicting actions that a user of a computing device may take for various contexts. The data store 270C may receive the context as input and provide as output predicted actions taken by the user in the context. The data store 270C may further store other rules for determining a degree of likelihood that indicates whether a user of a computing device is able to perform future "expected" actions for future contexts by performing current actions for the current context. The data store 270C may receive the current action and the future action as inputs and output a likelihood indicating whether the user is able to perform the future action if the current action continues to be performed.

Task and demand rules data store 270C includes one or more previously developed rules upon which demand prediction module 264 relies to predict tasks or actions that a user of a computing device may perform for a current context, as well as information that the user may need to complete the task. For example, the data store 270C may store rules for machine learning or artificial intelligence systems of the demand prediction module 264. The machine learning or artificial intelligence system of the demand prediction module 264 can access the rules of the data store 270C to infer tasks and demands associated with the user of the computing device 110 for the current context.

In some examples, prediction module 264 can provide a context of the computing device as input to data store 270C and receive information regarding tasks or actions that a user of the computing device can perform in the context as output. In some examples, the rules of data store 270C may output a degree of likelihood (e.g., count, probability, etc.) associated with the task or action for the context. And in some examples, the prediction module 264 may provide the current action or task and the predicted or expected action or task as inputs to the data store 270C and receive as output information regarding whether the current action or task is compatible with future actions or tasks.

Training module 268 may generate action rules stored at data store 270C. For example, the prediction module 264 may initially receive the context from the context module 162 and provide the context as input to the training module 268. Training module 268 may generate a model of the user's actions and train or refine the model over time to better learn and be able to predict the actions and generate rules for inferring the user's actions for a given context. Once generated by training module 268, prediction module 264 can query context module 262 for an indication of the current or future context of computing device 110. The prediction module 264 can provide information received from the context module 262 regarding the current or future context of the computing device 110 as input to the model and, in response, can receive as output an indication (e.g., data) indicative of one or more tasks, actions, or other physically or virtually observable actions that a user typically performs for a particular context.

The training module 268 of the prediction module 264 can generate rules stored at the data store 270C for predicting and determining the degree of likelihood associated with the task performed by the user for the particular context. For example, the machine learning or artificial intelligence system of training module 268 can analyze the context information acquired by context module 262 and stored at data store 270B and identify associations with user information stored at data store 270A. The types of machine learning systems used by training module 268 include Bayesian networks, neural networks, and other types of artificial intelligence models. For example, training module 268 may develop a table of computing device groups for each computing device 110 or sharing similar contexts and having users performing similar actions at similar contexts per row. Based on the association between the context information and the action, training module 268 may develop rules for predicting future actions or current actions performed by a user of the computing device for a particular context.

In operation, the ISS 260 may predict future actions to be taken by a user of the computing device, such as attending an event at a future time. For example, the prediction module 264 may query the context module 262 for updates regarding the predicted location or expected context of the computing device 100 the next day, next hour, next week, etc. Prediction module 262 may receive as output an indication that a user of a computing device will be attending an opera event, movie, meeting, or other appointment or event the next day (e.g., sunday).

ISS 260 may determine the current action taken by the user based on context information associated with the computing device. For example, by chance, prediction module 264 may query context module 262 for the current context of computing device 110 and determine whether the current action likely to be performed in the current context is compatible with or contradicts the user's ability to perform future actions to take part in operas, movies, meetings, or other appointments the next day by using one or more of the rules stored at action rules data store 270C. The prediction module may input the current context into the action rules data store 270C and receive as output an indication that the user of the computing device 110 is taking preparatory actions to prepare for the participation event at a future time by traveling to or having arrived at a local dry cleaner and deducing that the user is likely to place laundry to be dry cleaned in a dry cleaner wash for the next day participation event.

The ISS 260 may determine a degree of likelihood of whether the user is able to take future actions based on the current actions. For example, the prediction module 264 can output the current context, the previously determined future context, the current action, and the expected or future action into the action rules data store 270C and receive an indication of whether the user should be able to place their laundry for dry cleaning required for the next day event in a dry cleaner wash and still be able to attend the next day event to the extent of likelihood. Action rule data store 270C may decrease the likelihood in response to determining that the preparatory action cannot be completed within a threshold time of the future time and then output the likelihood. For example, because the dry cleaning shop is not open on the next day, the user will not be able to remove the laundry he is dry cleaning for the event, so action rules data store 270C may output a low likelihood (e.g., less than 50% probability).

ISS 260 may predict whether the user is able to take future actions based on the degree of likelihood and send information to computing device 110 indicating that the current action taken by the user will result in the user not being able to take future actions. For example, in view of the low likelihood, prediction module 264 can generate notification data that ISS 260 can send to computing device 110 to alert the user that the user is not open on the day of the event due to the dry cleaning store in which the user is located, which may be making an avoidable mistake by placing the laundry for the next day of participation in the event in the dry cleaning store for cleaning.

In some examples, ISS 260 may determine that a user may take corrective action to increase the degree of likelihood of whether the user is able to take future actions, and the information sent to computing device 110 indicating that the current action taken by the user will result in the user being unable to take future actions may include an indication of the corrective action. For example, prediction module 264 can perform a search (e.g., an internet search) for dry cleaning stores located near the current location of computing device 110 that are open on sunday (e.g., the day of the event) and suggest that the user travel to one of these other dry cleaning stores instead.

FIG. 3 is a conceptual diagram illustrating an example navigation scenario in which an example computing system, such as information server system 160 of FIG. 1 or information server system 260 of FIG. 2, predicts whether a user of a computing device, such as computing device 110 of FIG. 1, is able to take future actions in accordance with one or more aspects of the present disclosure. FIG. 3 is described below within the context of the system 100 of FIG. 1, where the computing device 310 is an example of the computing device 110 of FIG. 1.

In the example of fig. 3, context module 162 of ISS 160 may determine that computing device 310 is at current location 301 at the current time and that the future context of computing device 310 places computing device 310 at future location 303 at the future time (e.g., three hours after the current time). Fig. 3 illustrates paths 304A, 304B, 304C, and 304D that a user of computing device 310 may begin traveling from location 301 to 303.

Path 304A is approximately 4 distance units in length and it will take approximately two hours of travel time for the user of computing device 310 to travel from location 301 to 303. Path 304A intersects location 302A. Path 304B is approximately 5 distance units in length and it will take approximately two and a half hours of travel time for the user of computing device 310 to travel from location 301 to 303. Path 304B intersects location 302B. The length of path 304C is approximately 6 distance units and it will take approximately three hours of travel time for the user of computing device 310 to travel from location 301 to 303. Path 304C intersects location 302C. Path 304D also intersects location 302C. However, the length of path 304D is approximately 8 distance units and it will take approximately four hours of travel time for the user of computing device 310 to travel from location 301 to 303.

In one example, as computing device 310 moves along the trajectory of path 304A toward location 302A, context module 162 of ISS 160 may detect the movement associated with computing device 310 and send the updated current context associated with computing device 310 to prediction module 164. The prediction module 164 may determine a degree of likelihood of whether the user of the computing device 310 can reach the location 303 before the future time by traveling toward the location 302A at the current time. For example, prediction module 164 may determine an estimated time of arrival for the user of computing device 310 to reach future location 303 after traveling from location 301 to location 302B. The prediction module 164 may increase the degree of likelihood in response to determining that the estimated time of arrival is within a threshold time of the future event (e.g., within an acceptable tolerance of the arrival location 303 to arrive at the user on time). For example, prediction module 164 may determine that path 304A taken directly to location 302A, the user of computing device 310 will certainly still arrive at location 303 on time.

In another example, as computing device 310 moves along the trajectory of path 304B toward location 302B, context module 162 of ISS 160 may detect the movement associated with computing device 310 and send the updated current context associated with computing device 310 to prediction module 164. Even though prediction module 164 may determine that computing device 310 is moving on a trajectory away from location 303 at the current time, prediction module 164 may avoid alerting computing device 310 to potential errors until prediction module 164 determines a degree of certainty as to whether the user can reach location 303 before the time of arrival by traveling along path 304B toward location 302B at the current time. The prediction module 164 may increase the degree of likelihood in response to determining that the estimated time of arrival is within a threshold time (e.g., less than) of the future time. For example, prediction module 164 may determine that a user of computing device 310 will certainly arrive at location 303 on time even by taking path 304B directly to location 302B.

In a different example, as computing device 310 moves along the trajectory of path 304D toward location 302C, context module 162 of ISS 160 may detect the movement associated with computing device 310 and send the updated current context associated with computing device 310 to prediction module 164. The prediction module 164 may determine the degree of likelihood of whether the user of the computing device 310 is able to reach the location 303 before the future time by proceeding toward the location 302C at the current time. For example, prediction module 164 may determine an estimated time of arrival for the user of computing device 310 to reach future location 303 after traveling from location 301 to location 302C while continuing on the trajectory of path 304D. The prediction module 164 may reduce the degree of likelihood in response to determining that the estimated time of arrival is not within a threshold time (e.g., outside of the tolerance of acceptable time of arrival at location 303 to arrive at time by the user). For example, prediction module 164 may determine that path 304D taken directly to location 302C, a user of computing device 310 must not arrive at location 303 on time. In response to determining that the user of computing device 310 is likely not to arrive on time, prediction module 164 may send information to computing device 310 to alert the user to potential errors in continuing to travel along path 304D.

In addition to alerting the user to their potential error, prediction module 164 may also determine corrective actions that the user may take to increase the degree to which the user is able to take future actions, and include an indication of the corrective action in the information indicating that the current action taken by the user will result in the user being unable to take future actions. In other words, rather than merely having computing device 310 output an audible alert to alert the user to an error in continuing to travel along path 304D, prediction module 164 may send information to alert the user of computing device 310 that the user is still able to reach location 303 by changing direction and taking path 304C before the future time.

Fig. 4 is a flowchart illustrating example operations 410 through 420 performed by an example computing system configured to predict whether a user of a computing device is able to take future actions in accordance with one or more aspects of the present disclosure. Fig. 4 is described below in the context of the system 100 of fig. 1. For example, ISS 160 may perform operations 410 through 420 whether a user of computing device 110 is able to take future actions, and provide computing device 110 with information to alert the user as to whether the predicted user is able to take future actions, in accordance with one or more aspects of the present disclosure.

As shown in FIG. 4, in operation, ISS 160 may predict future actions to be taken by a user of computing device 410 (410). For example, the context module 162 of the ISS 160 may receive context information associated with the computing device 110 indicating that a user of the computing device 110 is generally viewing a scheduled program and generate future contexts defining future broadcasts of the scheduled program for viewing by the user of the computing device 110. The prediction module 162 may receive an indication of a future context and predict that the user will view the scheduled program at a future time.

ISS 160 may determine a current action to take by the user based on context information associated with computing device 110 (412). For example, the context module 162 may receive data via the network 130 indicating that a user of the computing device 110 is using the computing device 110 to navigate via a web browser or some other form of graphical user interface to rent pay-per-view (PPV) broadcasts of sporting events for the same future time schedule as the scheduled programs, and generate a current context associated with the computing device 110 defining the user viewing the graphical user interface. Prediction module 164 may receive an indication of the current context from context module 162 and predict that the user is purchasing or renting PPV broadcasts for a future time.

ISS 160 may determine a degree of likelihood of whether the user is able to take future actions based on the current actions (414). For example, the prediction module 164 may input into one or more rules of the machine learning system future actions of viewing the scheduled program and current actions of purchasing PPV broadcasts, and receive as output an indication of the likelihood of whether a user is more likely or less likely to be able to view the scheduled program that he is typically viewing by purchasing PPV programs. The degree of likelihood may be low (e.g., 10%) because the PPV program may run within the start time or begin before the end time.

ISS 160 may determine whether the degree of likelihood meets a threshold (416). The prediction module 162 may determine that the degree of likelihood is not high enough (e.g., greater than 90%) to indicate that the user is able to perform future actions if the user continues to perform the current action.

ISS 160 may predict that the user will not be able to take future actions based on the degree of likelihood (418). In response to determining that the likelihood is not high enough (e.g., greater than 90%), prediction module 162 may determine that the user will not be able to view the scheduled program.

ISS 160 may send information to computing device 110 indicating that the current action taken by the user will result in the user failing to take future actions (420). For example, before the user completes the purchase of the PPV program, computing device 110 may receive information from prediction module 162 indicating that the purchase of the PPV program may conflict with the ability to view the scheduled program and output an alert to UID 112 indicating this.

There are several other examples of ISS 160 that predict whether a current action of a user of computing device 110 is compatible with future actions of the user. For example, ISS 160 may determine that a user is interacting with computing device 110 to purchase a ticket for one event that will overlap with another event and block participation in another event for which the user has purchased a ticket, and cause computing device 110 to alert the user of a potential error. ISS 160 may be aware of more than just the event start time and stop time for determining potential conflicts. For example, the rules of the prediction module 164 may learn from previous observations of other users of other computing devices that, although the start time of an event is at a particular time, the reserved seat for the event is open fifteen minutes before the actual start time, and if the attendees cannot be at the event fifteen minutes in advance, the reserved seat is open to other event attendees. Thus, even if the formal start time and end time of two events do not overlap, ISS 160 may alert a user of computing device 110 that there is a potential conflict with a ticket ordering the two events if the end time of an earlier event overlaps with the "informal" start time of the event (e.g., a reserved seat time).

FIG. 5 is a conceptual diagram illustrating an example timeline 500 of an example computing system, such as ISS 160 of FIG. 1 and ISS 260 of FIG. 2, predicting whether a user of a computing device, such as computing device 110 of FIG. 1, is able to take future actions in accordance with one or more aspects of the present disclosure. Fig. 5 is described in the context of fig. 1 to 4.

The example timeline of fig. 5 follows the following scenario that occurs between times t _S and t _F. The user of computing device 110 may be flying from london to new york city. As in many metropolitan areas, london has more than one international airport. While the user typically flies out of airport B, in the example of fig. 5, the user receives an email confirmation at time t _s regarding the flight he subscribes to that was taken off of airport a.

The timeline 500 is divided into an early programming phase between times t _s and t ₁, a late programming phase between times t ₁ and t ₂, and an execution phase between times t ₂ and t _F. During the early and late planning phases, ISS 160 may view context information that does not necessarily include movement information for future actions to predict when a user may be doing something that may prohibit performance of the future actions.

For example, as shown in timeline 500, context module 162 may record the reservation of a flight to take off from airport a at time t _F in a future context history associated with the user of computing device 110. During the early planning phase of timeline 500, a user of computing device 110 may enter a search query for navigation directions to airport B, and may later begin moving toward airport B. Because the early planning phase and the execution phase are not sufficiently close in time, ISS 160 may determine that a user may make a low likelihood of mistakes that might prevent the user from taking an aircraft at airport a by searching airport B or driving to airport B. ISS 160 may avoid outputting an alert indicating any potential conflict between the user's current action and the intended action.

During the later planning phase of timeline 500, a user of computing device 110 may enter a search query for navigation directions to airport B, and may later begin moving toward airport B. Because the post-planning phase is closer in time to the execution phase, ISS 160 may treat some actions as future actions that are likely to interfere with takeoff from airport a. For example, ISS 160 may receive context information indicating that the user is searching for navigation directions for airport B, at which time ISS 160 may determine that by searching airport B the user may be more likely to make mistakes that may prevent the user from boarding an aircraft at airport a in response to a search query that conflicts with the intended route or destination for airport a. When the user's expected location in the near future is airport a, ISS 160 may output an alert indicating that there is a potential conflict with the current action of the user searching for directions to airport B.

ISS 160 may receive context information indicating that the user is traveling toward airport B, at which time, although upcoming destination airport B conflicts with the intended route or destination airport a, the user has sufficient time to travel to airport B and still travel to airport a, so ISS 160 may determine that by traveling to airport B the user may be less likely to make mistakes that might prevent the user from taking an aircraft at airport a. ISS 160 may avoid outputting an alert indicating that there is a potential conflict with the current action of the user traveling to airport B when the user's intended location is airport a.

During the execution phase, ISS160 may rely more on (or give more weight to) movement of computing device 110 than other types of contextual information to infer whether the user is likely to make an error that may prevent the user from taking the intended action or may not make an error. ISS160 may determine that computing device 110 is moving along a trajectory toward airport B based on context information associated with computing device 110. ISS160 may compare the current trajectory of computing device 110 with the location of airport a, airport B, and other potential locations where other users would typically miss airports a and B. In response to determining that the path of computing device 110 is closest to or likely to intersect airport B (e.g., airport B is the same type of future location associated with future actions), ISS160 may predict that the user of computing device 110 is driving to airport B. In response to determining that the user of computing device 110 is driving to airport B when the user is expected to drive to airport a to take a flight, ISS160 may send data to computing device 110 that triggers computing device 110 to sound an alert or output some other tactile, visual, or audible alert indicating that the user is potentially driving to the wrong airport.

While the described examples use different "airports" as example first and second locations that a user may erroneously search for or navigate towards, other example locations exist. For example, when the user is desired to reach a second location, the user may perform a current action of searching for or navigating to the first location, where the first and second locations are different locations of a particular type of transportation service (e.g., bus station, train station, airport, offshore station, etc.). In other examples, the first location and the second location may be different respective offices of a single commercial establishment (e.g., two different locations of the same coffee or fast food chain, two different locations of a similar branded movie theatre, department store, etc.). In other examples, the first and second locations may be different respective cities of two different states or countries. In other examples, the first and second locations may be different respective physical locations sharing the same street address of two different cities, states, or countries.

In this manner, unlike other computing devices and systems that merely provide reminders and access to information, but may still allow a user to enter directions or take actions that may lead to mistakes in their daily lives, example systems take additional steps to ensure that errors are avoided and that future actions associated with reminders and information are actually implemented. Even if the user does not know that he is making an error, the example system will automatically provide information to guide the user back to avoid making an error.

For example, an example system may detect when a user searches for something that would be incompatible with a person's daily life and trigger an alert at a computing device alerting the user of the potential incompatibility. Other examples include: detecting when a user travels to the wrong appointment location (i.e., goes to the wrong location), searching for or purchasing a medication that may be incompatible with the current medication the user is or will be taking, searching for a brewery when the system has inferred that the user should not drink, purchasing the wrong portion of the adapter that was previously shopping for errors, ordering travel plans with inconsistent dates or locations, etc.

Clause 1: a method, comprising: predicting, by the computing system, future actions to be taken by a user of the computing device; determining, by the computing system, a current action to take by the user based on context information associated with the computing device; determining, by the computing system, a degree of likelihood of whether the user is able to take the future action based on the current action; and predicting, by the computing system, that the user will not be able to take future actions based on the likelihood; and sending information from the computing system to the computing device indicating that the current action taken by the user will result in the user failing to take the future action.

Clause 2: the method according to clause 1, further comprising: a corrective action that a user may take to increase the degree of likelihood of whether the user is able to take the future action is determined by the computing system, wherein the information indicating that the current action taken by the user will result in the user being unable to take the future action includes an indication of the corrective action.

Clause 3: the method according to any one of clauses 1 to 2, wherein: future actions include reaching a first location before a future time; the current action includes traveling from a current location to a second location different from the first location; and determining the degree of likelihood of whether the user is able to take future actions includes: determining, by the computing system, an estimated time of arrival of the user to reach the future location after traveling from the current location to the second location based on the context information, and at least one of: increasing, by the computing system, a degree of likelihood in response to determining that the estimated time of arrival is within a threshold time of the future time; or in response to determining that the estimated time of arrival is not within a threshold time of the future time, reducing, by the computing system, the degree of likelihood.

Clause 4: the method according to any one of clauses 1 to 3, wherein: future actions include reaching a first location before a future time; the current action includes searching for information associated with a second location different from the first location; and determining the degree of likelihood of whether the user is able to take future actions includes: in response to determining that the second location is the same location type as the first location, the degree of likelihood is reduced by the computing system.

Clause 5: the method of clause 4, wherein the first location and the second location are different locations of a particular type of transportation service.

Clause 6: the method of any of clauses 4 to 5, wherein the first location and the second location are different respective offices of a single business.

Clause 7: the method according to any one of clauses 1 to 6, wherein: future actions include starting at a first location before a future time; the current action includes sleeping at a current time; and determining the degree of likelihood of whether the user is able to take future actions includes: the degree of likelihood is reduced by the computing system in response to determining that the current time is within a threshold time of the future time.

Clause 8: the method according to any one of clauses 1 to 7, wherein: future actions include: attending to the event at a future time; the current actions include: performing a preparatory action to prepare for attending the event at a future time; and determining the degree of likelihood of whether the user is able to take future actions includes: the degree of likelihood is reduced by the computing system in response to determining that the preparatory actions cannot be completed within a threshold time of the future time.

Clause 9: a computing system, comprising: at least one processor; and at least one module operable by the at least one processor to: predicting future actions to be taken by a user of the computing device; determining a current action taken by the user based on context information associated with the computing device; determining, based on the current action, a degree of likelihood of whether the user is able to take the future action; and predict, based on the likelihood, that the user will not be able to take future actions; and sending information to the computing device indicating that the current action taken by the user will result in the user failing to take the future action.

Clause 10: the computing system of clause 9, wherein the at least one module is further operable by the at least one processor to: a corrective action is determined that the user can take to increase the degree of likelihood of whether the user is able to take the future action, wherein the information indicating that the current action taken by the user will result in the user being unable to take the future action includes an indication of the corrective action.

Clause 11: the computing system of any of clauses 9 to 10, wherein: future actions include reaching a first location before a future time; the current action includes traveling from a current location to a second location different from the first location; and the at least one module is further operable by the at least one processor to: determining a degree of likelihood of whether the user is able to take future action by determining an estimated time of arrival of the user at a future location after traveling from the current location to the second location based at least on the context information, and one of: increasing a degree of likelihood in response to determining that the estimated time of arrival is within a threshold time of the future time; or reduce the degree of likelihood in response to determining that the estimated time of arrival is not within a threshold time of the future time.

Clause 12: the computing system of any of clauses 9 to 11, wherein: future actions include reaching a first location before a future time; the current action includes searching for information associated with a second location different from the first location; and the at least one module is further operable by the at least one processor to: the degree of likelihood of whether the user is able to take future action is determined at least by reducing the degree of likelihood in response to determining that the second location is the same location as the first location type.

Clause 13: the computing system of any of clauses 9 to 12, wherein: future actions include starting at a first location before a future time; the current action includes sleeping at a current time; and the at least one module is further operable by the at least one processor to: the degree of likelihood of whether the user is able to take the future action is determined at least by decreasing the degree of likelihood in response to determining that the current time is within a threshold time of the future time.

Clause 14: the computing system of any of clauses 9 to 13, wherein: future actions include attending an event at a future time; and the current action includes performing a preparatory action to attending the event at a future time; and the at least one module is further operable by the at least one processor to: the degree of likelihood of whether the user is able to take the future action is determined at least by reducing the degree of likelihood in response to determining that the preparatory action cannot be completed within a threshold time of the future time.

Clause 15: a computer-readable storage medium comprising instructions that, when executed, cause at least one processor of a computing system to: predicting future actions to be taken by a user of the computing device; determining a current action taken by the user based on context information associated with the computing device; determining, based on the current action, a degree of likelihood of whether the user is able to take the future action; and predict, based on the likelihood, that the user will not be able to take future actions; and sending information to the computing device indicating that the current action taken by the user will result in the user failing to take the future action.

Clause 16: the computer-readable storage medium of clause 15, wherein the instructions, when executed, cause at least one processor of the computing system to: a corrective action is determined that the user can take to increase the degree of likelihood of whether the user is able to take the future action, wherein the information indicating that the current action taken by the user will result in the user being unable to take the future action includes an indication of the corrective action.

Clause 17: the computer readable storage medium of any one of clauses 15 to 16, wherein: future actions include reaching a first location before a future time; the current action includes traveling from a current location to a second location different from the first location; and the instructions, when executed, cause the at least one processor of the computing system to: determining a degree of likelihood of whether the user is able to take future action by determining an estimated time of arrival of the user to reach the future location after traveling from the current location to the second location based at least on the context information, and one of: increasing a degree of likelihood in response to determining that the estimated time of arrival is within a threshold time of the future time; or reduce the degree of likelihood in response to determining that the estimated time of arrival is not within a threshold time of the future time.

Clause 18: the computer readable storage medium of any one of clauses 15 to 17, wherein: future actions include reaching a first location before a future time; the current action includes searching for information associated with a second location different from the first location; and the instructions, when executed, cause the at least one processor of the computing system to determine a degree of likelihood of whether the user is able to take the future action at least by reducing the degree of likelihood in response to determining that the second location is the same location as the first location type.

Clause 19: the computer readable storage medium of any one of clauses 15 to 18, wherein: future actions include starting at a first location before a future time; the current action includes sleeping at a current time; and the instructions, when executed, cause the at least one processor of the computing system to determine a degree of likelihood of whether the user is able to take the future action at least by decreasing the degree of likelihood in response to determining that the current time is within a threshold time of the future time.

Clause 20: the computer readable storage medium of any one of clauses 15 to 19, wherein: future actions include attending an event at a future time; and the current action includes performing a preparatory action to attending the event at a future time; and the instructions, when executed, cause the at least one processor of the computing system to determine a degree of likelihood of whether the user is able to take the future action at least by reducing the degree of likelihood in response to determining that the preparatory action cannot be completed within a threshold time of the future time.

Clause 21: the computing system of clause 9, comprising means for performing any of the methods of clauses 1-8.

Clause 22: the computer-readable storage medium of clause 15, comprising further instructions that, when executed, cause at least one processor of the computing system to perform any of the methods of clauses 1-8.

In one or more examples, the described functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium, and executed by a hardware-based processing unit. The computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium, such as a data storage medium; or a communication medium including any medium that facilitates transfer of a computer program from one location to another, e.g., according to a communication protocol. In this manner, a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium or (2) a communication medium such as a signal or carrier wave. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data structures for implementing the techniques described in this disclosure. The computer program product may include a computer-readable medium.

By way of example, and not limitation, such computer-readable storage media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other storage medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Further, any connection is properly termed a computer-readable medium. For example, if the instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but rather relate to tangible storage media that are not transitory. Disk and disc, as used herein, includes Compact Disc (CD), laser disc, optical disc, digital Versatile Disc (DVD), floppy disk and blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

The instructions may be executed by one or more processors, such as one or more Digital Signal Processors (DSPs), general purpose microprocessors, application Specific Integrated Circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Thus, the term "processor" as used herein may refer to any of the foregoing structure or any other structure suitable for use in implementing the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware modules and/or software modules. Also, the techniques may be adequately implemented in one or more circuits or logic elements.

The techniques of this disclosure may be implemented in various apparatuses or devices including a wireless handset, an Integrated Circuit (IC), or a set of ICs (e.g., a chipset). In this disclosure, various components, modules, or units are described as being used to enhance the functional aspects of an apparatus configured to perform the disclosed techniques, but are not necessarily implemented by different hardware units. Rather, as described above, the various elements may be combined in hardware elements or provided by a collection of interoperable hardware elements including one or more of the processors described above, in combination with suitable software and/or firmware.

Various embodiments have been described. These and other embodiments are within the scope of the following claims.

Claims

1. A method, comprising:

Predicting, by the computing system, a future action to be taken by a user of the computing device, wherein the future action includes reaching the first location before a future time;

Determining, by the computing system, a current action taken by the user based on context information associated with the computing device, wherein the current action includes searching, using the computing device, for navigational directions to a second location different from the first location;

Determining, by the computing system, that the first location and the second location are different locations of a transportation service;

Determining, by the computing system, a degree of likelihood that the user will be able to take the future action based on the current action, wherein determining the degree of likelihood that the user will be able to take the future action comprises determining the degree of likelihood based on both:

determining that the first location and the second location are different locations of the transportation service, and

Determining that a current time of the current action is within a threshold amount of time of the future time;

Predicting, by the computing system, that the user will not be able to take the future action based on the degree of likelihood; and

Information is sent from the computing system to the computing device that actively informs the user of the current action taken by the user will result in the user failing to take the future action and actively informs the user of one or more corrective actions,

Wherein transmitting the information occurs automatically in response to searching for navigation directions to the second location, an

Wherein sending the information causes the computing device to automatically present the information to the user in response to searching for navigation directions to the second location.

2. The method of claim 1, further comprising:

Determining, by the computing system, a corrective action that the user can take to increase the degree of likelihood that the user will be able to take the future action, wherein the information actively informing the user that the current action taken by the user will result in the user failing to take the future action includes an indication of the corrective action.

3. The method of claim 1, wherein determining the degree of likelihood is further based on a current location of the computing device.

4. The method of claim 1, further comprising:

before determining the degree of likelihood:

obtaining previous context information associated with the computing device from one or more databases;

Identifying previous actions associated with the previous context information from one or more of the databases, and

A machine learning model is trained based on the prior actions and associated prior context information.

5. A computing system, comprising:

at least one processor; and

At least one module operable by the at least one processor to:

Predicting a future action to be taken by a user of a computing device, wherein the future action comprises reaching a first location before a future time;

determining a current action taken by the user based on context information generated by the computing device, wherein the current action includes traveling to a second location different from the first location;

Determining that the first location and the second location are different locations of a transportation service;

Determining, based on the current action, a degree of likelihood that the user will be able to take the future action, wherein the at least one module is operable by the at least one processor in determining the degree of likelihood that the user will be able to take the future action to determine the degree of likelihood based on both:

Predicting that the user will not be able to take the future action based on the degree of likelihood; and

Sending information to the computing device that actively informs the user of the current action taken by the user will result in the user failing to take the future action and actively informing the user of one or more corrective actions,

Wherein sending the information occurs automatically in response to predicting that the user is unable to take the future action and while the user is traveling to the second location different from the first location, and

Wherein sending the information causes the computing device to present the information to the user.

6. The computing system of claim 5, wherein the at least one module is further operable by the at least one processor to determine a corrective action that the user can take to increase the degree of likelihood that the user will be able to take the future action, wherein the information actively informing the user that the current action taken by the user will result in the user failing to take the future action comprises an indication of the corrective action.

7. A non-transitory computer-readable storage medium comprising instructions that, when executed, cause at least one processor of a computing system to:

determining that the first location and the second location are different respective offices of a single business;

Determining a degree of likelihood that the user will be able to take the future action based on the current action, wherein in determining the degree of likelihood that the user will be able to take the future action the computing system is further operable to determine the degree of likelihood based on both:

determining that the first location and the second location are different respective offices of a single business, and

Information is sent to the computing device indicating that the current action taken by the user will result in the user failing to take the future action and actively informing the user of one or more corrective actions,

Wherein sending the information occurs automatically in response to predicting that the user will not be able to take the future action and while the user is traveling to the second location different from the first location, and

8. The non-transitory computer-readable storage medium of claim 7, wherein the instructions, when executed, cause the at least one processor of the computing system to determine a corrective action that the user can take to increase the degree of likelihood that the user will be able to take the future action, wherein the information actively informing the user that the current action taken by the user will result in the user failing to take the future action comprises an indication of the corrective action.