JP5243679B2 - Notification platform architecture - Google Patents

Abstract

The present invention relates to a system (10) and methodology (170, 173, 180, 400, 600, 1000) to enable a variety of information associated with one or more notification sources (26-28) to be directed to one or more notification sinks (36-38) via a notification platform architecture (10). The architecture (10) includes a context analyzer (22) for determining a user's state such as location and attentional focus, wherein the user's state is employed by a notification manager (24) to make decisions regarding what, when and how information generated by the notification sources (26-28) should be forwarded to the notification sinks (36-38), for example. These decisons can include a cost benefit analysis wherein considerations are given as to whether the benefits of notifying the user are outweighed by the costs of disrupting the user. Decision-theoretic policies (177, 180) and/or somewhat less formal heuristic policies (177) can be employed to enable the decision-making process within the notification manager (24).

Description

(Technical field)
The present invention relates generally to computer systems, and more particularly to systems and methods that provide an architecture that facilitates receipt and notification of alerts generated by various devices and applications for communication to a user. .

(Background technology)
Many computer users currently receive information from a number of different sources and use a number of different devices or modalities to access this information. For example, a user may send an email or instant message via a computer, a call via a pager, a voice mail via a phone, such as a mobile ("cell") phone or landline phone, via a computer. Receive news information. As the amount of information available is increasing and there are many modalities that communicate such information, it depends on where the user is present, what mood or state the user is in, and the communication modality that the user can access , It becomes difficult for the user to receive and process the information.

  As an example, a user may be away from his computer but receive an important email. However, in many cases, the user only has access to a cell phone or pager. Thus, messages sent via one modality (eg, email) are not automatically transferred or communicated to another modality. As a result, significant time may elapse before the user actually receives the message. In some cases, a message may become unusable before it is actually received because the message itself requires a response or action by the user within a given time frame. As another example, a user who is using a computer but has turned off the phone ringer and voice mail indicator to avoid distractions while concentrating on the computer There is. However, if an important voice mail is left during this time, the user generally has no way of knowing whether an important message has been received unless the voice mail is regularly checked.

  Many messages / alerts received may not be important to the user as opposed to potentially not responding to important messages or alerts. For example, emails from user managers or collaborators generally must be received with higher priority than receiving or reviewing the latest sports score. Therefore, the value of the information contained in the message or alert must be balanced with the cost associated with the interruption to the user. However, cost and value can be context sensitive. This can include where the user is present, the activity the user is currently involved in, and the communication modalities that the user can access. In addition to the communication and associated modality management described above, the user also receives various other messages and / or alerts for later processing. This can include, for example, alerts from an increasing number of services, error messages, and computerized assistance presentations.

(Summary of Invention)
The following presents a simplified summary of the invention in order to provide a basic understanding of some aspects of the invention. This summary is not an extensive overview of the invention. It is not intended to identify key or critical elements of the invention or to delineate the scope of the invention. Its sole purpose is to present some concepts of the invention in a simplified form as a prelude to the more detailed description that is presented later.

  The present invention relates to a system and method for providing a notification platform architecture. According to one aspect of the invention, the architecture includes a context analyzer or context component, one or more notification sources and notification sinks, and a notification manager. The context analyzer stores user profile information regarding user notification parameters, such as the user's default notification preferences, and provides user context identification and update services. The notification source generates notifications directed to the user, and the notification sink provides notifications to the user. The notification manager directs the notification generated by the source to the sink based on the information stored and determined by the context analyzer and the provided or estimated information regarding the urgency of the notification. For example, the notification manager can access or infer the user's context (e.g., the user's current location and attention focus). This can be achieved based on registering multiple sources of context information. Such sources of information can include, for example, the user's context profile, the user's online calendar, time of day, world events, organizations, systems, and / or user activity. The notification can then be determined via an informational context and urgency analysis. This can include determining which of the notifications must be communicated to the user via which sink, in the form or modality provided by that sink.

  According to another aspect of the invention, a user can receive e-mail alerts, for example, and can automatically direct e-mails to a mobile phone if desired. Similarly, voice mail can be directed to a desktop computer as determined appropriately by the notification manager. Thus, notifications from the notification source are processed by the notification manager, which determines whether the user should be notified. If the manager determines that the user should be notified, the manager also determines how to notify the user. This can be based on information stored in the user profile, including information such as user preferences and current context to notify the desired notification sink. The sink can include, for example, a desktop computer, mobile phone, pager, and / or other device / application.

  Furthermore, the architecture of the notification platform can be generalized to virtually all notifications, including those related to the potential provision of services by software components, for example in desktop or mobile settings. Such notification may include the following:

Alerts on services such as services that automatically provide assistance or hints to users of software applications and / or services that automatically perform scheduling by inspecting emails with the focus of user attention Alerts that inform users about upcoming appointments or appointments Alerts that relay important changes in the location, proximity, or attention status of friends and coworkers Based on the text that is being created or that the user is reviewing Alerts that issue background queries and present the results of such background searches to the user

  As described above, the context analyzer determines the user's current context, such as the user's current location and attention status. The determined context can be used to determine, for example, when and how to communicate notifications directed to the user. According to other aspects of the invention, the context is specified directly by the user, directly measured using one or more sensors, a user-modifiable profile indicating the context, and potentially user-modifiable indicating the context Through one or more of one or more rules and / or estimation analysis using a model such as a Bayesian model or a statistical model. Thus, the user's context, including the user's location and attention state (or focus), can be used to communicate notifications to the user.

  According to another aspect of the present invention, decision-theoretic analysis is used by the notification manager to determine which of the notifications received from the notification source via any of the one or more modes associated with the notification sink. , You can decide to tell the user. The expected value of the information contained in the notification is subtracted from the expected cost of interruption to convey the notification through the mode of the sink, and the expected value of the user independently learning the information contained in the notification without notification, A value minus the actual cost of communicating the notification through the mode and sink is determined for the notification sink and associated mode. If this value is greater than a predetermined transmission threshold, a notification is communicated, for example via the mode of the sink with the highest value. According to another aspect of the invention, a heuristic communication policy must be used by the notification manager to communicate which notifications received from the notification source to the user via which mode of the associated communication sink. Judge whether it should be.

  The following description and the annexed drawings set forth in detail certain illustrative aspects of the invention. However, these embodiments are illustrative of only a few of the various forms in which the principles of the present invention can be used, and the present invention is intended to include all such embodiments and their equivalents. Other advantages and novel features of the invention will become apparent from the following detailed description of the invention when considered in conjunction with the accompanying drawings.

(Mode for carrying out the invention)
The present invention relates to systems and methods that enable various information related to one or more notification sources to be directed to one or more notification sinks (eg, modalities for receiving information) via a notification platform architecture. . This architecture includes a context analyzer for determining the user's status, such as position and focus of attention, and the user's status is transferred to the notification sink, for example, what information generated by the notification source. Used by the notification manager to make decisions about what must be done. These decisions can include a cost-benefit analysis that gives consideration as to whether the cost of interrupting the user is greater than the benefit of notifying the user. A decision-theoretic policy and / or a somewhat less formal heuristic policy may be used to enable decision processing within the notification manager.

  Referring first to FIG. 1, system 10 illustrates a notification architecture according to an aspect of the present invention. The system 10 includes a context analyzer 22, a notification manager 24 (also referred to as an event broker), one or more notification sources (eg, modalities that provide information) 1 to N, 26, 27, 28, and one or more. Notification sinks 1 to M, 36, 37, and 38 are included, and N and M are integers. The source is also referred to as an event publisher, and the sink is also referred to as an event subscriber. Any number of sinks and sources can be provided. In general, the notification manager 24 sends notifications, also referred to as events or alerts, from the sources 26-28 to the sinks 36-38, in part in the parametric information stored in and / or accessed by the context analyzer 22. Communicate notifications based on.

  The context analyzer 22 stores / analyzes information about user variables and parameters that affect notification decision making. For example, parameters include additional information that is conditioned by such parameters, such as the user's normal location and attention focus or context information such as time of day and activity by day of the week, and devices that the user tends to have access to in different places Can be included. Such parameters can also be a function of observations made autonomously via one or more sensors. For example, one or more profiles (not shown) that can be provided by the Global Positioning System (GPS) subsystem, information about the user's location, the type of device being used and / or the use of the device Selection or modification can be based on information about the pattern and the time that a particular type of device was last accessed by the user. Further, as described in detail below, automated estimation can also be used to dynamically estimate parameters or conditions such as location and attention. The profile parameters can be stored as a user profile that can be edited by the user. In addition to relying on a predefined set of profiles or dynamic estimation, the notification architecture allows the user to exclude important notifications from the user, for example, during the next “x” time, or until a given time. Be able to specify your status in real time, such as being unable to respond.

  The parameters can also include default notification preference parameters for user preferences regarding being disturbed by different types of notifications at different settings, which can be used as a basis for notification decisions by notification manager 24. Based on this, the user can initiate a change. The parameters can include default parameters regarding how the user wants to be notified in different situations (eg, by mobile phone, by pager, etc.). The parameters can include an assessment of the cost of interruption associated with being alerted by different modes at different settings. This includes the likelihood that the user is at a different location, the likelihood that different devices are available, and a context parameter that indicates the likelihood of the user's attention situation at a given time, as well as the user's given time Can include a notification parameter that indicates how it wants to be notified.

  The information stored by the context analyzer 22 includes context information determined by the analyzer in accordance with an aspect of the present invention. The context information is determined by the analyzer 22 by identifying the user's location and attention status based on one or more context information sources (not shown), as will be described in detail in later sections of this description. Determined. The context analyzer 22 may be able to accurately determine the user's actual location via, for example, a global positioning system (GPS) that is part of the user's car phone or cell phone. The analyzer considers the user's given attention by considering the background assessment and / or observations collected through consideration of information such as day of the week, time of day, date in the user's calendar, and observations about user activity. Statistical models can also be used to determine the likelihood of being a state. A given attention state may include whether the user is not obstructed to receive notification, busy, or obstructed to receive notification, such as weekdays, weekends, holidays, and / or other times Other considerations such as / period can be included.

  Sources 26-28 generate notifications directed to users and / or other entities. For example, sources 26-28 can include communications such as Internet and network-based communications, local desktop computer-based communications, and telephone communications, and software services such as intelligent help, background queries, and automated scheduling. . Notification sources generally generate events, also referred to herein as notifications or alerts, directed to a user or a user's proxy for information, services, and / or system events or world events. Is defined as Notification sources can also be called event sources.

  For example, an email can be generated as a notification by an email notification source and prioritized, so that the application program or system that generates the notification has an appropriate importance to the email for the user. Or assign a relative priority corresponding to urgency. Emails can also be sent regardless of their relative importance to the user. Desktop-centric notifications include potentially valuable services that the user wants to perform (eg, scheduling from messages), information that the user wants to review (eg, derived from background queries) Or an automated dialog with the ultimate goal of alerting the user about errors and / or other alerts generated by the desktop computer. Internet-related services can include notifications containing information that users subscribe to, such as occasional latest news headlines and stock quotes.

  Other notifications include background queries (eg, while the user is working, review the text that the user is currently working on, and a background query for that text is formulated and issued to the search engine. And scheduling tasks from scheduling programs and / or other programs. Notification sources 26-28 themselves can be push-type or pull-type sources. Push-type sources are sources that automatically generate and send information without a corresponding request, such as headline news and other Internet-related services that automatically send information after a subscription. A pull-type source is a source that sends information in response to a request, such as an email received after polling a mail server. Other notification sources include:

・ E-mail desktop application such as calendar system,
A computer system (for example, a message can be used to warn the user about information about warnings about system activity or system problems),
・ Internet-related services, reservation information, scheduling inquiries,
Change of documents in one or more shared folders or change in the number of documents of a certain type,
The availability of new documents in response to persistent or permanent inquiries about information, and / or a person and his presence, a change of location, proximity (for example, another colleague or friend within 10 miles of a trip Or if you are free "(for example, if you have time to speak and you are near a high-speed link that can support full video conferencing") Source of information.

  Notification sinks 36-38 can provide notifications to the user. For example, such notification sinks 36-38 include computers such as desktop and / or laptop computers, handheld computers, mobile phones, landline phones, pagers, car-based computers, and other systems / applications that can be identified. Can be included. Note that some of the sinks 36-38 can convey notifications more abundantly than other sinks. For example, desktop computers are usually connected with speakers and a relatively large color display, and have more bandwidth to receive information when coupled to a local network or the Internet. Thus, the notification can be communicated to the user in a relatively rich manner by the desktop computer. Conversely, many cell phones have smaller displays, which may be black and white, for example, and receive information with a relatively narrow bandwidth. Correspondingly, the information associated with notifications carried by the mobile phone is generally shorter and may be adapted to, for example, the interface function of the phone. Thus, the content of the notification may vary depending on whether it is sent to a mobile phone or a desktop computer. According to one aspect of the invention, a notification sink can refer to a notification sink that subscribes to events or notifications via an event subscription service, for example.

  The notification manager 24 accesses the information stored and / or determined by the context analyzer and determines which notifications received from the sources 26-28 are communicated to which of the sinks 36-38. Further, the notification manager 24 can determine how to communicate the notification depending on which of the sinks 36-38 has been chosen as the destination to send information. For example, it may be determined that the notification must be summarized before being provided to the selected sinks 36-38.

  The present invention is not limited to the manner in which manager 24 determines which notifications are communicated to which notification sinks and how notifications are communicated. According to one aspect, decision-theoretic analysis can be used. For example, the notification manager 24 may be adapted to estimate important uncertainties regarding variables including the user's location, attention, device availability, and time before the user accesses information in the absence of alerts. it can. The notification manager 24 can make notification decisions regarding whether to alert the user to notifications, and if so, the nature of the summary and the appropriate device or devices used to relay the notifications. In general, the notification manager 24 determines the net expected value of the notification. In doing so, the notification manager considers:

The fidelity and transmission reliability of each available notification sink,
The careful cost of distracting the user,
・ Newness of information for users,
The time it takes for the user to review the information
The value of the potential contextual response of the information, and / or the value of the information contained in the notification that increases and / or decreases over time.

  Thus, generating an estimate made about the uncertainty as the expected likelihood of value, such as the cost of interruption to the user due to the use of a particular mode on a particular device, for example, for some state of attention of the user Can do. Notification manager 24 may make decisions regarding one or more of the following.

What the user is currently paying attention to (eg based on context information),
Where the user is currently
How important the information is
-What is the cost of deferring notifications?
How notifications distract,
How likely it is to reach the user, and
• What is the loss of fidelity associated with using a particular mode for a given notification sink?

Thus, the notification manager 24 performs analysis, such as a decision-theoretic analysis of pending and active notifications, evaluates context-dependent variables provided by information sinks and information sources, and allows the user to review the information Selected uncertainties, such as time to high, user position and current attention state, can be estimated.

  As used herein, an estimate is generally an inference of the system 10, environment, and / or user or an estimate of their state from a set of observations captured via events and / or data. Refers to processing. Inference can be used, for example, to identify a specific context or action, or can generate a probability distribution over states. The estimation can be a probabilistic, i.e., calculation of a probability distribution for the current state based on data and event considerations. Inference can also refer to techniques used to synthesize higher-level events from events and / or sets of data. Such an estimate can be a set of observed events and / or stored event data, whether events correlate in close temporal proximity, and events and data are one or more event sources and data sources. Bring in a new event or action composition from whether or not you came from.

  Further, the notification manager 24 can access information stored in the user profile by the context analyzer 22 in place of or to support personalized decision-theoretic analysis. For example, a user profile may indicate that a user prefers to be notified via a pager only at a given time when the notification has a predetermined classification (eg, importance) level. . Such information can be used as a baseline for initiating a decision-theoretic analysis, or it can be in the form where the notification manager 24 determines whether and how to notify the user.

  In accordance with one aspect of the present invention, the notification platform architecture 10 can be configured as a layer that resides on top of an eventing or messaging infrastructure. However, the present invention is not limited to a particular eventing infrastructure. Such eventing and messaging systems and protocols may include:

HyperText Transport Protocol (HTTP) or an HTTP extension known in the art,
Simple Object Access Protocol (SOAP), known in the art
-Windows (R) Management Instrumentation (WMI), known in the art
Jini known in the art, and
-Virtually all types of communication protocols, for example based on packet switching protocols.

Furthermore, the architecture can be configured as a layer that resides on top of a flexible distributed computing infrastructure, as can be appreciated by those skilled in the art. Thus, the notification platform architecture can use the underlying infrastructure, for example, as a form where sources send notifications, alerts and events, and a sink receives notifications, alerts and events. . However, the present invention is not limited to this.

  With reference to FIG. 2, the context analyzer 22 of the notification architecture described in the previous section of this description will be described in detail. The context analyzer 22 shown in FIG. 2 includes a user notification preference store 52, a user context module 54 including a user context profile store 55, and a whiteboard 57. The context analyzer 22 according to one aspect of the present invention may be implemented as one or more computer programs that are executed by a computer processor from a computer-readable medium such as a memory.

  The preference store 52 stores notification parameters for the user, such as the user's default notification preferences, such as user profiles that can be edited and modified by the user. The preference store 52 can be regarded as a store in which information regarding parameters affecting the method of notifying the user is stored. The user context module 54 determines the user's current context based on, for example, one or more context information sources 60 published to the whiteboard 57. The user context profile store 55 stores user context parameters such as default context settings for the user that can be edited and modified by the user. That is, the user context module 54 accesses information from the profile store 55 and / or uses a previous set of beliefs in the profile store 55 using raw sensing via one or more context information sources 60. Updating (belief) provides the best guess or estimate for the user's current context information. The profile store 55 can be considered, for example, a store that stores a priori user locations and what the user is doing.

  The user context profile store 55 can be a pre-assessed and / or pre-defined user profile that captures information as a deterministic or probabilistic profile. Profiles can be costs and values for different classes of notifications as a function of normal location, activity, device availability, time of day, day type, user interaction with one or other devices, etc. it can. Day types can include, for example, weekdays, weekends, and holidays. The user context module 54 can actively determine or estimate aspects of the user's context or state, such as the user's current or future location and state of attention. Further, the actual state of the context can be accessed directly from the context information source 60 via the whiteboard 57 and / or various methods via estimation methods such as Bayesian estimation described in detail below. It can be estimated from such observations.

  The context information source 60 provides information to the context module 54 via the whiteboard 57 regarding the state and location of the user's attention, from which the module 54 can provide the user's current context (eg, the user's current context). Judgment can be made regarding the state and position of attention. Further, the present invention is not limited to any particular number or type of context source 60 nor to the type of information estimated or accessed by the user context module 54. However, the context source 60 may include, for example, mouse information, keyboard information, application information (eg, whether the application is currently receiving the user's focus), surrounding audio and utterance information, text in a window on the desktop. Multiple desktop information and events, such as information, can be included. The whiteboard 57 can include a common storage area in which the context information source 60 exposes information from which multiple components, including the source and context module 54, access this information. can do. Events, also called notifications or alerts, can generally include information regarding observations about one or more states of the world. Such conditions may include measurements regarding system component status, user activity, and / or environment. In addition, events can be generated by active polling of the measurement device and / or the source of the event, by receiving information sent upon change, and / or for every constant or variable event heartbeat.

  Other types of context sources 60 include user personal information management (PIM) information, which can generally provide scheduling information regarding, for example, the user's schedule. For example, the Global Positioning System (GPS) and / or the user's location as determined by the user's cell phone, PDA, or laptop that can determine the location and the current time are also types of context sources 60. Further, real-time mobile device usage is a type of context source 60. For example, a mobile device such as a cell phone is currently being accessed by the user, as well as the orientation and tilt of the device (eg, also provides information about device usage), acceleration and velocity (eg, the user is moving) Indicating whether or not there is).

  Referring to FIG. 3, the notification source described above is shown in detail. Notification sources 26-28 typically generate notifications that are communicated to notification manager 24, which notification manager 24 knows when notifications should be made and, if so, which notifications of notification sinks 36-38. Determine which and in which order they must be communicated.

  In accordance with one aspect of the present invention, notification sources 26-28 may have one or more of the following parameters within a standard description of attributes and relationships, referred to herein as notification source schemas or source schemas. it can. Note that schemas can be provided for the sources, sinks, and context information sources described above. Such a schema may provide declarative information about the different components and allow sources 26-28, notification manager 24, sinks 36-38, and context analyzer 22 to share semantic information with each other. Thus, different schemas provide information regarding the nature, urgency, and device signaling modalities regarding notifications. That is, a schema can generally be defined as a collection of classes and relationships between classes, such as information about an event or notification class, source, target, event or notification semantics, ontological context information, Notification and event structures are defined, including observation reliability and virtually all quality of service attributes.

  Notification source schema parameters (not shown) include one or more of message class, association, importance, time criticality, novelty, content attributes, fidelity tradeoff, and / or source information summary information. Can be included. The notification message class generated by the notification source indicates the type of notification communication, for example, email, instant message, numerical financial update, and desktop service. The notification association generated by the notification source indicates the likelihood that the information contained in the notification is related with respect to one or more specified contexts. For example, the association can be provided by a logical flag that indicates whether the source is associated with a given context. The novelty of the notification indicates the likelihood that the user already knows the information included in the notification. That is, novelty is whether the information is new to the user over time (indicating whether the user now knows the information and if the user knows the information in the future without being warned about it). , Indicate when it is).

  The fidelity trade-off associated with the notification indicates the loss of value of the information in the notification, which can result from, for example, specified allowed truncation and / or different forms of summarization. Such truncation and / or summarization communicates notifications to certain types of notification sinks 36-38 that have bandwidth and / or other limitations that prevent the sink from receiving the first generated complete notification. May be needed. Fidelity generally refers to the nature and / or degree of completeness of the original content associated with the notification. For example, long e-mail messages may be truncated or otherwise summarized to a maximum of 100 characters allowed by the mobile phone, resulting in loss of fidelity. Similarly, when an original message containing text and graphics content is sent through a device that has only text capability, it suffers a loss of fidelity. Furthermore, a device may only be able to show some of the full resolution available from the source. Fidelity trade-offs are how ordering (eg, the importance of rendering first in graphics and then in sound order) and / or the overall value of the content of notifications decreases with changes in fidelity. Refers to the set of source fidelity preferences, shown for any of the cost functions. For example, a fidelity trade-off can describe how all the values associated with sending a complete email message change as the amount of truncation increases. For example, context attributes can include a summary of the nature of the content that represents information such as whether the core message includes text, graphics, and audio components. The content itself is the actual graphics, text, and / or audio that make up the notification message content.

  The importance of notification refers to the value to the user of the information contained in the notification, assuming that the information is relevant to the current context. For example, importance can be expressed as the amount of information value to the user. Time criticality indicates a time-dependent change in the value of information included in the notification, that is, how the value of the information changes over time. In most, but not all cases, the value of notification information decreases over time. This is shown in FIG. Graph 80 shows the usefulness of notifications mapped over time. The importance of the notification is shown at point 84 in the graph representing the beginning, and curve 86 shows a decrease in usefulness over time.

  Returning to FIG. 3, default attributes and schema templates for different notification sources or source types can be made available in a notification source profile stored in a user notification preference store, such as store 52 of FIG. Such default templates can be instructed to override the values supplied by the notification source or to supply attributes when not in the schema supplied by the source. Source summary information allows the source to post a general summary of the status of the information and potential notifications available from the source. For example, source summary information from a messaging source can include at least information regarding the total number of unread messages that are at a certain priority, the status of attempts by a person trying to communicate with the user, and / or other summary information.

  Notification sinks 36-38 can be virtually any device or application by which a user or other entity can be notified about information contained in the notification. The choice regarding which sink to use to convey a particular notification is determined by the notification manager 24.

  Notification sinks 36-38 may have one or more of the following parameters supplied in the schema. This parameter includes, for example, fidelity / rendering capability, transmission reliability, actual cost of communication, and / or cautionary cost of interruption for device class, signaling (warning) mode, and related modes be able to. For a device that conforms to parameterized control of alert attributes, the alert schema and description of the parameters that control that attribute, and other attributes (eg, transmission reliability, cost of distribution, etc.) It is possible to include a function for changing the warning attribute using different settings. The notification sink schema provides a way for the notification device to communicate semantic information regarding its nature and functionality with the notification manager 24 and / or other components of the system. Default attributes and schema templates for different device types can be made available in device profiles stored in a user notification preference store, such as the store 52 of FIG. 2 described in the previous section. Such default templates can be instructed to override values supplied by the device or to supply attributes when not in the schema provided by such device.

  Each schema parameter will now be described in turn. A device class refers to a type of device, such as a mobile phone, desktop computer, and laptop computer. Classes can also be more general, such as mobile devices or stationery devices. The mode of signaling refers to the way that a given device can alert the user about notifications. A device can have one or more notification modes. For example, a mobile phone can provide only vibration, only a certain volume of ringing, and / or both vibration and ringing. In addition, the alert system desktop display can be broken down into multiple distinct modes (eg, a small notification window in the upper right corner of the display versus a small thumbnail at the top of the screen, with or without audio notification). In addition to being restricted to a predefined set of behaviors, a device can enable a mode with warning attributes that are a function of parameters as part of the device definition. Such continuous alert parameters for the mode represent controls such as volume to play alerts on the desktop, ringtone on mobile phone, size of alert window, etc.

  The transmission reliability of the mode of notification sinks 36-38 indicates the likelihood that the user will receive a communicated alert regarding notifications communicated to the user via the mode's sink. Transmission reliability can depend on device availability and user context, and the transmission reliability of different modes of a device can be conditioned by context attributes such as user location and attention. Defined as a cross product of attributes such as unique position and unique attention state, and abstraction of such attributes (for example, all times from 8 am to noon for all positions other than home) It is also possible to specify the transmission reliability for one or more unique context states defined as disjunction created as abstraction. For example, depending on where the user is currently, the information sent to the mobile phone is particularly in the area where the user has an intermittent reception area or when the user does not tend to have a mobile phone at this location ( For example, it may not always reach the user on a family holiday. The context can also affect transmission reliability due to environmental noise and / or other masking or distracting characteristics of the context.

  The actual cost of communication indicates the actual cost of communicating information to the user when included in the notification communicated to the sink. For example, this cost can include charges associated with cell phone transmissions. The cost of interruption includes the cost of attention associated with interruptions associated with alerts used by a particular mode of the device in a particular context. The cost of attention is usually sensitive to the user's particular attention focus. A fidelity / rendering function is a description of the device's text, graphics, and audio / tactile functions, also in certain modes. For example, the cell phone text limit may be 100 characters for a single message, and the phone may not have graphics capabilities.

  Turning to FIG. 5, the interface 90 shows user-selectable context designations that can be used by the context analyzer 22 in determining the user's current context. The determination of the user context by direct specification by the user and / or by a user modifiable profile will be described. The user's context can include the user's attention focus, ie whether the user is sensitive to receiving current notification alerts, as well as the user's current location. However, the present invention is not limited to this.

  Direct specification of the context by the user allows the user to indicate whether he / she has time to receive the alert and whether he / she wants to receive the alert. A default profile (not shown) can be used to indicate a default attention state and a default location where the user can receive alerts. The default profile can be modified by the user as desired.

  Referring to FIG. 5, the interface 90 illustrates how direct context specification can be implemented in accordance with an aspect of the present invention. For example, the window 91 has an attention focus section 92 and a position section 94. In the focus section 92, the user can check one or more check boxes 96, for example, whether the user is always available to receive alerts (always available), and the user is free to receive alerts. Indicate whether it is never available and whether the user has time to receive only alerts that have an importance level that exceeds a given threshold (available to receive alerts that has an important level glitter than) Can do. It should be appreciated that other availability sections can be provided. As can be seen from FIG. 5, the threshold can be measured in dollars, but this is for illustrative purposes only and the invention is not limited thereto. The user can increase the threshold value in box 98 by entering a new value directly or by increasing or decreasing the threshold value via arrow 100.

  In the location section 94, the user can check one or more check boxes 102 to specify where the user wants the alert to be communicated. For example, a user can have alerts communicated to a desktop, by e-mail, to a laptop, to a mobile phone, into a car, to a pager, or to a personal digital assistant (PDA) device, or the like. However, it should be understood that this is only an example and the present invention is not limited thereto.

  Window 91 can be pre-set with defaults for checkbox 96 and box 98 in section 92 and checkbox 102 in section 94 and can be considered a default user profile. The profile is user modifiable in that the user can override the default selection with his desired selection. Other types of profiles can also be used in accordance with the present invention.

  Referring to FIG. 6, the determination of user context by direct measurement, for example using one or more sensors, according to the present invention is shown. The user's context can include the user's attention focus as well as the user's current location. However, the present invention itself is not limited to this. Direct measurement of context indicates that the sensor can be used to detect whether the user is sensitive to receiving the current alert and to detect where the user is currently. According to one aspect of the invention, estimation analysis can be used in conjunction with direct measurements to determine user context as described in a later section of this description.

  With reference to FIG. 6, illustrated is a system 110 that can achieve direct measurement of user context. The system 110 includes a context analyzer 112 that communicates with, for example, a plurality of sensors 114-120, a mobile phone 114, a video camera 115, a microphone 116, a keyboard 117, a PDA 118, a vehicle 119, and a GPS 120. Combined. The sensors 114-120 shown in FIG. 6 are for illustrative purposes only and do not represent a limitation or restriction on the invention itself. The term sensor is used herein to be a general and overly inclusive term whereby the context analyzer 112 can determine what the user's current attention focus is and / or where the user's current location is. Means any device or means that can be determined.

  For example, if the user is turning on the mobile phone 114, this indicates that the user can receive alerts on the mobile phone 114. However, if the user is currently on a call on the mobile phone 114, this will cause the user to focus his attention on something else (ie, the current call) and the user will now be notified alerts. Indicates that you must not be disturbed. The video camera 115 is, for example, in the user's office and can detect if the user is in his office (ie, the user's location), indicating that other people are also in the office and are meeting with them. It is possible to detect whether the user should not be disturbed (ie the user's focus). Similarly, the microphone 116 is also in the user's office and the user is talking to someone and should not disturb the user, or the user is typing on the keyboard (e.g., via a sound coming from it). It is still possible to detect whether the user should not be disturbed now. The keyboard 117 can also be used to determine whether the user is currently typing, for example, if the user is typing very quickly, this allows the user to perform computer-related activities. It can indicate that it is in focus and should not be overly disturbed (and that the user is actually in his office).

  If the PDA device 118 is being accessed by a user, this can indicate that the user device 118 can receive alerts. That is, the place where the notification must be communicated is where the device 118 is. The device 118 can also be used to determine the user's current focus of attention. The vehicle 119 can be used to determine whether the user is currently in the vehicle, i.e., whether the vehicle is currently being manipulated by the user. Further, for example, the speed of the vehicle can be examined to determine what the user's focus is. For example, if the speed exceeds a predetermined speed, it can be determined that the user is focused on driving and should not be bothered by a notification alert. The GPS device 120 can also be used to ascertain the user's current location, as is known in the art.

  The following section of this detailed description describes the determination of user context with user-modifiable rules. The user's context can include the user's attention focus as well as the user's current location. However, the present invention is not limited to this. Determining context via a rule indicates that the user's location and / or attention focus can be determined according to a hierarchical set of if-then rules.

  Referring to FIG. 7, a diagram illustrates a hierarchical ordered set 130 of rules. In the rule set 130, for example, rules 132, 133, 134, 135, 136, 137, and 138 are shown. Note that other rules can be similarly configured. As can be seen from FIG. 7, rules 133 and 134 are subordinate to 132, rule 134 is subordinate to rule 133, and rule 138 is subordinate to rule 138. The rule is tested if rule 132 is first tested and found to be true, rule 133 is tested, if rule 133 is found to be true, rule 134 is tested, and so on. As ordered. If rule 133 is found to be false, rule 135 is tested. If rule 132 is found to be false, rule 136 is tested, and if it is found to be false, rule 137 is tested and if it is found to be true, Rule 138 is tested. The rules are preferably user-createable and / or amendable. Otherwise type rules can also be included in the rule set 130 (eg, if an if-then rule is found to be false, the otherrule becomes a control rule).

  Thus, the user can configure a set of rules such that the user's context is determined. For example, with respect to location, a set of rules may be tested to see if the current day is a weekday in the first rule. If so, a second rule subordinate to the first rule tests whether the current time is between 9 am and 5 pm. If so, the second rule indicates that the user is located in his office, otherwise the user is at home. If the first rule is found to be false (the current day is a weekend and not a weekday), the otherwise rule can indicate that the user is at home. It should be noted that this example is not intended to be a limiting or limiting example for the invention itself, and that one or more other rules may be similarly configured.

  The following sections of this description describe user context determination by estimation analysis, such as by using statistical models and / or Bayesian models. It should be noted that context determination via estimation analysis can in some ways rely on other determinations such as direct measurements via sensors already mentioned. Predictive analysis as used herein refers to using an estimation process on a plurality of input variables to yield output variables, ie the user's current context. The analysis can include, in one aspect, the use of statistical models and / or Bayes models.

  Referring to FIG. 8, a diagram of a system 140 that performs estimation analysis by an estimation engine 142 to determine a user's context 144 in accordance with an aspect of the present invention is shown. The engine 142, in one aspect, is a computer program that is executed by a computer processor from a computer-readable medium such as memory. The user context 144 can be regarded as an output variable of the engine 142.

  The engine 142 can process one or more input variables to make context decisions. Such input variables include, for example, one or more sensors 148, such as the sensors described in the previous section of this description for the direct determination method of context determination, as well as the user's scheduling computer program or personal information management (PIM). ) The current time and date represented by clock 150 and calendar 152 that can be accessed by the computer program and / or the user's PDA device. Other input variables can be considered in addition to those shown in FIG. The variables of FIG. 8 are not intended to be a limitation or restriction on the invention itself.

  Referring to FIGS. 9 and 10, there is shown an exemplary estimation model, such as that provided by a statistical model and / or a Bayesian model, that can be executed by the estimation engine described above according to the present invention. In general, a computer system can be somewhat uncertain regarding the details of a user's condition. Thus, a probability model can be constructed that allows estimations regarding the user's attention or other conditions to be made with uncertainty. The Bayesian model can estimate the probability distribution of the user's attention focus. Such attention states can be formulated as a more abstract representation of a set of prototype situations or a separate set of classes of cognitive issues addressed by the user. Instead, a model can be formulated that makes an estimate for continuous measurement of attention focus and / or a model that directly estimates the probability distribution of interruption costs for different types of notifications.

  A Bayesian network can be used that can estimate the probability of alternative activity contexts or states based on a set of observations about the user's activity and location. As an example, FIG. 9 illustrates a Bayesian network 154 that estimates the user's attention focus for a single time period. The variable state of attention focus 156 refers to desktop and non-desktop contexts. Exemplary contexts of attention considered in this model include, for example, situational awareness, understanding, unspecified background tasks, focused content generation or review, light content generation or review, documentation Includes browsing, office meetings, non-office meetings, listening to presentations, private time, family time, personal focus, relaxed conversation and travel. Bayesian network 154 shows that the user's current attention and location is affected by the user's scheduled visit 158, time 160, and deadline proximity 162. The probability distribution of the user's attention is also affected by a summary of the situation of the environmental acoustic signal 164 monitored at the user's office, for example. Over time segments of the environmental acoustic signal 164 provide clues / inputs regarding the presence of activity and conversation. The status and configuration of the software application and the ongoing stream of user activity generated by the user interacting with the computer also provide a source of evidence regarding the user's attention.

  As shown in the network 154, the software application 166 that currently has the highest focus in the operating system or other environment affects the nature of the user's focus and task, and the user's attention status and focus. There are applications that, together, affect computer-centric activities. Such activities include a stream of user activities created from a sequence of mouse and keyboard actions and a high level pattern of application usage over a wider time range. Such patterns include email centrics and word processor centrics, and such patterns refer to prototype classes of activities that accompany multiple applications being interleaved.

  FIG. 10 shows a Bayesian model 168 of user attention focus between context variables at different time periods. A set of Markov time dependencies is shown by model 168, in which the past state of the context variable is considered in the current determination of the user's state. In real-time, such a Bayesian model 168 examines the information provided by, for example, an online calendar and a stream of observations about room acoustics and user activity reported by an event sensing system (not shown), and Continue to provide estimation results on the probability distribution of attention.

  11, 12, 13, 15, 17, and 21 illustrate a method for providing portions of a notification architecture, such as a context analyzer, notification manager, and user interface according to the present invention. For simplicity of illustration, the methods are illustrated and described as a series of acts, but in accordance with the present invention, some acts may be in a different order and / or others than those shown and described herein. It should be understood and understood that the present invention is not limited by the order of actions, since it can be performed concurrently with these actions. For example, those skilled in the art will understand and appreciate that a method could instead be represented as a series of interrelated states or events, such as a state diagram. Moreover, not all illustrated acts may be required to implement a methodology in accordance with the present invention.

  This method can be computer-implemented in several ways. The computer-implemented method is preferably implemented as one or more programs operating at least in part on a computer, i.e., as a program executed by a computer processing system from a computer-readable medium such as a memory. The program is preferably storable on a computer readable medium such as a floppy disk or CD-ROM for distribution, installation and execution on another computer. The one or more programs can be part of a computer system or computer, such as that described below with respect to FIG.

  Referring to FIG. 11, a flow diagram 170 illustrates determining a user's context according to the present invention. This process includes determining the user's position at 171 and determining the user's focus at 172. These actions can be accomplished by one or more of the techniques previously described. For example, profiles can be used, the user can specify his / her context, direct measurement of context can be used, a set of rules can be followed, such as via Bayesian models or statistical models An estimation analysis can also be performed. It should be appreciated that other analyzes can be used to determine the user's context. For example, an integrated video camera source can be provided that recognizes when someone is in front of a computer and realizes whether the person is looking at the computer. However, it should be noted that the system of the present invention can operate with or without a camera. For all of the sources, the system can operate with virtually any number of available input sources without requiring a specific source for contextual estimation. Furthermore, in other aspects, an integrated accelerometer, microphone, and proximity detector on a small PDA can be provided that provides sensing of the user's position and attention.

  Referring to FIG. 12, a flowchart 173 illustrates notification manager determination processing in accordance with an aspect of the present invention. At 174, one or more notification sources generate a notification that is received by the notification manager. At 175, the context analyzer generates / determines context information about the user, which is received at 176 by the notification manager. That is, according to one aspect of the invention, at 175, as described in the previous section of this description, the context analyzer has access to the user's context information profile that indicates the user's current attention status and location; And / or real-time information regarding the user's current attention situation and location is assessed from one or more contextual information sources.

  At 177, the notification manager determines which notification to communicate to which notification sink based in part on the context information received from the context analyzer. The notification manager also makes decisions based on information about the user's notification parameters stored by the context analyzer. That is, according to one aspect, at 177, the manager performs a decision-theoretic analysis as to whether and how to notify the user about the given notification. Decision theory and / or heuristic analysis, decision, and policy can be used at 177 as described in detail below. Notification parameters for the user can be used to personalize the analysis by writing to missing values or by overriding parameters provided in the source or sink schema. Notification preferences can also provide policies (eg, heuristics) that are used in place of decision-theoretic analysis. Based on this determination, at 178, the notification manager communicates the notification to the sink.

  Various aspects of the present invention have been described so far as applicable to users. However, the present invention itself is not limited thereto. That is, the present invention is applicable to virtually all types of entities including users. Other types of entities include, for example, agents, processes, computer programs, threads, services, servers, computers, machines, companies, organizations, and / or stores. For example, the agent can be a software agent, which typically performs a background task for the user and reports to the user when the task ends or some expected event occurs. It can be defined as a program. Other types of entities can be included under the present invention so that one skilled in the art can appreciate. For example, a context analyzer according to another aspect of the present invention can be generalized as a component applicable to substantially all types of entities. As another example, a notification sink can generate notifications, alerts, and events for entities other than users. Similarly, a notification sink can receive notifications, alerts, and events regarding entities other than users.

  Turning to FIG. 13, a flowchart 180 illustrates a decision-theoretic decision that can be performed by a notification manager according to an aspect of the present invention. At 182, one or more notifications are received. Notifications provide information that can be communicated to the user via the associated notification sink mode. At 184, a decision-theoretic analysis is performed on the notification received at 182 for multiple modes of multiple sinks. It is desirable for the analysis to provide the net value of communicating notifications through the mode associated with the sink. The analysis can be performed using a probabilistic model, such as a Bayesian network.

  According to one aspect of the present invention, the determination of the net value of delivering a notification by the mode of sink at 184 includes performing 186, 188, 190, and 192 of FIG. At 186, the expected value for the user of the information included in the notification is determined. This is the value of information that occurs to the user when the user is notified. At 188, the expected cost for the user to interrupt notification is determined. This is the cost of interrupting the user to convey the notification, for example, the user may be busy in a meeting, and the interruption of the user due to the notification may result in a cost for the user. At 190, the expected value of the user independently knowing the information contained in the notification without actually transmitting the notification is determined. This value may be less than the value determined at 186. This is because the user may know the information independently after being notified of the information. At 188, the actual cost of communicating the notification to the user is determined. For example, sending a message through a pager may result in a communication charge incurred by the user from the user's pager company, such pager being billed by the company on a per-call basis. The

  The net value of notifying the user via the sync mode is 184, the expected value of the information determined at 186, the expected cost of interruption determined at 188, and the user at 190 This can be determined by subtracting the expected value of knowing independently and the actual cost of 192 communications. At 194, it is determined whether the net value of any of substantially all modes of substantially all sinks is greater than a predetermined transmission threshold. For example, if the net value is measured in dollars ($), the predetermined transmission threshold can be zero. If the net value of the notification is greater than the sink mode threshold, the process proceeds to 196 for such a notification and sets such notification to the mode of the sink with the highest net value for the notification. To the user via Otherwise, for notifications that do not have a net value exceeding the threshold for substantially all modes of substantially all sinks, the user is not currently notified of the information contained in such notifications and this process Proceed to 198 for such notification and perform post-processing, but this processing also proceeds from 196 to this post-processing.

  The present invention is not limited by the manner in which post-processing is performed at 198. According to one aspect of the present invention, the notification communicated to the user at 196 can be deleted assuming that 196 has been performed. In another aspect, such notifications are deleted when a confirmation is received that the user has actually received the notification from the notification sink in which the notification was communicated. The notification can also be deleted after transmission if it is determined that the notification sink to which the notification is communicated has a transmission reliability higher than the threshold with respect to the mode of the sink used. Further, the process of FIG. 13 can be repeated at predetermined intervals and / or when a new notification is received. For example, as long as the net value of the notification determined at 184 is time-dependent, a given notification that may later have a net value less than the transmission threshold has a net value greater than the threshold, There is a possibility of being told. A disjunctive situation can also be true. Thus, the process shown in FIG. 13 shows a form in which a decision-theoretic analysis can be performed to determine whether notifications should be communicated to the user via the mode of the sink, Can be repeated as desired.

  Note that the process shown in FIG. 13 has been described with respect to performing a decision-theoretic analysis of notifications for multiple modes of multiple notification sinks. However, the present invention itself is not limited to this. For example, only one such mode can be provided implicitly for any or all of the sinks. In this way, the analysis of notifications can be performed on the sink without explicitly entering the mode. Further, as noted, the determination of the net value of the notification regarding the mode of the sink can be performed as described in the next section of this description.

  According to one particular aspect of the present invention, the decision-theoretic notification presented in the previous section of this description can be implemented in the manner described in the following section, but the present invention is not so limited. For example, iterative “greedy” decision-theoretic analysis can be used. During the analysis, consider the current context and the associated expected value of the alert transmission. A dynamic Bayesian network or Hidden Markov Model (HMM) with less approximate, more accurate decision-theoretic analysis that performs future estimations, considers future time ranges, contexts, and associated expected values A model such as an approximation of a dynamic Bayesian network can be used. Such techniques can be used to make notification decisions based on a less “myopic” analysis that “predicts” context in a future state. It is known in the art to generalize myopic analysis to a richer, less myopic analysis. With respect to the notification platform, an additional amount of computation is used in these “less greedy” analyses. An aspect of the notification manager is configured to be able to shift to a less myopic mode based on computational considerations that are currently available or available by monitoring the status of available computational resources Is done. That is, the present invention is not limited to the greedy technique described. Ideal time and device optimization for notifications, which are less approximate and less greedy, predict the likelihood of future context and device availability, thereby reducing the scope of future context and associated device availability. Can be considered.

  The expected value of the notification N at time t can be regarded as the current value of the notification. The informational value of the notification is considered sensitive to the user's context and knowledge. The context includes context information such as the user's location and attention situation, the user's goal, and context (eg, the user has just opened an email). The initial value of notification N in context C is still familiar to the user from the value of the notification in the context when the notification was first generated by the source (eg, it can be measured in dollars). Minus the probability. The probability that the user is not familiar with the information is called information novelty. This probability is based on evidence E, such as the type of information and the form in which the information is distributed (eg, newspaper articles become known over time through other channels, and thus the evidence includes Can include key points and age).

  When considering the value of information already known to be zero, the value of notification is

It is. The concept of contextuality can be introduced by conditioning the value in context C and assessing the value based on context.

At some new time t, the value of sending a notification may change based on the time dependence of the value.

The value function can be represented by a time-dependent function that takes as an argument the time difference or delay between the time the alert was sent or received by the notification manager and the current time, where the delay is expressed as t−t _0. The Such functions can include, for example, linear functions, exponential functions, and sigmoid functions that indicate a loss of value delay. To a more complex function, the “shelf life” refers to the period of time that follows the time the alert was sent or received, before the value of the information was changed (eg, started to decay) life) ", and the concatenation of linear functions, exponential functions, and sigmoid functions. Other functions can capture the notion that alerts can increase in value with some delay.

  According to one aspect of the invention, the fact that the context also changes and is different at the new time is taken into account. Thus, equation (3) can be rewritten using C (t), or the context can always be shown as the current context. Under the context uncertainty, the different potential contexts are summed. Therefore, the expected value of information is

become. This is worth the user receiving all the content of the notification in context C at some time t.

  The expected value of communicating information using the device's mode M is due to the loss of fidelity associated with rendering and whether information was sent to the user when signaled using mode M in context C: Reduced. For simplicity, it can be assumed that the fidelity of the transmission is captured as a variable that ranges from 0 without content transmission to 1 with full content transmission. In accordance with other aspects of the invention, additional details of the loss of dropping one or more components of the initial content and truncation and summarization of the content in various forms (e.g., text of an email message A more detailed usability model is considered that captures a percentage of the whole or another method of summarizing into smaller, more compact messages for display on a limited display on mobile phones . In the general case, the fidelity associated with sending information in mode M of the device depends on the context. For example, in a noisy environment, it may be difficult to hear the audio portion of the audio content.

  The probability that information is transmitted to the user can also be considered. This is also context dependent in the general case. This dependency is usually more pronounced than the fidelity context dependency, so it can be specified explicitly. The transmission of information as the probability that the user has received the information is represented as p (received | M, C, E, e), where e is the user's notification, eg, pause, mouse over, dialogue, etc. Represents additional evidence regarding the response to.

  Next, the expected value of notification communication is

As determined. Note that the expected value of the communication in equation (5) is described in terms of the expected value of the notification information. this is

Similar to. In one aspect, the expected value of communication performed in equations (5) and (6) can be used as the expected value of information for the user described in the previous section of this description. Instead, the expected value of the information can be the expected value without consideration of fidelity and other parameters, ie ExpValInfo (N _i ). However, the present invention is not limited to these methods.

  Next, consider the cost of information. The cost associated with interruptions depends mostly on the mode and context of transmission through the context of the user's attention. The expected cost of user interruption in each context can also be measured in dollars in one aspect, and is equal to the amount that the user is willing to pay to avoid interruptions associated with the transmission of information via mode M. In the general case, this also depends on the details of what is sent. However, according to one aspect, different costs under context uncertainty are specifically considered. Therefore, the expected cost of mode M interruption is

It is. The value of signaling to the user now using notification via mode M is the difference between the value of information and the cost. Costs in dollars of actual communication are also considered, for example, the cost of sending a bit for each fee charged by the service. This can be a function of the notification content and the mode selected. This is also referred to as (actual) communication cost ComCost (N, M).

  Second, the opportunity for the user to review for email stores or general purposes where the user is not actively signaled by notification but receives the information later when the user has time to review the information. A (net) value can be considered non-zero if information can be actively retrieved from a store, such as a store of potential notifications that are maintained until obtained. This is referred to as the expected value ExpValSeek of searching for information contained in the notification mentioned in the previous section of this description as the expected value of the user knowing the information independently without notification. This value can be determined by considering the time until the user reviews the information included in the notification. This time is usually context-responsive, for example because the time it takes for a user to retrieve information from such a store may depend on location, time, and current attention focus. . Consider that the novelty of the information can change and can be a function of the length of time the notification was held. For simplicity, fidelity can be considered the maximum when a user searches for information, but in the general case the user can search for information via a device that provides lower fidelity There is sex. Also, it can be assumed that the cost of interruptions associated with information retrieval is about zero because the user is in a state of care to actively track information.

  Therefore,

It is. Regarding the determination of the waiting time between the notification time and the time until the search, it should be noted that there are a plurality of methods for performing and determining the equation (8). In one aspect, it can be assumed that t is distributed in a Poisson distribution and the time of retrieval is the (memoryless) average time from the time of analysis until the user reviews the notification. The waiting time can be determined as a difference between the time and the notification time. In addition, Bayesian networks or other probability models can be used to estimate probability distributions for different average times reviewing email or more general notification stores. As explained above, a Bayesian network or other probability model can also be used to determine the probability distribution of the user's attention focus, location.

  Therefore, the expected value NetExpValCom of the notification communication regarding the communication of the notification N in mode M is

It is. This is what we called net value in the previous section of this description.

  For decision making, consider NetExpValCom for virtually all modes M of virtually all devices with respect to incoming notifications. The device with the largest positive NetExpValCom is considered (ie, a pre-determined transmission threshold of $ 0 is assumed since this term was represented in the previous section of this description). If NetExpValCom is positive for multiple devices (eg, communication sinks), select the device with the greatest value and use it to signal the user. If the value is negative for substantially all modes of virtually all devices, the notification can be postponed and journaled for later review. The value of rendering notifications continues to be reviewed in one aspect, by updating variables that change over time. This includes variables such as the current time, the expected time for the user to review their email or more generally their notification store, and the current context and novelty of information . Such a review can be performed as part of the post-processing described in the previous section of this description.

  Note that this iterative estimation of current versus past is a type of decision-theoretic analysis performed in one particular aspect of the present invention. This is a greedy decision making strategy. However, a less greedy strategy can be formulated that relies on a somewhat more complex prediction model that considers the value and cost of active notifications at future times. For example, a probabilistic model can be used to predict a user's future state of attention, and such a prediction can be used to estimate an increasingly less greedy form.

  Furthermore, even after signaling once, in some aspects, notifications are not immediately discarded (ie, deleted). For example, after a notification has been rendered, it can usually not be guaranteed that the notification has arrived to the user. However, between the user and the system, for example, the fact that the user hovers the cursor over a notification rendered in a desktop scenario is a form that the user has "received" Such a guarantee is possible if processing such as shared understanding is being implemented, or otherwise automatically monitoring notification access. An example of the latter is to monitor whether the user has examined the message on his mobile phone. Such a monitoring report may be a confirmation of receipt of the notification noted in the previous section of this description.

  The mode of the sink is considered to have the context response transmission reliability (also referred to as transrel) transrel (M, C) in that context. That is, for that mode, and for that context, the transmission reliability gives the probability that the user observed the notification based on its rendering. As noted, sometimes confirmation that transrel is 1.0 may be received, for example, a user has observed a notification with a probability of 1.0, either by interacting with the notification or by mouse-over of the notification. It will be. At other times, the transmission reliability of the mode and context can be relied upon.

The likelihood p (receive) that the user has received the information of each notification is updated after each transmission. H ^A (N _i ) generally refers to the alert history of a particular notification being held in the internal inbox. The warning history shows the sequence of notifications attempted,

It is. A (N _i , M) indicates an alert related to the notification Ni in mode M. Given the notification history, it is possible to determine the novelty p (notificationunsen | ^HA , E, e) of the current notification. By including this factor, the expected value of viewing the notification is appropriately reduced.

  Specifically, first, the updated ExpValCom and ExpValSeek are

および

and

become. Next, NetExpValCom is determined using the new ExpValCom and ExpValSeekInfo values, similar to the previous one. Therefore,

It is.

Further, the notification novelty p (notification unsenen | ^HA , E, e) is generally updated. According to one aspect of the present invention, this is determined after a new attempt at alerting (notification rendering or notification delivery) is made, for example by considering the attempt as a binary attempt, as will now be described. be able to. Warning history
H ^A (N _i ): {A ₁ (N _i , M, C (t ₁ )), A ₂ (N _i , M, C (t ₂ )), A (N _i , M, C (t ₃ )) ),. . . A _n (N _i , M, C (t _n ))}
The novelty of the notification is

become.

  You can also chunk notifications by reviewing a notification set that includes a set of simultaneous notifications, ie grouping together for transmission as a grouping of notifications through a given mode of a given notification sink Note also that you can.

Therefore, a summary of the value and cost of the notification is considered and one interruption penalty is expected.

  In this section of the description, various extensions to the aspects of the invention described in the previous section are presented. First, it should be noted that in one aspect, a decision-theoretic policy can be edited and / or approximated to simpler rules and policies. This can use a formal method of compiling such a decision-theoretic analysis into a policy. In addition, there are various policies, such as heuristic policies, that perform a coarser cost-benefit analysis, as described in detail below.

  In addition, decision-theoretic policies can be used for "pull information" situations. That is, when the user requests information from the system, including requests in the desktop and mobile contexts, the distracting cost of being zero is considered and the next most valuable value that information is sent to the user. Relevant for notifications. Such information can be ordered by the next highest value, or can be grouped into categories for recognition. For example, the next “n” most valuable notifications can be examined and commands can be related to stream notifications in this order, or “next” in the order of expected utility. You can wait for a "notification of" request.

  Instead, the information can be related with respect to the category of the source, eg based on the order of the source containing the notification with the highest expected usefulness. Notification relaying can continue in the source category until the expected value threshold is reached before moving to the source containing the notification with the next highest value, after which the process is repeated. Instead, the information can be relayed by a predefined ordering of the sources (eg, voice mail message first, instant message next, email next, then financial notification), then The notifications from each of the sources, sorted by expected usefulness within the category, can be relayed until the expected usefulness threshold for that category is reached, and then proceed further.

  The expected value of information can be used to adjust a high level summary of the current situation. For example, a cross-source estimate can be provided to create a text-to-sound summary of pending notifications for communication of current notification status via a mobile phone. In addition, the expected value determination can be used to achieve caching. To better hear by assuming that the user is most interested in the item with the highest expected value, for example, to enhance dialogs in mobile and desktop settings using expected value determination Information can also be given to the voice recognition system.

  Furthermore, another extension of the described invention is that the summary can be adjusted using a set of expected values within the source category. Such a summary can appear in a permanent summary to relay a summary of the status of each source's notification. For example, an e-mail summary might be: “32 unread messages; 9 of high urgency; most source from Andy on 'Meeting this after afternoon' (32 unread messages, 9 high urgency, The most urgent message is “Tonight's meeting” from Andy).

  Heuristic communication decisions and policies may be performed by the notification manager, which will be described in accordance with the present invention. For example, gross cost-benefit analysis can be used, bypassing the more formal decision-theoretic analysis. Such policies and associated notification components and interfaces can be viewed as approximate or heuristic versions of decision-theoretic policies. In this approach, high urgency, medium urgency, low urgency (or any range of urgency), depending on the notification, by source, or by user-specified notification profile (eg, per message attribute and / or message class) ) Can be labeled. Can create a list of conditions for when users are likely to receive notifications, perform coarse context monitoring, and are likely to have time to receive notifications with minimal interruption Identify the situation. These states are referred to as “probably like free” states.

  This list can include one or more of the following (and other states):

The user exists and entered and paused for x seconds. The user saved the file and paused for x seconds. The user sent an email and paused for x seconds. The user closed the application ・ The user switched from one application to another

A maximum postponement time can also be set for the urgency level. For example, internally, the following can be set in an exemplary table:

Maximum postponement (high priority): 2 minutes Maximum postponement (medium priority): 7 minutes Maximum postponement (low priority): 15 minutes

This can be set by the user, but can instead be set by the system developer for default behavior, which may or may not be modifiable by the user.

  In addition, the user can list exceptions or emergencies, for example as receiving an immediate passthrough.

  The following is an exemplary algorithm according to an aspect of the present invention.

When a notification is received, set its age to 0, note the priority, and examine the list of exceptions.
• Pass through notifications to the user if a free state is observed via monitoring the user's activity before the maximum postponement time for that urgency.
Otherwise, relay the notification when the maximum free state of notification is reached.

  On average, most notifications are generally delivered before the maximum postponement time. However, the user will usually be glad that when a notification is received, if the notification is simply passed through, it will tend to occur when it is more liberal. Therefore, the probability of reaching the free state increases with time. Since the probability of being free is likely to increase with time, low priority messages tend to occur at a higher likelihood, probably during free states, and the probability of interruption is the priority of the message. As it goes up, it gets higher.

  This technique can be generalized as follows. According to one aspect, display of notifications can be enabled to include multiple or polled waiting notifications, and the user can be sent a single notification that includes chunks of grouped notifications. Such chunking may present chunks of notifications in a list ordered by, for example, maximum priority, maximum age, or maximum priority by group. For example, if there is probably no free state and the maximum postponement time for high priority notifications has been reached, the maximum postponement for high priority notifications is pending in grouped notifications Information about low priority notifications is included. This is the case even if the low priority notification has not reached its maximum postponement at this time.

  Furthermore, instead of a few categories of priorities, a continuous range of e.g. 0-100 can be enabled for the urgency score, and the maximum deferral can be set to various linear and non-linear functions (e.g. increasing priorities). Can be a function of notification priority, including the exponential decay of the maximum postponement time associated with. For example

Maximum delay (priority) = e ^-k (priority) x 15 minutes or Maximum delay (priority) = e ^-k (priority) x maximum delay (0 priority)
It is.

  The probability of free time can be learned within the user's next x minutes. This can be accomplished by learning the frequency of the probably free state and the expected time to the next possibly free state. The time until the next possibly free state can be determined from the user's activity, so that instead of the maximum postponement time, the user can specify the probability of priority classes to be disturbed. The maximum postponement time for the notification priority class can be set automatically. That is, the user can specify a target “permissible probability” for priority class interruption, and the system can set the maximum postponement time for this class. That is, the user (or alternatively, the system developer by default), for example, “0.5 probability of interruption for high priority notifications, 0.25 possibility of interruption due to medium priority messages, low priority notifications. The notification system is configured in such a way as to allow 0.05 possibility of disturbances.

  Below, an overview of a user interface according to an aspect of the present invention is shown. An example of such an interface is shown in FIG. 14, in which a predetermined area 302 (in a desktop screen 300 of a computer display (eg, a laptop computer, desktop computer, or other display) is displayed. For example, an output display and / or for providing user interaction is provided. As can be seen from FIG. 14, the predetermined area 302 is in the upper right of the screen 300, but it should be appreciated that other areas of the screen (eg, lower left, right, etc.) can be used. For example, in the stream stacking aspect of the invention described later in this description, region 302 can be the right column of screen 300. The screen 300 is preferably one on which the user can control the movement of the cursor cursor 304, as used in a graphic user interface. The cursor 304 shown in FIG. 14 is an arrow pointer, but it should be appreciated that other cursors can be used.

  The predetermined area 302 can be used in connection with a display associated with various aspects of the invention. Information used herein can refer to a single piece of information and / or multiple pieces of information. According to one aspect of the invention, the information includes a notification alert, also referred to as an alert or notification, as described above. Accordingly, various aspects of the present invention are directed to the display of such information within a predetermined area 302 of the desktop screen 300 described below. In one aspect, the desktop screen 300 can be used by a user for major tasks, such as operating a word processing document, spreadsheet workbook, or other application.

  However, the information displayed in region 302 may not be relevant to the main task. As an example, the displayed information may be information that is not requested by the user. For example, a user has requested an email that exceeds a predetermined classification threshold that is communicated to him (eg, information classified according to importance), but did not request that the email be displayed in area 302 (“non- The information can also alert the user about the email.

  The screen 300 can be part of a display that can provide generalized rendering, including content formatted according to, for example, the Hypertext Markup Language (HTML) format. In addition, multiple sources of information can send a “rich” interface (eg, source branding) that includes buttons, links, animations, audio, etc., and the information is user interface described herein. Rendered within the constraints and high-level design or style conventions. However, the present invention is not limited to this.

  The following sections of this description describe the pulsing aspect of the invention, the stream cycling aspect of the invention, and the stream stacking aspect of the invention. These are specific aspects by which information can be displayed, for example, in a predetermined area 302 of the desktop screen 300. The following sections show at least one example of these aspects, but it should be noted that the invention itself is not limited to these examples. In addition, there can be a combination of pulsing mode, stream cycling mode, and stream stacking mode that the user can switch between modes. For example, the system can include a display, a processing system, and a computer readable medium having stored thereon a computer program that causes one of the modes to enter when executed by the system.

  Instead of the user switching modes, in one aspect, the notification manager described above can perform a switching decision, for example. In one aspect, the user or notification manager may also make decisions regarding features that can be switched within a given mode, such as a pulsing mode, a stream cycling mode, and / or a stream stacking mode. The presence or absence of a voice announcement may also be a decision delegated to the user and / or notification manager in one aspect.

  Referring to FIG. 15, a flow diagram of a method 400 of the pulsing aspect according to the present invention is shown. At 401, information is received. As previously described, the information may be unsolicited information that is not related to the user's primary work. The information can include a notification alert having an assigned classification, such as being associated with an importance value that is greater than a threshold defined by a predetermined threshold. The importance value measurement is not limited by the present invention, nor is it limited by the threshold.

  At 402, information is faded into a predetermined area of the display. In one aspect, information is displayed in a predetermined area and an alpha value of information displayed in the predetermined area (eg, a luminance value associated with a display pixel) is incremented to a first predetermined level at a given rate. Information to fade in. The first predetermined level may be based on the importance of the information defined by the importance value. For example, the level can be proportional to the importance of the information. Increasing the alpha value of the information increases the opacity of information display in a given area. Thus, increasing the alpha value to a level based on the importance of the information means that more important information is displayed more opaque, i.e., less transparent than less important information. However, in one aspect, the predetermined level is less than 100%, ie not 100% opaque. Furthermore, a voice announcement that warns the user about information that is faded into a predetermined area can be played back at 402. An audio announcement can be a predetermined sound or sounds, and the importance value of the information can be associated with various aspects of the sound (e.g., volume that increases or decreases based on importance, Increase or decrease the number of sounds based on importance).

  At 404, there is a delay related to the length of time based on the importance of the information. For example, the length of time can be proportional to the importance of the information. Thus, the delay is preferably the length of time that information is displayed to the user. Therefore, information with higher importance can be displayed longer than information with lower importance. In one aspect, processes 406, 408, 410, and 412 of process 400 are performed during the length of time that is delayed, but the invention itself is not limited thereto.

  At 406, a first predetermined user gesture associated with a fade of information to a predetermined area of the display is detected. For example, the first gesture can be a cursor movement over a predetermined area of the display (eg, by causing the user to cause such movement through the use of a pointing device such as a mouse). However, the present invention itself is not limited to this. Another gesture may include a specific language or voice by the user that is detected. In response to the first gesture, at 408, the first action is performed. In one aspect, the action includes increasing the alpha value of the information displayed in the predetermined area to a second predetermined level, such as 100%, that is higher than the first predetermined level. Therefore, information can be made more opaque by the first gesture. In another aspect, in response to the first gesture, at 408, more detailed information (eg, related to alerts) is displayed in a predetermined area of the display.

  At 410, a second predetermined user gesture associated with fading information to a predetermined area of the display is detected. For example, this second gesture may cause the cursor to no longer be over a predetermined area of the display (eg, by causing the user to perform such movement through the use of a pointing device such as a mouse or keyboard movement). Thus, the cursor can be moved to the display area. Another gesture includes a specific phonetic pronunciation by the user that is detected. In response to the second gesture, at 412, the second action is performed. This action may include reducing the alpha value of the information displayed in the predetermined area from the second predetermined level previously adjusted at 408 to the first predetermined level. According to another aspect of the invention, more detailed information that may have been displayed in a predetermined area of the display at 408 is replaced with information previously faded there at 402.

  At 414, information is faded out of a predetermined area of the display when the delay of 404 has elapsed. For example, in one aspect, this includes reducing the alpha value of information displayed in a predetermined area at a predetermined speed, and then no longer displaying that information in the predetermined area. As indicated by 416, the process indicated at 400 can be repeated. That is, new information that may have new importance can be received at 401 and the new information is faded into a predetermined area of the display at 402. Note that in one aspect, everything already displayed in a given area remains there so that fade-ins and fade-outs to the given area are understandable. That is, information that fades into a given area is displayed over what is already there, and the alpha level of the information that is faded in is increased, so how transparent or opaque the information that fades in is. Thus, it is determined how much information the user can observe. Information (although not completely propagated in the space) can be partially observed.

  The process described in FIG. 15 is referred to as a pulsing aspect because information is “pulsed” to a determined alpha during a determined time associated with an alert or notification classification (eg, importance value). This is illustrated with respect to FIG. 16, in which a diagram 500 of such a pulse 502 is shown, according to aspects of the present invention. The pulse 502 has a height 506 representing the alpha value level at which information displayed in the predetermined area is increased to that extent, and a length 504 representing the length of time that information is displayed in the predetermined area at this alpha value level. And a first slope 508 representing the speed at which information is faded to this alpha value level and a second slope 510 representing the speed at which information is faded from this level. In one aspect, height 506 and length 504 are based on the importance of the information being pulsed (eg, in one aspect, height is proportional to the importance value). In one aspect, the first slope 508 and / or the second slope 510 are constants, but the invention itself is not so limited. Further, the gradients 508 and 510 can be similar to each other.

  In one aspect of the invention, tabs, buttons, and / or other items are on the display, and selecting these items can cause the user to immediately cause the next notification to be displayed. For example, a button click indicates that the user wants to see the next notification even if the next notification has not reached its own display importance value or threshold. Such a notification may not have importance over a threshold for independent display, for example.

  Referring now to FIG. 17, a method 600 for a stream cycling aspect according to the present invention is illustrated by a flow diagram. At 601, a related display time for each number of different information packets (eg, information related to a notification or alert from a notification source) is determined. A display plan for an information packet is the length of time that this information is displayed within a predetermined area of the display. In one aspect, the length of time is based on the importance of the information, and an importance value is assigned to each information packet. For example, the display time can be directly proportional to the importance. However, the present invention is not so limited. Further, as described above, the information can be unrequested information that is not related to the user's primary task. The information can include a notification alert.

  At 602, in one aspect (ie, 602 is optional), a period for each information packet is determined. The period for an information packet is the number of times information is displayed within a given area of the display during a given period. For example, the period can be displayed according to a predetermined protocol (eg, a proportional relationship associated with the classification) based on the classification. In one aspect, the period can be based on the importance of the information, for example, directly proportional to the importance value. Thus, during a given period, more important information can be displayed more frequently than less important information. In one aspect of the invention where 602 is not performed, each information packet may have a period approximately equal to 1, that is, each piece of information can be displayed once during a given period. .

  At 604, during a given period, each information packet is displayed in a predetermined area of the display for a time period approximately equal to the display time of the information packet and a number of times approximately equal to the period of the information packet. . Thus, the first information packet is displayed, then the second information packet is displayed, and so on until all information is displayed during the given period, and so on. . In one aspect, each piece of information may fade in a predetermined area with a delay approximately equal to the display time and then fade out (eg, increase the alpha value) as described in the previous section of the description. , By delaying and then lowering the alpha value). According to such an aspect of the invention, the first predetermined level at which the alpha value of the information packet is raised can be based on the importance of the information, as described above. That is, the alpha value is eventually set to the first predetermined level for a length of time approximately equal to the display time. In one aspect, an audio alert that alerts the user to each displayed information is played, or alternatively, an audio alert is sent for information that exceeds a threshold, such as a predetermined threshold. The audio notification can be a predetermined sound or sounds, as described above. In one aspect, processes 606, 608, 610, and 612 are performed during a predetermined period of time (not specifically limited to the present invention). However, the present invention itself is not so limited.

  At 606, a first predetermined user gesture related to a current information packet displayed within a predetermined area of the display is detected. For example, the first gesture can be a cursor movement on a predetermined area of the display (eg, by a user that causes such movement through the use of a pointing device such as a mouse). Another gesture may include a specific utterance or voice by the user that is detected and / or processed. In response to the first gesture, at 608, a first action is performed. In one aspect, the action “holds” the current information displayed in the predetermined area, and the other information is substantially in the predetermined area until a second gesture is detected at 610. To prevent it from being displayed at all.

  That is, the display time of the currently displayed information is substantially increased temporarily, and the current information is kept within a given area until a given period of time is detected at 610 for the second gesture. Increased to a length of time equal to the length of time while held. In another aspect, the first action performed at 608 includes an alpha of current information displayed within a predetermined region to a second predetermined level that is greater than the first predetermined level, eg, 100%. Increasing the value is included. In such an aspect, the first gesture thereby causes the current information to be displayed to become more ambiguous. In another aspect, in response to the first gesture, at 608, more detailed information (eg, related to the alert) is displayed within a predetermined area of the display.

  At 610, a second predetermined user gesture related to current information displayed within a predetermined area of the display is detected. For example, this second gesture causes such movement through the use of a pointing device, such as a mouse, to move the cursor to an area of the display so that the cursor is no longer over a predetermined area of the display. Depending on the user who Another gesture is a specific utterance of the user being recognized. In response to the second gesture, at 612, a second action is performed. In one aspect, the second action can “release” current information that was previously “held” in a predetermined display area and continue to display subsequent information in the predetermined area. To be included. The action includes increasing the alpha value of the information displayed within the predetermined area previously at 608 or reducing it again from the second predetermined level set to the first predetermined level. Is possible. In another aspect, more detailed information that may be displayed within a predetermined area of the display at 608 is replaced with the same information that was previously displayed at 602.

  At 614, the information is updated after a given period of time when substantially all information is displayed at 604. For example, 614 can include adding new information and deleting old information. Deletion of information can be based on, for example, information with the lowest priority, a predetermined number of times, information already displayed, and the like. Similarly, new information that is added may include information of importance that exceeds a predetermined threshold associated with the importance of the information. Next, as indicated by 616, the process 600 of FIG. 17 is repeated. Accordingly, at 601 a new display time for each updated information packet is determined.

  The aspect of the invention described in connection with process 600 of FIG. 17 is referred to as stream cycling. This is because the information is “streamed”, ie, during a given period, the first information is displayed within a predetermined area, then the second information is displayed, and so on. Because it is done. This will be described with reference to FIG. 18, where a diagram 700 of a stream cycling wheel 702 according to the present invention is shown. The wheel 702 has several slots 1 to N, where N is an integer 704-708. Slot 704 corresponds, for example, to an example of a portion of information displayed during a given period. Each slot has a time delay corresponding to how long that portion of information is displayed during a given period. For example, slot 706 has a time delay represented by the length of arc 710 and slots with longer arcs have a larger corresponding time delay. Each piece of information is allocated to a number of slots approximately equal to the period of information. Therefore, information having one cycle is allocated to one slot. The number of slots and a given period are increased and decreased to be approximately equal to the sum of all the periods of information, and the given period during which substantially all instances of information are displayed Note that it is possible to be approximately equal to the sum of the example time delays.

  Wheel 702 rotates as indicated by arrow 712 such that viewing arrow 714 pointing to wheel 702 points to a different slot in wheel 702 for a given period of time. The slot 704 that the arrow 714 points to contains information that is currently displayed within a predetermined area of the display. Thus, as the wheel 702 rotates during a given period, different slots are pointed to by arrows 714 so that different information is displayed within a given area. The rotational speed of the wheel 702 is such that the wheel 702 can make one complete revolution during a given period. It should be noted that the wheel 702 of FIG. 18 is a conceptual diagram of the stream cycling aspect of the present invention, and in practice it is not necessary to provide such a wheel to implement this aspect.

  According to one aspect of the present invention, a portion of the information that can be streamed is more likely to be retrieved from the most critical notification in the current cycle, or more generally from a larger notification store. A summary page, which is information that includes a high-level summary of a large amount of information. The notification is immediately displayed by the user selecting a notification that is specifically referred to in the summary. In one aspect, each page contains one or more summary pages containing a cluster of information sets, each containing a group of related information, eg, virtually all communications (eg, instant messages, emails, incoming calls) Call) summary page and / or summary for virtually all automated services. Furthermore, according to another aspect of the present invention, a user can stop cycling information, click quickly through the cycle, pause where he wants to pause, and / or drill down for other information. There can be an explicit set of controls that allow to do that. In one aspect, information presented by stream cycling can be displayed on a separate display.

  The following section of the description describes the stream stacking aspects of the present invention. The diagram of FIG. 19 shows a display 800 with such stream stacking. Display 800 includes a main notification window 802, a journal window 804, and several source summary windows 806, substantially all of which are within a predetermined area of display 800 (eg, the screen of display 800). It is supposed to be displayed. As mentioned above, there are several information sources such as notification sources. As described above, each information source generates information such as unrequested information that can include information not related to the user's primary task, and displays the information in the corresponding source summary window 806. indicate. As also mentioned above, the information can include a notification alert.

  An importance value can be assigned to each piece of information, and the criteria for the importance value is not limited by the present invention. For example, information having an importance level higher than a threshold value, such as a predetermined threshold value, is displayed in the main notification window 802 in a stream cycling manner. For example, stream cycling can follow the stream cycling aspect of the present invention described above, where each piece of information is faded in for a length of time that is in the main notification window 802 and then faded out. However, the present invention itself is not so limited. In the present specification, displaying information in a stream cycling manner is also referred to as display streaming of information. In one aspect, the information displayed in the main notification window 802 can be a more detailed version of the information displayed in the source summary window 806.

  Further, in one aspect of the invention, information streamed display in the main notification window 802 is journaled in the journal window 804 according to predetermined criteria. For example, when a particular portion of information is displayed in the main notification window 802, a one-line summary of that information is added to the journal window 804, commonly referred to herein as a journal entry, It is possible to display a list of such summaries. In one aspect, the list of windows 804 is scrollable by the user so that the user can examine substantially all information already streamed displayed in the main notification window 802.

  According to one aspect of the present invention, the predetermined criteria that limits the information that is journaled and / or added to the journal window 804 is that the user has indicated that it has not been viewed by the user. is there. For example, when the user performs a predetermined gesture such as moving the cursor on the main notification window 802, which is also called hovering, the information currently displayed in the main notification window 802 is browsed by the user himself / herself. Can be shown. It is also possible for the user to control a predetermined standard for journaling. In general, journals are used to capture a complete history of attempts to relay information to a user. Journal entries may include information about the source of information, high-level titles and / or summaries, and / or actions that may have been taken on notifications or alerts.

  Actions can be taken in response to predetermined user gestures related to information displayed in main notification window 802, source summary window 806 and / or journaled in journal window 804. is there. For example, a given user gesture may move the cursor over the main notification window 802, the source summary window 806, or select a journal entry to the journal window 804 and information displayed in the journal window 804. It is possible that there is. The selection can be made by the user clicking on an appropriate input device button, but the invention is not so limited. The action performed in response to this gesture is not limited by the present invention. However, in one aspect, the action includes displaying additional information, such as more detailed information related to the information that was the subject of the relevant gesture.

  The above example is shown in FIG. On display 900, the user shows in FIG. 20 as a pointer, but the present invention moves cursor 904 over source summary window 904 of source summary window 806 in particular so that it is not so limited in source summary window 904. Assume that you have selected the displayed information. The information source for window 904 is referred to as the user desired source. This is because this information source is the source from which the user made the relevant gesture. In response to the gesture, an action has been performed, specifically a window 906 is displayed that can contain more detailed information about the information displayed in the source summary window 904. Yes. The example of FIG. 20 is specific to a user making a gesture related to information displayed in one of the source summary windows 806, but the invention itself is not so limited, and the gesture is the main Note that it is also possible to relate to the information displayed in the notification window 802 or to the information of journal entries being journaled in the journal window 804.

  As can be appreciated, the stream stacking aspects of the present invention described above and described in connection with FIGS. 19 and 20 can be extended in various ways. For example, there may be a “simple mode” toggle in which one or more respective windows 802, 804, and 806 are displayed. Further, the number of source summary windows 806 can be increased or decreased by the user. Also, in one aspect, the information that is displayed in the source summary window 806 by minimizing the source summary window 806 is an icon representing an information source related to the window 806, and the user on a specific window among the windows 806 can display the information. Information generated by the corresponding source may be displayed by the cursor hovering performed.

  Referring now to FIG. 21, a method 1000 of the stream stacking aspect of the present invention is illustrated by a flow diagram. The method 1000 can include the stream stacking aspects described in connection with FIGS.

  At 1002, information from several sources is displayed. Information from each source is displayed in the corresponding source summary window. The information can be unrequested information that is not related to the user's primary task. At 1004, display streaming of information having importance that exceeds a threshold, such as a predetermined threshold, is performed in the main notification window. In one aspect, the information displayed in the main notification window can be more detailed than the information displayed in the source summary window corresponding to the source of the information. At 1006, information that has been display streamed in the main notification window is journaled into the journal window, such as by a journal entry that is added to the journal window according to predetermined criteria.

  As described above, the user can perform a predetermined user gesture on specific (user-desired) information to be displayed in any window of the source summary window, the main notification window, and / or the journal window. Actions relating to virtually any information that is journaled into can be performed. Accordingly, at 1008, such user gestures are detected for particular information that is displayed in either the source summary window, the main notification window, and / or has a journal entry in the journal window. In response to this detection, at 1010, an action related to this information is performed. For example, in one aspect of the present invention, a more detailed version of the information can be displayed.

  Various aspects of the invention described in the preceding sections of the description are also referred to as stream stacking. This is because information can be “streamed” in the main notification window and stacked in both the source summary window and the journal window. Thus, the user can know about important information by referring to the main notification window, and by referring to the corresponding journal entry for information in the journal window, the past displayed in the main notification window can be displayed. You can look up information. The user can also observe current information being generated by a given source, such as a notification source, by referring to the source summary window for the source. Information in the source summary window can be displayed regardless of importance, while more important information is generally displayed in the main notification window and journaled in the journal window.

  Further, in one aspect of the invention, high level summary information is associated with each source. For example, an email-related source displays information about the overall status of information from that source, there are 10 unread messages with a given priority, and the highest priority message It is possible to be from a specific user. In this case, by clicking or hovering over the source, a wider user interface, latest notifications, etc. for the source application can be displayed. In another aspect of the invention, such as an independent version of the stream cycling aspect of the invention described in the previous section of the description, information is streamed or cycled within each source display. In addition, in other aspects of the invention that include a larger main notification window, clicking on a specific source or selecting it differently displays details of the source information and focuses on this information. Be able to. Thus, subsequent selection of notifications may display further details of this information and / or a wider user interface for this source.

  The previous section of the description has described various modes in which information can be presented to the user, including pulse mode, stream cycling mode, and stream stacking mode. This section of the description provides further explanation on how the user can interact with the information presented through each mode. While the previous section of the description has described various user gestures and audio notifications, this section of the description provides a more detailed explanation as to how user interaction can occur.

  For example, a user gesture for communicating a request for additional information and a user gesture for responding to questions presented regarding possible links and services will be described. In one aspect, it may be a signal to the system that the user is hovering over the source in stream stacking mode to provide more detailed information regarding the summary. The detailed information can appear in the pop-up window as described above. Thus, in this aspect, the fact that the user is hovering the cursor over the window is used as an implicit request from the user for further details regarding the notification content. For example, if there is a weather report, cursor hovering is a way for the user to request further details about the weather, such as humidity, a 5-day forecast.

  Other gestures can also be detected according to the present invention. For example, the user can use various ways to select information by placing the cursor over the information being streamed and then clicking a button on a pointing device such as a mouse. . For example, selection of a Universal Resource Locator (URL) address provided in the display can, for example, access information referenced by that address. As another example, clicking within an unspecified area of an information display asking a question (eg, “Do you want to schedule it for you”) confirms the intention of the user receiving the service “Yes” It can be determined that the default, with nothing selected, on the other hand, is a “no” response.

  In addition, actions and timing to communicate with the application, notification manager, and / or providing evidence regarding the user's awareness of the notification are described. For example, a user gesture using an input device such as a keyboard or mouse may be provided by the user within a certain time after the notification appears to convey "Please let me know more about this notification". In response to the initial notification delivered to the user, such as shaking a pointing device such as a mouse or moving the cursor to a predefined corner of the display, It was possible to tell "Was there?", "Show me it again" or "Please let me know more about this". For example, if the notification was an audio report, such a user gesture (such as in the corner of the display) would be interpreted as a user asking "what was it" Information can be displayed in the notification window according to the mode.

  It is also possible to use gestures in human-computer interactions to collect information that indicates to the notification manager that the user has seen the notification or is more explicitly relayed to the notification manager. For example, as a way of indicating to the notification system that he has seen the notification, the user can hover the cursor over the notification during a certain time frame after the notification is displayed. Thus, in this case, the system can decide not to try again to relay the notification to the user. It is also possible that a more complex interaction, such as the user selecting a link that exists in the window, provides this indication.

  User interaction with a notification journal such as the journal described in the previous section of the description is described next. That is, as described above, the notification summary can be stored in the notification journal in the stream stacking mode of the present invention. Notification summaries can be organized by time, notification source, message classification, etc., allowing the user to revisit or review notifications that may have been missed previously. Thus, by selecting a journal entry, the user can redisplay the notification.

  In accordance with another aspect of the present invention, the use of audio to replace or enhance the display of information is described. For example, an audio notification can be used to announce a display of a notification that exceeds a threshold (such as a predetermined threshold) and can be further used to alert the user to the notification. In addition, different sounds can be associated with different types of notifications. For example, a scheduling related notification may have a different sound than an email related notification.

  Also, although this application has described the use of text and / or text and graphics to display information, it is noted that the invention is not limited to text and / or text and graphics. I want to be. For example, in one aspect, information is displayed in graphics, and different shapes and different colors can be utilized to indicate the nature and priority of the information. As another example, the closer the displayed object is to the center of the display, the more important the object is, and different color regions represent different sources of information. That is, the present invention is not limited to the specific conception of high-level graphics and / or text associated with information.

  An example of an alternative information display mode of the present invention is shown in the example diagram of FIG. According to this aspect of the present invention, information can be displayed, for example, within a predetermined area 302 of the desktop screen 300 depicted in FIG. Further, in one aspect, the user can switch between different modes including a scope mode. For example, the system can include a machine-readable medium that stores a computer program that is executed by a display, a processing system, and a processor to allow entry into any of the modes, such as scope mode. Furthermore, in one aspect, in addition to switching between modes by a user, the notification manager described in the previous section of the detailed description can make a determination to switch modes, for example.

  In the example scope interface depicted in FIG. 23, different shapes and different colors are used to indicate the nature and priority of information. For example, there may be a circular display object 1100 (eg, a wheel) in the corner of the screen that is divided into one or more parts. Note that other shapes are possible. Each part is a different color and can indicate a different type or source of information. For example, objects within each part of the display object 1100, such as circles, squares, arrows, lines, etc., are notifications from each part's information source, priorities, and / or events, and / or information types of each part. Can be expressed. The closer an object is to the center of the display object 1100, the more important it is, i.e., notifications, messages, and / or other types of information assigned importance values. Thus, in one aspect, concentric circles in the wheel define different priority levels. By hovering over an object with a cursor, text information about the object can be displayed. For example, hovering over a portion of the display object 1100 with a cursor (but not over an object in the display object) produces text information indicating the information type or information source for that portion of the display object 1100. be able to. The text information can be displayed as, for example, a tool tip.

  In order to provide context for various aspects of the present invention, FIG. 23 and the following description provide a brief general description of a suitable computing environment in which various aspects of the present invention may be implemented. Although the present invention has been described above in the general context of computer-executable instructions for a computer program executing on one or more computers, the invention may be implemented in combination with other program modules. This will be appreciated by engineers in the field. Generally, program modules include routines, programs, components, data structures, etc. that perform particular tasks and / or implement particular abstract data types. In addition, the method of the present invention includes single processor or multiprocessor computer systems, minicomputers, mainframe computers, as well as personal computers, handheld computer devices, microprocessor-based home appliances or programmable home appliances, and the like. Those skilled in the art will appreciate that other computer system configurations can be implemented. The aspect of the present invention can also be implemented in a distributed computing environment where tasks are performed by remote processing devices linked via a communication network. However, some, if not all, aspects of the invention can be implemented on a stand-alone computer. In a distributed computing environment, program modules can be located in both local and remote memory storage devices.

  With reference to FIG. 23, an exemplary system for implementing various aspects of the invention includes a processor 1221, a system memory 1222, and a system bus 1223 that couples various system components including the system memory to the processor 1221. Including a computer 1220. The processor 1221 can be any of various commercially available processors. It will be appreciated that dual microprocessors and other multiprocessor architectures may be used as the processor 1221.

  The system bus can be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of commercially available bus architectures. The system memory can include read only memory (ROM) 1224 and random access memory (RAM) 1225. A basic input / output system (BIOS) is stored in ROM 1224 that includes basic routines that help transfer information between elements within computer 1220, such as during startup.

  The computer 1220 reads from or writes to the hard disk drive 1227, for example, a magnetic disk drive 1228 that reads from or writes to a removable disk 1229, for example, a CD-ROM disk 1231, or other optical And an optical disk drive 1230 for reading or writing to the medium. The hard disk drive 1227, magnetic disk drive 1228, and optical disk drive 1230 are connected to the system bus 1223 by a hard disk drive interface 1232, a magnetic disk drive interface 1233, and an optical disk drive interface 1234, respectively. These drives and associated computer-readable media provide non-volatile storage for data, data structures, computer-executable instructions, etc. for computer 1220. The above description of computer readable media relates to hard disks, removable magnetic disks, and CDs, but other types of computer readable media such as magnetic cassettes, flash memory cards, digital video disks, Bernoulli cartridges, etc. Those skilled in the art can use in an exemplary operating environment and that any such media can include computer-executable instructions for performing the methods of the present invention. Will be understood.

  A number of program modules including operating system 1235, one or more application programs 1236, other program modules 1237, and program data 1238 may be stored in the drive and RAM 1225. Note that the operating system 1235 in the illustrated computer can be virtually any suitable operating system.

  A user can enter commands and information into the computer 1220 through a keyboard 1240 and a pointing device, such as a mouse 1242. Other input devices (not shown) can include a microphone, joystick, game pad, satellite dish, scanner, and the like. These and other input devices are often connected to the processor 1221 via a serial port interface 1246 coupled to the system bus, but may be a parallel port, game port, or universal serial bus (universal serial bus) ( It is also possible to connect via another interface such as USB). A monitor 1247 or other display device is also connected to the system bus 1223 via an interface, such as a video adapter 1248. In addition to the monitor, computers typically also include other peripheral output devices (not shown) such as speakers and printers.

  Computer 1220 can operate in a networked environment using logical connections to one or more remote computers, such as remote computer 1249. The remote computer 1249 can be a workstation, server computer, router, peer device, or other common network node and typically includes many or all of the elements described in connection with the computer 1220. However, only the memory storage device 1250 is shown in FIG. The logical connections depicted in FIG. 23 can include a local area network (LAN) 1251 and a wide area network (WAN) 1252. Such network environments are commonplace in offices, enterprise-wide computer networks, intranets, and the Internet.

  When used in a LAN network environment, the computer 1220 can be connected to the local network 1251 via a network interface, ie, an adapter 1253. When utilized in a WAN network environment, the computer 1220 can typically include a modem 1254 and / or be connected to a communication server on a LAN and / or communicate via a wide area network 1252 such as the Internet. Have other means for connecting. A modem 1254, which can be internal or external, can be connected to the system bus 1223 via a serial port interface 1246. In a networked environment, program modules or portions of program modules drawn with respect to computer 1220 can be stored in a remote memory storage device. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers can be used.

  In accordance with the practice of those skilled in the computer programming arts, unless otherwise stated, the invention has been described with reference to actions and symbolic representations of operations that are performed by a computer, such as computer 1220. Such treatments and actions are sometimes referred to as computer-implemented treatments and actions. Actions represented by actions and symbols include manipulation by the processor 1221 of electrical signals representing data bits, resulting in conversion or reduction of the electrical signal representation, and memory systems (system memory 1222, hard disk 1227, floppy (registered). Including data bits in memory locations (including trademarked disk 1229 and CD-ROM 1231) to reconfigure or otherwise modify the operation of the computer system and other signal processing It will be understood. A memory location where such data bits are held is a physical location having specific electrical, magnetic, or optical properties corresponding to the data bits.

  What have been described above are various aspects of the present invention. Of course, to illustrate the invention, it is not possible to describe every possible combination of components or methods, but many additional combinations and modifications of the invention are possible in the art. It will be recognized by the person. Accordingly, the present invention is intended to embrace all such alterations, modifications and variations that fall within the scope of the appended claims.

(Industrial applicability)
The present invention has industrial applicability in the fields of computers, computer software, and information technology.

  1 is a schematic block diagram of a system illustrating a notification platform architecture according to an aspect of the present invention.   FIG. 6 is a schematic block diagram illustrating a context analyzer according to an aspect of the present invention.   FIG. 6 is a schematic block diagram illustrating a notification source and a notification sink according to an aspect of the present invention.   FIG. 6 illustrates the use of a notification curve according to one aspect of the present invention.   FIG. 6 illustrates a user specified interface for notification according to an aspect of the present invention.   FIG. 6 illustrates a context information source according to an aspect of the present invention.   1 illustrates a rule-based system for determining context according to an aspect of the present invention. FIG.   1 is a schematic block diagram illustrating an estimation-based system for determining context, according to one aspect of the invention. FIG.   FIG. 6 illustrates an estimation model for determining context according to an aspect of the present invention.   FIG. 4 illustrates a temporal estimation model for determining context according to an aspect of the present invention.   3 is a flow diagram illustrating a method for determining context, according to one aspect of the invention.   3 is a flow diagram illustrating a method for notification decision making according to one aspect of the present invention.   6 is a flow diagram illustrating a method for providing a decision-theoretic analysis of a notification platform according to an aspect of the present invention.   FIG. 6 illustrates an exemplary display according to one aspect of the present invention.   6 is a flow diagram illustrating a method for providing various displays according to an aspect of the present invention.   FIG. 6 illustrates value versus time according to one aspect of the present invention.   4 is a flow diagram illustrating a method for providing stream cycling in accordance with an aspect of the present invention. FIG. 4 illustrates an exemplary stream cycling display according to an aspect of the present invention.   FIG. 6 illustrates an exemplary stream stacking display according to an aspect of the present invention.   FIG. 6 is a more detailed diagram illustrating an exemplary stream stacking display according to an aspect of the present invention.   4 is a flow diagram illustrating a method for providing stream stacking according to an aspect of the present invention.   FIG. 6 illustrates an exemplary display according to an alternative aspect of the present invention.   FIG. 2 is a schematic block diagram illustrating a suitable operating environment, according to one aspect of the invention.

Claims (13)

A context analyzer configured to access and store information about context information and notification parameters, wherein the notification parameters are related parameters indicating the likelihood that the information contained in the notification is related to the context, and the information is new to the user A mechanism comprising at least one of a novelty parameter indicating whether or not and a fidelity parameter indicating loss of value of information by cutting and / or summarizing information;
At least one notification source, each source configured to generate a notification;
At least one notification sink, each sink receiving a notification, and a) the transmission reliability of the notification sink indicating the likelihood that the user will receive the information contained in the notification conveyed to the notification sink;
b) the communication cost of the notification sink indicating the communication cost that the user incurs when receiving information contained in the notification conveyed to the notification sink;
c) the interruption cost of the notification sink indicating the interruption cost that the user incurs due to the interruption caused by the notification when receiving information contained in the notification conveyed to the notification sink;
A notification sink configured to have an associated parameter indicating at least one of
A notification manager configured to determine and communicate which notification generated by a notification source is communicated to which notification sink based on the context information stored in the context analyzer and information about the notification parameters; With
The notification manager determines a net value of notifying a user based on a parameter of the notification sink, and transmits the notification when the net value is larger than a predetermined threshold. System.
The system of claim 1 , wherein the context analyzer includes a notification parameter store that stores default notification preferences for the user as a profile.
The system of claim 1 , wherein the context analyzer determines a user's current context based on at least one context source.
The at least one context source includes one or more of the user's personal information management (PIM) information, current time and date, the user's mobile device usage, and the user's location. The system according to claim 3 .
The context analyzer uses one or more of a statistical model, a profile, direct access to the user's location and activity, and a user state real-time user designation indicating that the user can only respond to important notifications 4. The system of claim 3 , wherein the system determines the current context based on the at least one context source.
The notifications generated by the at least one notification source may include email, instant messaging, system messages, automated assistance, updates to the number or content of documents available in a shared file system, updates to organizational information, specific content Including one or more of: information available on a document, topic or topic, Internet-related information and news services, human availability, location, proximity, scheduling queries, company and organization proximity and location The system according to claim 1 .
The at least one notification source is
A classification of the current notification generated by the notification source indicating the value of the information contained in the current notification;
Wherein indicating the attenuation of time-dependent of the value of the information contained in the current notification, representative of at least one of the time critical of the generated said current notified by the notification source, associated with one or more parameters The system of claim 1 , wherein:
The at least one notification source is
The current notification message class generated by the notification source, indicating a type of communication for the current notification;
The association of the current notification indicating the likelihood of one or more associations of a context configured to relate to information included in the current notification;
The novelty of the current notification indicating the likelihood that an entity already knows the information;
The system of claim 1 , wherein the system is associated with a parameter representing at least one of the current notification fidelity indicating a loss of value for the entity of the information upon truncation of the information. .
The system of claim 1 , wherein the at least one notification source includes at least one of a pull type notification source and a push type notification source.
The system of claim 1 , wherein the at least one notification sink includes one or more of a desktop computer, a PDA, a mobile phone, a pager, and an automotive computerized device.
In order to determine which of the notifications from the at least one notification source should be communicated to which of the at least one notification sink, the notification manager is based on the information stored by the context analyzer. 2. The system of claim 1, performing a decision-theoretic analysis of the notifications from the at least one notification source.
In order to determine which of the notifications from the at least one notification source should be communicated to which of the at least one notification sink, the notification manager is based on the information stored by the context analyzer. The system of claim 1, wherein heuristic analysis of the notification from the at least one notification source is performed.
One or more notification sources generate the intended notification or notifications;
A notification manager receives one or more notifications and determines a context for the one or more notifications;
The user mechanism stores information about the context information and the user's notification parameters, the notification parameters including a related parameter indicating the likelihood that the information contained in the notification is related to the context, and whether the information is new to the user. At least one of a novelty parameter that indicates or a fidelity parameter that indicates a loss of value of information by cutting and / or summarizing the information;
A notification manager determines which notifications are delivered to which one of the one or more notification sinks based on the context information stored in the mechanism and information about the notification parameters;
Each notification sink
a) the transmission reliability of the notification sink indicating the likelihood that the user will receive the information contained in the notification conveyed to the notification sink;
b) the communication cost of the notification sink indicating the communication cost that the user incurs when receiving information contained in the notification conveyed to the notification sink;
c) the interruption cost of the notification sink indicating the interruption cost that the user incurs due to the interruption caused by the notification when receiving information contained in the notification conveyed to the notification sink;
Configured with an associated parameter indicating at least one of
Including a notification manager determining a net value of notifying a user based on a parameter of the notification sink and transmitting the notification when the net value is greater than a predetermined threshold. Feature method.

Images