US20170039598A1

US20170039598A1 - Remote prompting infrastructure

Info

Publication number: US20170039598A1
Application number: US15/233,938
Authority: US
Inventors: Devon Welles; Chris Gough
Original assignee: Intel Corp
Current assignee: Intel Corp
Priority date: 2006-09-29
Filing date: 2016-08-10
Publication date: 2017-02-09
Also published as: US20080082404A1; US20200202387A1; US9436931B2

Abstract

The invention relates to a method and apparatus for sending and receiving prompts to end-users inside and outside the home. A prompt, for example, a message, image, or sound is presented to the end user in order to notify them of a health event, serve as a simple reminder, helps them through their daily activities. The invention includes, for example, the following components: a remote prompting client which runs on the end-user's home network and is typically associated with a physical display device. This entity has the ability to receive a prompt request from a remote prompting host, and display the prompt to the end-user; and a remote prompting host which runs on the end-user's home network and has the ability to scan the network and discover all existing remote prompting clients.

Description

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. application Ser. No. 11/529,575 filed Sep. 29, 2006, which is hereby incorporated by reference in its entirety.

FIELD OF INVENTION

The invention relates to a method and apparatus for sending and receiving prompts to end-users, and in particular, to end-users located in a home.

BACKGROUND

Currently, many elder adults rely on a variety of low-tech methods to remember important events such as handwritten messages posted throughout the home. Concerned family members must visit/phone frequently to gather this type of information which is problematic for the elder adult (they don't want to be a burden on their family members) as well as the family caregiver (feelings of guilt for not calling/visiting often enough).
Personal health has been targeted as one of the major growth areas for the foreseeable future. Elder care is likely to be one of the primary (future) use cases in this area. A remote prompting infrastructure interoperable with a wide variety of devices that can inform elder adults and remote caregivers of critical/non-critical health events is going to be a key building block that will enable elder care related solutions in the home.
Currently there is no advertising tied into the prompting systems out there. By associating a relevant ad with a specific prompt to each individual user, the effectiveness of the ads is increased. Also, since the ad is tied to a prompt that the user needs to see, then the ad will be seen too.
There are ads that are associated with gross classifications like television audience, and there are targeted ads that are associated with finer classifications like web-browsing choices, but neither of these are delivered in conjunction with a prompt that the user needs to comprehend and respond to, nor do they guarantee that the user will see them.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a high-level platform architecture.

FIG. 2 illustrates an exemplary Transducer Access Point architecture.

FIG. 3 illustrates the inferred services architecture of the system.

FIG. 4 illustrates the device manager architecture of the system.

FIG. 5 illustrates an exemplary display setting for a graphical user interface.

FIG. 6 illustrates an exemplary history for a graphical user interface.

FIG. 7 illustrates an exemplary remote prompting infrastructure in accordance with the invention.

FIG. 8 is a flow diagram of pairing an advertisement with a remote prompting in accordance with the invention.

DETAILED DESCRIPTION

The embodiment of the invention relates to a system for sending prompts to a user in a network including at least one remote prompting client to receive a prompt and notify the user of an event and a remote prompting host to scan the network to detect the at least one remote prompting client, wherein the remote prompting host identifies an event of the user and sends the prompt to the detected at least one remote prompting client. Preferably, the remote prompting host is hardware and/or software running on a computer which interfaces with sensors located throughout the network to determine the event of the user. Preferably, the remote prompting client is a module on the user's network associated with at least one of a network display device, television, phone, cell phone, picture frame display, and mirror. According to a further embodiment, the prompt is at least one of a message, image and sound. Preferably, an advertisement is paired with the prompt and/or the advertisement is loaded to the remote prompting host. According to yet another embodiment, the remote prompting host determines when the advertisement is paired and sent with the prompt based on the event, sends the advertisement with the prompt, and notifies a central server responsible for the advertisement that the advertisement was sent and displayed at the user's network.
According to yet another embodiment, a method is provided for receiving a prompt at a remote prompting client and notifying the user of an event and scanning the network at a remote prompting host to detect the at least one remote prompting client, wherein the remote prompting host identifies an event of the user and sends the prompt to the detected at least one remote prompting client. Preferably, the remote prompting host is hardware and/or software running on a computer which interfaces with sensors located throughout the network to determine the event of the user. Preferably, the remote prompting client is a module on the user's network associated with at least one of a television, phone, cell phone, picture frame display, and mirror. According to a further embodiment, the prompt is at least one of a message, image and sound and/or an advertisement is paired with the prompt. In another embodiment, the advertisement may be loaded to the remote prompting host.
According to this embodiment, the remote prompting host determines when the advertisement is paired and sent with the prompt based on the event, sends the advertisement with the prompt, and notifies a central server responsible for the advertisement that the advertisement was sent and displayed at the user's network.
According to a further embodiment, a system is provided for monitoring a user in a network, the system including a sensor/actuator network to determine when an event occurs in the system, a computing device to monitor and prompt the system when an event occurs, the computing device including an inference engine and a decision manager, a prompting device to receive prompts from the computing device and for displaying the prompt to the user, and a gateway or transducer access point to bridge the sensor/actuator network with the computing device.
Preferably, when the sensor/actuator network determines that an event has occurred, sending a response to the computing device. According to this embodiment, when the computing device receives the response, the inference engine analyzes the response and the decision manager initiates actuation of the associated prompting device. Preferably, the inference engine collects and interprets the response in order to analyze data sent from the sensor/actuator, and the decision manager initiates actuation of the associated prompting device based on rules defined by the system.
FIG. 1 illustrates high-level platform architecture. The “left-side” of the diagram is characterized by low-power, in some cases proprietary protocols, used by the sensors and actuators to handle communications. The “right-side” of the diagram represents the Digital Home, where WiFi, Ethernet, and UPnP are the communication technologies of choice The “Healthy Digital Home” is enabled (in part) by the Transducer Access Point (TAP), which bridges the various sensor/actuator networks (specifically Bluetooth and Z-Wave in this case), to WiFi/Ethernet, and exposes the various physical devices “on the left” as UPnP devices “on the right” such that intelligent control points can perform actions, request state, and subscribe to event notifications.
UPnP is an architecture for pervasive peer-to-peer network connectivity of intelligent appliances, wireless devices, and PCs. It is designed to bring easy-to-use, flexible, standards-based connectivity to ad-hoc or unmanaged networks whether in the home, in a small business, public spaces, or attached to the Internet. UPnP is a distributed, open networking architecture that leverages TCP/IP and the Web technologies to enable seamless proximity networking in addition to control and data transfer among networked devices in the home, office, and public spaces.
UPnP is more than just a simple extension of the plug and play peripheral model. It is designed to support zero-configuration, “invisible” networking, and automatic discovery for a breadth of device categories from a wide range of vendors. This means a device can dynamically join a network, obtain an IP address, convey its capabilities, and learn about the presence and capabilities of other devices.
Protocols supported by the TAP include, for example, Bluetooth and Z-Wave. Obviously, any protocol may be used as would be appreciated by the skilled artisan. The protocols tend to be flexible, extensible building blocks that allows radio protocols such as these to communicate with other devices inside and outside the home.
The following sensors and actuators are examples of devices that may be integrated into the platform: Scales, Blood Pressure Monitors, Pressure Sensors, Presence Sensors, Multi-Level Light, Television, Personal Computers, Bathroom Mirror Displays, Picture Frame Displays, Cell Phones, PDAs, etc. Again, this list of devices is exemplary and not exclusive. Due to limitations exhibited by typical wireless sensors (e.g. power profile, transmission range, memory capacity, cost, etc.) a TAP is used to allow communication to take place, as described above. FIG. 2 illustrates an exemplary TAP Architecture. The TAP is intended to serve as a proxy for the low-powered (Bluetooth and Z-Wave) transducers.
An Inference Engine is the computational entity responsible for computing and understanding the data collected from the home sensor network. Essentially, sensor data is collected, and compared against a probabilistic model in order to determine the events that the data represents. Based on the recognized events, the decision manager may initiate local and/or remote actuation. For example; the probabilistic model may determine, based on data collected from multiple sensors (e.g., bed pressure sensor, motion sensors, bedroom door contact switch, etc.) that the occupant is in bed.
In FIG. 3, a Sensor Interface is the block responsible for communicating with devices exposed by the TAP. This “raw” sensor data is collected and interpreted by the inference engine. The external interface to the inference engine is referred to as Inferred Services. Inferred Services are exposed as devices and are indistinguishable from their physical counterparts on the home network. In this way, Inferred Services can be thought of as “virtual sensors” that leverage the processing power of a computing device to increase the accuracy associated with the data they are transmitting, as well as collating disparate information from various sensors to represent higher level events. For example: The Activity Device depicted in FIG. 3 may utilize data from a multitude of sensors (e.g., motion, light, contact switches, RED, time of day, etc.) to determine that the home occupant is making breakfast. Interested control points may subscribe to the “making breakfast” event in the same manner they would use to communicate with a specific physical sensor device.
A Decision Manager, illustrated in FIG. 4, is an example of a general control point that subscribes to the Inferred Services. Based on event notifications received from the Inferred Services, and rules defined by users of the system, actuation is initiated either inside or outside the home environment. The Inferred Services Control Points block is the interface into the Inferred Services. In this example, both the Inferred Services and Decision Manager reside in the same computing device, but this is not a requirement of the system (nor is having one distinct set of Inferred Services and one distinct Device Manager). One characteristic about the system is the following: If the underlying set of physical sensors change, while the events subscribed to may be more (or less) accurate, there is no need for the Decision Manager to change.
The information retrieved from Inferred Services in conjunction with user-defined rules will be used within the framework of the platform to construct a state machine representing the activities taking place in the home. The following is an example of the Decision Manager in action:

- 1. Remote family caregiver accesses rule editor via web interface;
- 2. RFC creates a rule that specifies the following: If primary occupant is in the bathroom for over 2 hours, send SMS message to my cell phone;
- 3. Occupant enters bathroom (detected by motion sensor and/or weight scale measurement); and
- 4. 2 hours elapse (an unusually long time for the home occupant to remain in the bathroom), state machine raises alert to Rule Execution Engine, and SMS message is sent via actuator interface alerting the remote caregiver of a possible fall.

For the purposes of this example, the following types of actuation will be supported by the Decision Manager's Actuator Interface:

- Prompt, including text and color (to designate severity) overlaid on:
- Television
- Picture Frame
- Bathroom Mirror Display (in addition to ‘text’, the bathroom mirror will have a more sophisticated display combining the most recent weight scale measurement and a graphical depiction of measurement history).
- Cell Phone (SMS message)
- (Z-Wave) Light switch on/off (to show communication backchannel through the TAP).

It is readily understood that the above-mentioned types of actuation is not limiting, but rather exemplary.
FIGS. 5 and 6 are exemplary user interfaces. FIG. 5, for example, illustrates a display setting for a graphical user interface, while FIG. 6 illustrates the history of a graphical user interface.
There exist many reminder or prompting systems on computers or entertainment systems. Some examples are calendar applications that remind you to go to a meeting or appointment, or a digital video recorder that prompts you to change the channel to a show that's on.
Future prompting systems may be more intelligent and display prompts for users on one of many displays in the house based on the nature of the prompt or the location of the person. Such a system would have knowledge of the nature of the prompt (medical, entertainment, etc.) and the location of the person in their home.
This invention, in another exemplary embodiment, may also use the remote prompting infrastructure to deliver a targeted ad to an individual that is guaranteed to be seen because it is tied in with the prompt. The ad and prompt can be shown continuously until the prompt is acknowledged. Also, since the remote prompting infrastructure will have some knowledge of the user, advertisers can filter the type of individual they'd like to target by age, hobbies, or location.
Some Examples:

- A prompt pops up on the TV reminding a person that they need to refill their prescriptions. At the same time an ad is displayed for a local pharmacy at which the prescription can be filled.
- A graph is displayed on the bathroom mirror of a person's weight and blood pressure. Shortly afterwards the mirror displays an ad for a local health food store.
- A reminder comes up for a person to go to a doctor's appointment to see how their medications are working. At the same time an ad is displayed for a competing medication with different side effects or perhaps a generic brand that is cheaper. This way the person has the new medication fresh in their mind for the appointment.

FIG. 7 illustrates an exemplary remote prompting infrastructure. This infrastructure, in use with the above described system, discloses a method and apparatus for sending and receiving prompts to end-users inside and outside the home where prompt is defined as; a message, image, or sound (or some combination of the three) presented to the end user in order to; notify them of a health event, serve as a simple reminder, help them through their daily activities (e.g. an elder adult with mild cognitive decline). The invention includes the following components:
Remote Prompting Client: This is a software and/or hardware module that is running on the end-user's home network and are typically associated with a physical display device (e.g. television, cell phone, picture frame display, bathroom mirror with integrated TV, etc.). This entity has the ability to receive a prompt request from a Remote Prompting Host, and display (either visually, audibly or by touch) the prompt to the end-user.
Remote Prompting Host: This is a software and/or hardware module that is running on the end-user's home network and has the ability to scan the network and discover all existing Remote Prompting Clients. The Remote Prompting Host performs two critical functions:

- Interface to sensor/medical device network to determine where the end-user is and what they are doing
- Send prompt to appropriate Remote Prompting Client to notify the local/remote user of critical or non-critical health events, reminders, Activities of Daily Living (ADL) assistance, etc.

Example:

- Agnes is an elder adult with type II diabetes.
- Agnes hasn't measured her weight or blood pressure in several days.
- Because her weight scale and blood pressure monitor are both connected to her personal health system, the system is aware of this.
- Agnes is sitting on her couch watching TV.
- The system uses a pressure sensor in the couch and power clamp on the television to recognize this activity.
- The Remote Prompting Host via its interface to the sensor network knows that Agnes has not measured her weight or blood pressure recently and that she is watching TV.
- The Remote Prompting Host sends a prompt to the Remote Prompting Client associated with the television and a message/icon is overlaid on top of the TV signal informing Agnes that she needs to measure her weight and blood pressure, for example using a network laptop connected to the TV, or an IPTV or living room PC.
- Agnes presses a button on her remote control to clear the prompt; the system is now aware that she has seen it.
- Agnes measures her weight and blood pressure during the next commercial (otherwise, she could/would be prompted again at some point in the future).
- Agnes has lost a significant amount of weight since her last weight measurement.
- The Remote Prompting Host sends a prompt to the Remote Prompting Client associated with Agnes' daughter's cell phone. An SMS message is sent to her daughter's cell phone (via an Internet based SMS gateway service) informing her daughter of this potentially dangerous health event.

The remote prompting is part of a larger platform (i.e. architecture) that is responsible for, in addition to the remote prompting, all other aspects associated therewith.
FIG. 8 is a flow diagram of pairing an advertisement with a remote prompting in accordance with the invention. This invention would ensure that advertisements are hitting a targeted and relevant market by pairing the ad with a prompt that a person receives through a remote prompting infrastructure. Since the prompt would necessarily be specific to the person (take your medication, etc), an appropriate ad to that same person would have a much higher likelihood of success. It also guarantees that the ad is seen by the target individual. This method also can protect the privacy of the people receiving the ads from the advertisers.
This invention also increases the attractiveness of remote prompting infrastructures to service providers by creating another revenue stream.
It is readily understood by the skilled artisan that the embodiments disclosed herein are merely exemplary and are not intended to limit the scope of the invention.

Claims

1. A system comprising:

a sensor configured to collect data related to an event associated with a user, wherein the sensor is configured to support a wireless protocol operational in sub-gigahertz frequency range providing a data rate of up to 100 kbps; and

a remote prompting host device configured to:

receive, from the sensor, event data based on the data collected by the sensor using the wireless protocol,

identify the event based on the received event data, and

generate and provide to one or more client devices a prompt related to the identified event, wherein the prompt includes information about the identified event and an action responsive to the identified event.

2. The system of claim 1, further comprising a bridge device to bridge wireless communication based at least on the wireless protocol supported by the sensor and communication based at least on a communication protocol supported by the remote prompting host device.

3. The system of claim 2, wherein the communication protocol supported by the remote prompting host device includes Ethernet.

4. The system of claim 2, wherein the bridge device includes a Transducer Access Point (TAP) device configured to bridge communication between the sensor and the remote prompting host device, wherein the TAP device is configured to support communication based on the wireless protocol operational in sub-gigahertz frequency range providing a data rate of up to 100 kbps and Ethernet.

5. The system of claim 1, wherein the remote prompting host device is configured to provide a notification to a first client device indicating that the prompt associated with the identified event is received at a second client device.

6. The system of claim 1, wherein the one or more client devices include one or more of a network display device, a television, a phone, a cell phone, a picture frame display, and a mirror.

7. The system of claim 1, wherein the prompt is at least one of a message, an image and a sound.

8. The system of claim 1, wherein the remote prompting host device is configured to:

pair an advertisement with the prompt based on the event,

send the advertisement with the prompt, and

notify a central server responsible for the advertisement that the advertisement was sent and displayed at a client device.

9. A method comprising:

receiving data related to an event associated with a user, wherein the data is collected based on a wireless protocol operational in sub-gigahertz frequency range providing a data rate of up to 100 kbps; and

identifying the event based on the received data, and

providing, to one or more client devices, a prompt related to the identified event, wherein the prompt includes information about the identified event and an action responsive to the identified event.

10. The method of claim 9, wherein the receiving comprises receiving data from a bridge device that is configured to bridge wireless communication based at least on the wireless protocol and communication based at least on Ethernet.

11. The method of claim 10, further comprising providing a notification to a first client device indicating that the prompt associated with the identified event is received at a second client device.

12. The method of claim 9, wherein the one or more client devices include one or more of a network display device, a television, a phone, a cell phone, a picture frame display, and a mirror.

13. The method of claim 9, wherein the prompt is at least one of a message, an image and a sound.

14. The method of claim 9, further comprising:

pairing an advertisement with the prompt based on the event,

sending the advertisement with the prompt, and

notifying a central server responsible for the advertisement that the advertisement was sent and displayed at a client device.