WO2014105782A1 - Activation d'événement/application dépendant du contexte - Google Patents

Activation d'événement/application dépendant du contexte Download PDF

Info

Publication number
WO2014105782A1
WO2014105782A1 PCT/US2013/077400 US2013077400W WO2014105782A1 WO 2014105782 A1 WO2014105782 A1 WO 2014105782A1 US 2013077400 W US2013077400 W US 2013077400W WO 2014105782 A1 WO2014105782 A1 WO 2014105782A1
Authority
WO
WIPO (PCT)
Prior art keywords
computer
patient
portal
display
patient device
Prior art date
Application number
PCT/US2013/077400
Other languages
English (en)
Inventor
Jean-Claude Junqua
Ricardo Teixeira
Mike Wang
David Koo
Original Assignee
Panasonic Corporation Of North America
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US13/729,960 external-priority patent/US9208661B2/en
Priority claimed from US13/730,327 external-priority patent/US8803690B2/en
Priority claimed from US14/096,475 external-priority patent/US9335904B2/en
Application filed by Panasonic Corporation Of North America filed Critical Panasonic Corporation Of North America
Priority to JP2015550716A priority Critical patent/JP2016512623A/ja
Publication of WO2014105782A1 publication Critical patent/WO2014105782A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces
    • G06F9/452—Remote windowing, e.g. X-Window System, desktop virtualisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/10—Office automation; Time management
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/40—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the management of medical equipment or devices, e.g. scheduling maintenance or upgrades
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
    • G16H40/67—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for remote operation
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H80/00—ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring

Definitions

  • This disclosure relates to a computer-implemented system for assisting persons of reduced cognitive or physical ability to manage upcoming events and to stay in touch with loved ones.
  • the present disclosure provides a solution to the aforementioned problems in the form of an integrated patient device (e.g., portable device) and network-based portal combination.
  • the patient device is designed for use by patients of reduced cognitive or physical ability.
  • the patient device integrates with a network-based portal, designed for use by family members or nursing staff to customize the user experience of the patient. Using the portal the family member or nursing staff person pushes user interface customizations as well as content and applications to be run on the patient device.
  • the family member or nursing staff can also control the patient device remotely via the portal.
  • the patient device collects usage data on the set of customizable interface components, and on the pushed content and applications. These data are sent as feedback to the portal, allowing the portal to show the family member or nursing staff which user interface features, content and applications are being used and which are not. Aggregate data collected via the portal generate ranking metrics used to aid in user interface feature selection.
  • the portal includes an integrated database or kit of user interface (Ul) components from which the family member or nursing staff may select to construct a customized user interface and user experience specially designed for a particular patient.
  • each Ul component Stored within the integrated database, each Ul component has at least one associated cognitive ability or physical ability metric that ranks the understandability and usability of that Ul component for that particular patient.
  • the family member wishes to customize the user interface of a particular application, or even the global operation of the patient device itself, he or she can compare these cognitive ability metrics and physical ability metrics of each Ul component and select the ones that will best work for the specific patient.
  • the portal computer is also programmed to share usage statistics with a cloud-based aggregation computer that generates aggregate complexity statistics and metrics representing an entire population of patents who are using the patient device. Preferably these usage statistics are shared with the cloud-based aggregation computer without revealing any patient- specific information to ensure privacy of the patient is protected.
  • the integrated database may also be configured to store similar suitability metrics in association with various different selections of content pushed to the device.
  • suitability metrics in association with various different selections of content pushed to the device.
  • that usage data reflects the suitability level of the content for this particular patient is low and a low suitability metric is thus assigned to that content.
  • This low metric shows the family member or nursing staff that the content selected to be pushed to the patient device may be too "difficult" or uninteresting to the patient.
  • the integrated database may also be configured to store similar suitability metrics in association with various different selections of applications pushed to the device to be run on the device. For example, if an instant messaging application is pushed to the patient device, but usage metrics indicate that this application is never used, a low suitability metric is assigned. This low metric shows the family member or nursing staff that the instant messaging application selected to be pushed to the patient device may be too "difficult" or uninteresting for this particular patient.
  • the disclosure describes a computer-implemented system for assisting persons of reduced cognitive or physical ability comprising.
  • the system includes a patient device having a display, a processor coupled to the display, and a communication port through which it communicates with a portal computer.
  • the processor of the patient device is programmed to provide feedback data to the portal computer regarding use by the patient of the selected user interface component pushed to the patient device.
  • the portal computer has a processor and associated memory storing a plurality of user interface components according to a predefined data structure that associates an ability metric with each user interface component.
  • the portal computer is programmed to present the plurality of user interface components to a first user of the portal computer in a presentation arrangement based on the ability metric.
  • the portal computer is further programmed to allow the first user to select from the arranged presentation of user interface components at least one interface component and then to push said selected user interface component to the patient device.
  • Figure 1 shows a general system architecture showing users, display devices, a server-based system, and a network.
  • Figure 2 shows an example messaging seen on a display device.
  • Figure 3 shows an example messaging seen on a display device after a reminder was acknowledged.
  • Figure 4 shows examples of alternative approaches for where to place system logic within the overall system.
  • Figure 5 illustrates exemplary database elements used by the system.
  • Figure 6 provides an example of a simplified flow of logic for the display device.
  • Figure 7 shows the relationship of group messages and different types of users for a particular display device.
  • Figure 8 shows an example process for setting up a relationship between group and master users for a display device.
  • Figure 9 illustrates a subset of a system for illustrating reminder acknowledgement.
  • Figure 10 shows an exemplary hardware block diagram for a display device.
  • Figure 1 1 illustrates an example user interface for creating and/or editing reminder messages.
  • Figure 12 illustrates an example user interface as seen by a master user for reviewing all active reminders and messages for a particular display device.
  • Figure 13 illustrates an example user interface as seen by a regular user for reviewing all active reminders and messages for a particular display device.
  • Figure 14 illustrates an example of a similar user interface formatted for smart phones and other mobile devices.
  • Figure 15 illustrates an example user interface as seen by a group user for viewing group reminders and messages.
  • Figure 16 illustrates an example user interface for creating or editing an instant message.
  • Figure 17 illustrates an example user interface for managing parameters for a particular display device.
  • Figure 18 illustrates an example user interface showing existing preset reminders.
  • Figure 19 illustrates an example user interface for selecting a preset reminder.
  • Figure 20 is a diagram for showing how a display device's health can be monitored.
  • Figure 21 depicts the display and master user setup.
  • Figure 22 depicts an example display device showing a few example reminder messages.
  • Figure 23 is an entity diagram illustrating basic components of how an event or application is launched automatically using the disclosed system.
  • Figure 24 is a high level flowchart diagram illustrating how cognitive ability factors into the launching of an event or application.
  • Figure 25 is a flowchart depicting how context is gathered and used by various components within the system.
  • Figure 26 is a flowchart illustrating the trigger event flow implemented by the system.
  • Figure 27 is a flowchart illustrating the event launch flow implemented by the system.
  • Figure 28 is a use case diagram showing an exemplary use of the system.
  • Figure 29 is an interaction diagram showing how the interaction level of the system is customized based on cognitive ability and based on preferences and technology context information.
  • Figure 30 is a diagram showing how cognitive ability is modeled by the system, as reflected in the cognitive ability data structure maintained in computer memory by the system.
  • Figure 31 is a block diagram showing one tablet-based, web- enabled system embodiment.
  • Figure 32 shows an example screen display with several exemplary applications/events launched.
  • Figure 33 is a block diagram showing the computer-implemented system and its associated database and data structures.
  • Figure 34 is a block diagram showing how the patient device and the network-based portal computer are integrated.
  • Figure 35 is an exemplary display screen of the patient device featuring an exemplary calendar application.
  • Figures 36a and 36b are user interface and data structure diagrams showing how customizations are pushed to the patient device and how usage data are collected.
  • Figures 37a and 37b are comparative displays produced by the portal computer, showing how usage feedback from the patient device may be displayed.
  • Figure 38 is an exemplary display screen generated by the portal computer with which settings of the patient device are customized.
  • Figure 39 is an exemplary display screen generated by the portal computer with which settings of the patient device are customized and applications to be pushed to the patient device are selected.
  • Figure 40 is an exemplary display screen generated by the portal computer with which settings of applications (e.g., music and video applications) for the patient device are customized to add selected digital content.
  • applications e.g., music and video applications
  • Figure 41 is an exemplary display screen generated by the portal computer with which settings of an audio-video chat application (e.g., Skype) for the patient device are customized from the portal computer.
  • an audio-video chat application e.g., Skype
  • Figures 42a and 42b illustrate two different embodiments for displaying ability metrics in association with applications or Ul components, the display being presented on the display of the portal computer for use in customizing the patient experience when using the patient device.
  • Figure 43 illustrates the software architecture of the patient device.
  • Figure 44 illustrates the software architecture of the portal device, showing how the portal device communicates with the API of the patient device through messaging.
  • Figure 45 illustrates the software architecture of the portal device, showing how crash reporting is handled.
  • Figure 46 illustrates the software architecture of the portal device, showing the updater broadcast receiver.
  • Figure 47 illustrates the software architecture of the portal device, showing the wrapper activity start.
  • Figure 48 illustrates the software architecture of the portal device, showing the wrapper activity downloader.
  • Figure 49 illustrates the software architecture of the portal device, showing the wrapper activity crash.
  • Figure 50 illustrates the software architecture of the portal device, showing the bundle settings update.
  • the disclosed system allows people (e.g., friends, family, administrators) in a remote or local location to create reminder messages that will show at the appropriate times and with appropriate messaging on a relatively simple display device.
  • This display device need not have any controls that the viewer interacts with, so a person with Alzheimer's does not need to learn how to operate it. The only interaction that this display device needs happens during a one-time initial setup step, and optional reminder acknowledgements that require only the press of one button.
  • the system works via a network, such as the Internet and/or local area network (LAN). People (friends, family, administrators) interface to the system via any modern browser. The system, in turn, interacts with the display device via the network.
  • a network such as the Internet and/or local area network (LAN).
  • People friends, family, administrators
  • the system in turn, interacts with the display device via the network.
  • the system accommodates multiple display devices and multiple accounts. More than one person can be given the ability to create a reminder message. A master account(s) is also given the ability to edit messages from other accounts, as well as other privileges. For situations such as an assisted living home, a group administrator account can send messages to groups of display devices, or to just one display device. However, accounts that are associated directly with a particular display device can hide such group messages if needed.
  • Account holders associated with a particular display device can see each other's reminders, including group messages, so that friends and family can be informed about the planned or current activities of the person for whom the reminders are intended.
  • group account holders can only see their own group messages, unless permission is granted otherwise, to preserve privacy.
  • Messaging can be set up in advance, and made to appear at the appropriate times relative to the event they refer to.
  • Reminders can be programmed to automatically repeat at specified intervals, from daily to yearly, to accommodate a variety of situations and events.
  • Reminders can optionally require that an acknowledgement by the viewer take place. Multiple acknowledgement requests can be active at one time. If such a reminder is not acknowledged, remote users (friends, family, and administrators) can check the status and/or receive an alert via a short message service or email.
  • Preset reminders exist to help save typing. Account holders can use system-defined preset messages or create their own for future use. Preset messages can be customized by the account user.
  • Messages can also be "instant messages" that are not tied to any particular event. Such instant messages appear relatively quickly and do not require any action by the viewer to see.
  • the system can monitor the health of each display device and alert the appropriate account holders and/or administrators of such a failure.
  • the system focuses on providing hybrid care assistance dependent on the cognitive abilities of a patient, ranging from full third-party control to shared control to an independently functioning patient, in an automatic and natural manner.
  • Third-party control of the system can be local or remote.
  • the system itself will adapt the level of interaction provided to the patient based on further improvement or decline in cognitive ability.
  • the system works to automatically and naturally adapt the triggering of events (e.g., launching applications/events on a display) based on the following core functionalities:
  • the context can take several forms
  • the system also offers a number of additional advantages, including the following:
  • An application can be triggered automatically in a non-intrusive way (e.g., audio message when the patient is not sleeping)
  • the launch and the interaction with the application can be customized to the patient's cognitive ability
  • the patient can enjoy a number of services (e.g., see family pictures, video conference or reminders, listen to music) without having to know how to launch the application/event
  • the solution includes implicit interaction (from the patient's point of view) with explicit interaction (from the third party who arms the event/application point of view)
  • Figure 1 illustrates the general system architecture, showing a set of different types of remote users, a server system and a set of display devices (or simply "display” or “displays” in subsequent descriptions).
  • Each User 100, 1 10 interacts with the system via the network 130, which can be a combination of wide area (such as the Internet) or local area networks.
  • Each user is associated with a particular display 140.
  • Master User A 100 and a related Normal User 1 10 interact with Display A.
  • o Create and edit preset event reminders that others can also use o Change the display's details, such as names, location and time zone o Change his or her own user details, such as names, username, email address and password
  • Group users 120 can be associated with more than one display.
  • Figure 1 only shows one group user to illustrate a situation where there are three displays (A, B, C) at a particular facility that this group user has access to.
  • Master Users can do the following:
  • the Server 150 manages the system, including the access to the system by each of the users and the updating of each of the displays.
  • Figure 1 only shows a subset of what is possible because one server 150 can manage a number of set of users and displays scattered around the world.
  • Databases 160 store all of the information on all users, displays and messages. Sensitive information, such as passwords and email addresses, are kept encrypted.
  • a user interacts via a web browser or dedicated application with the system to create a reminder. This reminder is stored in the database and the server then determines which reminders should go to each particular display at the appropriate times. Users can view the status of all reminders and messages, including making edits and hiding messages as appropriate.
  • the displays merely display the messages that they are sent. Optionally, they can do a small amount of management of these messages to minimize the amount of communications needed during operation. Optionally, these displays may provide a simple way (e.g., touch the display, verbally, etc.) for a viewer to respond to a message, if requested, and this response is sent back to the server.
  • Figure 2 illustrates a typical type set of messages that might be seen on a display. Because complex messages and even graphics can lead to confusion if a person has Alzheimer's, messaging must be kept simple, direct, and appropriate to the situation.
  • the top of the display 200 simply shows the current date and time. The part of the day, such as "Morning” or “Evening” is also shown. Time and date are automatically obtained from the network. Since the display can be in one time zone while a user is in another time zone, the display's time zone is determined by a selection 1710 made by the display's master user.
  • event or reminder "titles" 210 are shown on the left because the illustration assumes people tend to read from left to right. Of course, different cultures can work differently, and so adjustments to how the display is arranged can be adapted to different countries.
  • the size and font color used to display the message title (and other parts of the messaging) change according to how close they are to the event in question. The closer they are to the event's start (and end, if the event is of any length), the larger the font and more urgent the color.
  • a second line 210 is allowed for putting additional messages or instructions. This second line is optional and can be made to not appear until the time gets closer to the event. This delayed showing of the second line follows the assumption that showing too much information too early would only confuse the reader.
  • Additional messaging 240, 250 is added to reminders to give clues on when an event is to take place.
  • the wording of the supplemental timing messages is designed in a simple conversational style. It would be too confusing to the reader to say that an event is supposed to start at "1 1 :30 AM, April 10, 201 1 " if something like "In about 2 Hours" can communicate the same thing.
  • the algorithms for how such timing messaging works can be fairly involved and must be tailored to the cultural and language norms of the viewer. As always, messaging must be kept to a minimum; but, it can also be a problem if too little information is given.
  • Figure 22 shows more sample messaging on a working display.
  • the sample message "Morning Pills” is asking for a response - in this case the pressing of the "OK button” 220. Instructions on how to respond can be given verbally or by other means.
  • the OK button is simply a graphic on the display, and the system senses the pressing of this button by using a touch-sensitive display 1015 system. The status of the response can be monitored, as will be explained later.
  • Figure 3 shows a similar sample display to Figure 1 , but with one difference - the OK button has been replaced with a checked-box icon 320.
  • This icon or a similar type of indicator, tells the viewer that the message was acted on. Sometimes people will forget that they already acted on something that they regularly do, such as taking pills.
  • the checked box icon serves as another form of reminder.
  • Figure 4 shows a couple of ways to distribute the system's logic.
  • the top version places almost all logic in the server side 400, 41 0 and the display 430 is not much more than a thin client, such as a browser connected to the Internet 420.
  • a thin client such as a browser connected to the Internet 420.
  • Such an arrangement means that off- the-shelf products, such as modern tablet computers, can be used for the display.
  • the tablet computer is basically used as a browser display. HTML and PHP commands in various web pages determine what to display and when to display it.
  • Refreshing of the display just after the top of each minute, or at other selected times, is programmed into the webpage by reading the network time and calculating the time for the next auto-redirect command (header( 'Location: page_url.php' ) ).
  • header( 'Location: page_url.php' ) Upon each refresh the display can update the displayed time of day, retrieve new messages, and update the wording and fonts of currently active reminder messages. Audio can be played, if required, via commands found in HTML5, or alternatives.
  • the bottom version places some of the messaging logic into the client side 440.
  • Information on future events can be stored locally in the display's local database 460.
  • Algorithms that have been placed into the display's system can then determine what to display at any given time without having to communicate with the server's system.
  • the display will still need to periodically communicate with the server to get message updates, but such communication can be less frequent.
  • the display could contain a software application that stays resident in nonvolatile memory, if present. This software can be made to automatically execute when the display is first turned on. This means that power and communication interruptions can be automatically addressed.
  • Figure 5 shows categories of typical database 500 tables used in the system. During operation the algorithms stored in the server access the following database tables to determine how to handle each display, user and situation.
  • a table for Displays 505 contains information about each individual display, such as the names associated with the display and time zone.
  • the table for Users 510 contains information on each user, including names, contact information, passwords, and type of user account. Users found in this table are associated with a display or set of displays (if this is a group user).
  • the Messages table 515 holds all of the messages, including information on how and when each individual message should be displayed, who created the message, and the type of message. These three tables comprise the core of the database used by the system.
  • the Display Checks table 520 is used to store the health of each display.
  • the Presets table stores predefined messages that can be used to save some typing. These preset messages contain most of the same information as regular messages stored in the Messages table.
  • the Group Requests table is used to store requests that a Group User has made to Master Users to join a group.
  • the Group Hide table is used to store information that determines if a particular Group message should be displayed on a particular display, or not.
  • the OK Buttons table stores the status of responses for each message that requires such a response.
  • the Instructions table is used to store localized (different languages) instructions and wording for the user interface.
  • the Images table is used to store images that can be associated with particular messages.
  • the Audio table is used to store audio files in appropriate formats that can be associated with particular messages or situations.
  • Figure 6 begins to show how the algorithms and database tables work together to manage all of the displays.
  • the Group Hide table is accessed 615 to determine if this message should be displayed on this particular display.
  • the OK Button table is accessed 620 to determine if a response is required at this particular time or not.
  • a message can be displayed without requiring a response until a predetermined time before the event is to start. Thus, for example, a viewer can see that an event is about to come up, but a response from the viewer is not requested until the event is just about to happen.
  • Audio table is accessed and compiled 630 into the message as appropriate. As with the wording and fonts chosen earlier in the previous step, audio can be tailored, too.
  • the complete compiled message is rendered on the display 640. This includes any text, audio and/or images that were determined to be part of the message in earlier steps. The display and message is then refreshed as necessary based on the refresh timer.
  • Figure 7 shows a part of the relationship between a Group User 700, Master User B 715 and regular User B 720 when placing messages on a particular display.
  • Master User B also has the ability to edit or delete Message B2 that was created by regular User B. But, while regular User B can also edit Message B2, this user cannot edit Message B1 created by Master User B.
  • the Group User in this diagram is shown as creating two Group Messages 705, 710. These group messages potentially go to all displays 760, 770 that belong to this group, but not displays 780 that are not part of this group, even if such displays are on the same network.
  • a Group Message When a Group Message is directed at any display, the Master and regular Users associated with that display also see this message. If either the Master or regular User decides that a Group Message conflicts with a planned event, the user has the ability to hide this Group Message. Each individual Group Message can be allowed to show or be hidden, so Group Message 1 705 can be hidden 745 independently from Group Message 2 710 being hidden 750. Decisions by this Master User B and regular User B do not affect what is shown or not on other Displays 770, 780.
  • Figure 8 shows the flow of activities that determine if a particular display is part of a particular group. Control of the display belongs to that display's Master User, so the Group User must first ask for the Master User's username 800. If the Master User agrees 810, the Group User can then send that Master User an invitation via email to join the group 820. This email contains a special link with an encrypted key that, when clicked, takes the Master User to a web page that displays the group the display just joined 830. From this point the Group User's Group Messages will be seen on the display in question 840, unless the Master User decides to remove this display from the group 850 or hide that particular Group Message 860.
  • Figure 9 shows a variation of Figure 1 , and is used to illustrate how the OK button or acknowledgement system works.
  • a reminder message is created by the Master 900, regular 910 or Group User 915 that specifies the need for an acknowledgement by the display's viewer.
  • the message is saved in the database 980 and served up 970 to the display 940 at the appropriate time.
  • the OK button is displayed 950, along with any other verbal or visual prompts.
  • an external device 960 can be activated to ask for some type of action.
  • the requested acknowledgement is then made by the viewer and logged into the database 980.
  • the various users can then see via a web page if the acknowledgement was made.
  • the server can send a short message (SMS), email or even make a phone call.
  • SMS short message
  • Figure 10 shows a typical hardware block diagram of a display.
  • the display consists of a typical set of elements, including a processor(s) 1000, memory for instruction operation and variables 1040, nonvolatile memory 1045 for BIOS, operating system 1050 and applications 1055, power supply 1060 and optional battery 1065, display 1010 and optional touch panel system 1 015, networking (wired and/or wireless) 1030 for connecting to the WAN/LAN 1035.
  • a processor(s) 1000 memory for instruction operation and variables 1040, nonvolatile memory 1045 for BIOS, operating system 1050 and applications 1055, power supply 1060 and optional battery 1065, display 1010 and optional touch panel system 1 015, networking (wired and/or wireless) 1030 for connecting to the WAN/LAN 1035.
  • This display can be a stand-alone product or be part of another product.
  • this display can be integrated into a television. If so, the touch panel user interface might be replaced with a remote control arrangement. Since most all of the other elements are already part of today's televisions, these elements can be shared and leveraged.
  • Figure 1 1 shows an example user interface screenshot for entering or editing a reminder message. This can be part of a webpage or be part of an application.
  • Each reminder message is then given a start date and time 1 120.
  • Some types of events such as holidays and birthdays, are really about the day itself, so events can be designated as being "All-Day" 1 125.
  • the event is not an All-Day event, the next thing to specify is how long the event lasts 1 130. If the event lasts less than a day, the length of the event can be specified in minutes, hours, etc. If the event takes place over multiple days, the end of the event can be defined by specifying a specific date and time 1 140.
  • audio reminders can be played to draw attention to an event.
  • the type of audio messaging can be chosen separately 1 165.
  • Figure 12 shows a typical user interface that Master Users see for reviewing and managing all of the messages scheduled to show on the display. This can be found on a webpage or be part of an application.
  • the interface shows information about which display it is showing 1200, plus other supplemental information such as the time where the display sits, and if this display is enabled to accept Group Messages 1205.
  • buttons for showing the information in a format that is friendlier for mobile devices 121 0 (automatic switching to this mode is also possible).
  • buttons for adding a new reminder message 1215 or instant message 1220 There is a button to see what the display itself looks like at the moment 1225.
  • buttons for displaying help and infrequently u sed administrative functions 1230 There are other buttons for displaying help and infrequently u sed administrative functions 1230.
  • the main table shows a summary of all of the active events currently lined up for this display.
  • Table columns show titles and notes (1240, 1245), information on when events start and what the display should do at various times 1250, information on when events end and if or how they should repeat 1255, information on when events should start to show, or if they are currently showing on the display 1260.
  • acknowledgements will be requested, or whether an acknowledgement has been given or not, and if an alert should be issued if an acknowledgement is missed 1270.
  • a final column shows who created the message 1275, and shows an edit button if the message is one that this user can edit 1280. Since Figure 12 is for a Master User, this user has edit privileges for any message created by any other Master or regular User. The edit button can be made to look slightly different if the particular message was made by someone else 1280.
  • Figure 13 shows a similar illustration for managing reminder messages, only this one shows what it might look like for Normal Users. Since Normal Users can only edit messages that they entered themselves, the edit button only shows on a subset of the listed messages 1300, and not for messages that others have created 1310. Since Normal Users can hide and show Group Messages, they still see the button for doing so 1320.
  • Figure 14 shows what these interfaces might look like for a mobile device. Only part of the overall interface is shown - the rest can be seen by scrolling or paging. There is also a way to get to the "full" interface 1400.
  • Figure 15 shows what the interface might look like for a Group User. Since the example being used only had one Group Message in it, only one message 1500 is shown on this table. Unlike Master and Normal Users, a Group User can edit a Group Message, so we also see an edit button.
  • Figure 16 shows a typical interface for sending an Instant Message to a display. A place for a message title 1600 and second line of details 1610 is given. A way to specify how long the message should be displayed is then provided 1620.
  • the viewer of the type of display described in this disclosure is a more passive viewer. No action is required by the viewer to get the message onto the display, but at the same time, there is no guarantee that this person will ever notice the message.
  • an audio notice can be specified 1630.
  • a message can be made to request an acknowledgement, similar to messages illustrated earlier.
  • Figure 17 shows a part of the system's administration functions - in this case the management of the display (or "Frame" as it is called in the illustration).
  • Each display can be given a name 1700, which is generally the name of the person that will be viewing the display.
  • Some form of location description such as city or room number, can be specified next 1720.
  • the combination of display names and location help to uniquely identify each display.
  • a checkbox 1730 is provided for doing so. This checkbox is automatically checked when the Master User clicks on the email invitation 830, but can be subsequently unchecked or rechecked at any time.
  • Figure 18 shows an interface for managing Preset reminders. Some Preset reminders are defined by the system and are shown here as coming from Admin 1820. Some presets might have been defined by a Master User, Normal User or Group User. If the user has edit privileges (which follow rules similar to regular reminder messages), an edit button will appear 1810.
  • Figure 19 shows a simple way for picking a Preset. Once Presets have been defined, they are available for picking 1 900 when creating a new reminder message by clicking on a button for Presets (not illustrated, but it would be found on an interface similar to that shown in Figure 1 1 ). Once a Preset is selected, it can be subsequently modified and customized. Thus, users are not locked into a particular set of parameters, dates, times, etc.
  • Figure 20 shows a low-level operation designed to monitor the health of a given set of displays. Displays can be accidently turned off, lose power or communications, or have a hardware failure. Since the display is usually not near the people that manage it, there needs to be a way to get some indication about its health.
  • the server accepts the keep-alive signals from all of the displays that it is monitoring 2030. If one or more of the displays fails to send a keep-alive signal 2040, an alert can be sent 2050. Alternatively, a webpage can be updated to show the suspect display name and location.
  • Figure 21 shows another low-level admin function, display, and Master User setup.
  • This account can be created by a system administrator 2100. This administrator can be a service provider or someone at a factory. If the display is a unique device made specifically to work in this system, a unique account code, probably algorithmically generated, can be stored in the display's nonvolatile memory. If a service provider is creating the display's account, any number of means may be used to create unique codes. Once created, these unique account codes are also stored in the system database 160, 505.
  • the system administrator creates a new Master User account.
  • This account consists of a unique username and a pointer to a specific display.
  • a password advisably unique, is also generated.
  • the Master User setup can be done in the display's factory.
  • a service provider can create the Master User account details. Either way, once created, this information is also stored in the same database 160, 510.
  • the new Master User is given the new username and password. This Master User then logs into the system 2120. Once logged in, this person can create new reminder messages 2130, create other users, etc., as described earlier.
  • the Master User installs the display where it is intended to be used (e.g., the person with Alzheimer's). Installation consists of logging the display into the system 2140. Logging into the system involves two steps. The first step is to establish a network connection. This connection can be accomplished in a number of ways depending upon the specific type of network connectivity used. Connectivity can be accomplished via various wired (e.g., LAN via a cable, modem via phone) or wireless (e.g., Wi- Fi, cellular, Bluetooth) means. For example, if there is an existing Internet service available via a Wi-Fi connection, the display would first need to establish a link to this Wi-Fi.
  • the second step for logging the display into the system is to make the system aware of the display's unique identification code established earlier 2100. This step can be done manually or automatically by the display.
  • a screen on the display would request the display's account log-in information, such as a username and password.
  • the user could use any of a variety of input devices (e.g., touchscreen, remote control or keyboard) to enter the required information.
  • the display would read its unique identification information from nonvolatile memory and pass this information to the system. Automatic logging-in of the display can be done once the display's nonvolatile memory is loaded with the required information, either by the factory or the Master User.
  • Passwords in particular, would be encrypted before being passed to the server. Encryption is necessary to preserve privacy.
  • Figure 22 shows a prototype display device. It is in a stand so that it can be placed on a tabletop. Alternatively, such a display can be built into a wall or be part of another device, such as a television.
  • a "Visit Alice” event shows a bit more detail 2230. This happens to be a multi-day event, so we see that it starts “In about a Week” on “December 17” and lasts “For 3 Days” 2235.
  • Each of these messages will automatically change over time, depending upon how close to the event it is, and if the event has started, or just ended.
  • the illustrated sample display has a white background because the photo was taken during the day. To reduce the possibility of disturbing someone sleeping, during night hours the display's background becomes black and font colors are adjusted accordingly for readability. Timing for when the display goes into night mode can be arbitrary, set by Master User selected options, or automatically adjusted according to where in the world the display is located, as determined by the geo-location of the IP address detected by the display.
  • the computer-implemented system is shown generally at 10, and includes a computer system 12, which may be implemented using a single computer or using a networked group of computers to handle the various functions described herein in a distributed fashion.
  • the computer system 12 manages an electronic database 14 and also optionally an analytics system used to analyze data stored in the database 14.
  • the database 14 functions as a data store configured to store plural items of information about time-based events (and other context-based events) for the patient.
  • the analytics system may be programmed, for example, to analyze trends in a particular patient's cognitive abilities, so as to adjust the performance of the system to match those abilities, and also to provide feedback information about the patient to interested parties such as the patient's caregiver.
  • one device may be a tablet computer operated by the patient, while another device may be a wall-mounted television display in the patient's room.
  • the system can dynamically control which device to use to interact with the patient. In some instances, both devices may be used simultaneously.
  • the system is able to customize the presentation sent to each device individually. Thus the level of complexity for the television display might be different than that used for the tablet computer, in a given situation.
  • the system is able to use context information and also the patient's cognitive ability to adapt each display as appropriate for the patient's needs.
  • the computer system 12 may also be programmed to generate memory games that are supplied to the patient.
  • a memory game generator 16 is shown as coupled to the computer system. It will be understood that the generator may be implemented by programming the computer system 12 to generate and make available the appropriate memory games, based on the patient's cognitive ability. Memory games can be extremely helpful to exercise the patient's memory, possibly slowing the progress of the patient's disease. In addition, feedback information captured automatically as the patient plays the game is used to gather information about the patient's current cognitive ability, which is used by other systems as will be more fully explained below.
  • the computer system 12 also preferably includes an application program interface (API) that presents a set of standardized inputs/outputs and accompanying communications protocols to allow third-party application developers to build software applications and physical devices that interact with the system 10, perhaps reading or writing data to the database 14.
  • API application program interface
  • the computer system 12 includes a web server 22 by which the caregiver 26 and patient 28 communicate with the computer system 12.
  • web pages are delivered for viewing and interaction by computer devices such as tablets, laptop computers, desktop computers, smartphones and the like.
  • the computer system 12 may also be connected to a local area network (LAN) 24, which allows other computer devices to communicate with the computer system 12, such as a workstation computer terminal utilized by a nursing home staff member 30, for example.
  • LAN local area network
  • the database 14 is configured to store data organized according to predefined data structures that facilitate provision of the services performed by the computer system.
  • the database includes a data structure 32 that stores plural items of information (informational content) that are each associated with a set of relevant context attributes and associated triggers.
  • an item of informational content might be a reminder message that the patient has an optometrist appointment.
  • Associated with that message might be a trigger datum indicating when the appointment is scheduled.
  • Also associated with the message might be other context attributes, such as how large the message should be displayed based on what device the message is being viewed upon. See Figure 32 as an example of a display of this message.
  • the informational content stored for that event might include a very general text reminder, stored as one record in the data structure 32.
  • the system might provide more detailed information about the event (such as a reminder to "bring your old glasses"). This would be stored as a second record in the data structure. The system chooses the appropriate item of information, by selecting the one that matches the current context.
  • the system also stores in another data structure, the current context for the patient, such as where the patient is located, any relevant medical condition attributes, and the like. These are shown as context data structure 34. Further details of the context attributes are discussed below.
  • the computer system 12 uses the current context attributes in structure 34 in determining which information content to retrieve from structure 32.
  • the computer system further maintains a cognitive ability data structure 36 which stores data indicative of the patient's cognitive ability. This may be quantified, for example, as a relative value suitable for representing as a sliding scale, e.g., a 1 -10 scale.
  • the patient's cognitive ability may be assessed by explicit entry by the caregiver or nursing home staff.
  • the system can establish the cognitive ability data itself through feedback from the memory game generator 18 or by analyzing how well the patient is able to interact with the system generally.
  • the system automatically launches specific applications and events based on set parameters configured by third parties, taking into account specific information, such as patient context, technology context, and situation context.
  • Figure 23 shows the key considerations that are taken into account during the process of determining which application/event to launch, when to launch it, and how to launch it. If the context information meets the parameter settings, the execution of an application and/or event is triggered. This provides some information or interaction for an individual to see or use on a computer terminal such as a tablet computer.
  • the system also adjusts the level of interactivity based on the cognitive ability of the patient. The goal is to provide a patient or user a non-intrusive, automatic way to obtain information and services that are relevant and sometimes necessary.
  • the third party is a person or entity that generally has at least some involvement in caregiving. Such third party may have control to put in reminders, start videoconferences, upload pictures, set appointments, and other features of remote care.
  • caregiver refers to such a third party and may include family members, doctors, nursing staff, and the like.
  • context also provides useful information.
  • the system is able to initiate some of events/applications with knowledge of other factors that the caregiver may not be aware of. These include the situation currently at the nursing home or other patient center (current situation detectable by cameras, microphones, nurse/doctor input, medical sensors, and the like), active/available technology information (e.g., don't send the reminder to the person's watch but put it on the TV), and medical information (data from medical sensors, current doctor reports, current status reports by users).
  • Patient cognitive ability also forms an important aspect of the system, as shown in Figure 23.
  • Patient cognitive ability is the current level (on a rating scale) of the patient's ability to interact with the electronics system, tablet, or other device in the system. If the rating is high, the patient likely can interact with the device himself or herself and may not need as much assistance from some context or third-party support. If the rating is low, the system and third parties can provide more support.
  • the cognitive ability scale, and how it is determined, is discussed more in relation to Figures 29 and 30 below.
  • the computer-implemented system captures and stores an electronic data record indicative of the patient's cognitive ability.
  • the electronic data corresponds to a collection of individual measurements or assessments of skill (skill variables), each represented numerically over a suitable range, such as a range from 0 to 10. If desired, an overall cognitive ability rating or aggregate assessment may also be computed and stored, based on the individual measurements or assessments.
  • the dynamic rendering system uses these skill variables to render facts in the most appropriate manner based on the patient's skill set. In this embodiment the collection of skill variables, stored in memory of the computer, thus correspond to the overall "cognitive ability" of the patient.
  • the skill variables comprise a set that can be static or dynamic.
  • these skill variables may be algorithmically combined by the computer system to derive a single value "cognitive ability" score.
  • a suitable scoring mechanism may be based on the clinically recognized stages of Alzheimer's disease, namely:
  • Stage 2 Very mild decline
  • Stage 3 Mild decline
  • Step 1 A third party (e.g., famiy, friend, caregiver) will input information and configure the paramenters for triggering of events and applications.
  • Step 2 Once the system has been armed, the system will gather and store contextual information. Such context information, about events and the like, are preferably composed of three sub-contexts: patient related information, situational/external information, and event/application/device information.
  • Step 3 If contexts meet the armed settings of the system, an event may be triggered.
  • Step 4) If triggered, the system will launch the application whilst customizing the interaction level for the patient.
  • the context of an event can be composed of 3 sub-contexts: a patient-related context, a situational or external condition context, and a technology context.
  • the state of these contexts is stored in a context data structure within the memory of a computer forming part of the system.
  • the patient-related context contains all the information that is available from the patient (this is not exclusive). This information is stored as data in the context data structure. Examples of patient-related context data include:
  • Digital medical record • Patient behavior (e.g., sleeping or not)
  • Patient location e.g., in the room, looking at the display
  • Patient preference e.g., audio trigger/notification preferences, preferences of sounds, videos, tv shows, pictures
  • the situational/external context contains all the information that is available from external sources to the patient (this is not exclusive). This information is likewise stored as data in the context data structure. Examples of situational or external condition context data include:
  • the Event Application/Device context contains all the information that is available from the devices that make up the system. This information, collected by communicating with the devices themselves, is likewise stored as data in the context data structure. Examples of technology context data include:
  • Type of display e.g., size
  • the technology context is useful because different devices may be added to the network at a future time to add additional functionality. For example, if the patient or patient's caregiver purchased a 'help me' necklace, a new TV, or a digital picture frame, the system can recognize contexts including these new technology (allowing the system to modify its behavior, for example, by displaying the pictures on the picture frame instead of the master tablet device).
  • Each event/application uses a specific context (subset of the most general context) to be triggered.
  • Figure 25 illustrates the polling contextual information that the system will gather from Step 2 of Figure 24.
  • the information will be divided into three categories.
  • Patient-related information such as medical records
  • live medical information will be constantly gathered by the system via sensors, stationary and mobile.
  • the system will analyze the data to determine whether an event will be triggered based on the parameters set by the third party ( Figure 26). If it is determined that the trigger contexts have been met, then the system will evaluate whether the event will be launched. For example, if the schedule reminder is to go off at a specific time, the system will need to determine whether the patient is present in the room. If the patient is not in the room, the system will not launch the alert; however, if the patient is observed to be present, then the system will trigger the event.
  • the system will not launch the alert or event when, for example, a family member wants to engage in a video call, but the context indicates that it is nap time, or that a doctor is currently providing therapy (based on a priority ranking). Likewise, the system would not launch a reminder about medication if the patient is still at dinner and the medication is to be taken after dinner.
  • trigger events include:
  • timed event e.g., medication/reminder
  • the method of obtaining the patient's attention may be more obtrusive and obvious.
  • the alert will draw the patient towards the tablet display screen, at which time the camera will detect the presence of the patient. If the presence is recognized as the patient (identification may be established by facial recognition, sound or other electronic identification system), the system will seamlessly launch the application/content. Upon launch of the event, the system will again consider the cognitive level of the patient to adjust the level of interactivity and necessity for interpretation appropriately. For example, if the patient is receiving a video call, then the system will alert the patient. If the patient is fully functional, the system will display the options for the patient to either accept or decline the video call. In the case of a low-cognitive patient, the system will authenticate the patient's identity prior to automatically launching the video call or the calling third party could automatically initiate the application, as shown in Figure 28.
  • the system When customizing the interface for the patient, the system will take into consideration several factors (Figure 29). Initially, the cognitive ability of a patient will need to be inputted manually by a third party; however, as the patient continues to utilize the system, the system will register and adapt to the history of skills of the patient from preveious interactions. The system also can provide mini-games specifically designed to test the current cognitive ability, and thus the system can automatically update how it should interact with the user.
  • the system will apply the preferences of the patient and third-party individuals (e.g., doctors, caregivers, family, friends, etc.). Thus, the patient will not have difficulties using or interpreting the system's events.
  • the interface of the system will change depending on the patient's preferences and cognitive level as well. From a simple and automatic interface for those who are cognitively (or technologically) incapable, to more complex and manual for the independent and cognitively high-leveled patient (Figure 30).
  • One embodiment requires manual input of the patient's cognitive level; however, the system will adapt to the patient by having the patient perform tests within the system.
  • the embodiment may be configured to day-to-day differing cognitive levels.
  • the system's parameters can be routinely adjusted by third parties.
  • the system can be configured to perform daily or weekly or bi-weekly testings to automatically accomodate the patient's needs.
  • Customization of the system is not limited to events launched when contexts are triggered.
  • the system may launch events when in a state of rest. For example, if there are no events set to launch, yet the patient is observed to be present in the room, the system tablet display may go dark or display a picture show or become a reflective mirror.
  • Figure 31 illustrates one embodiment in which the system is comprised of the tablet screen, event database, multiple web applications, RF communication (e.g., Bluetooth), and connections to a multitude of devices (e.g., cameras, microphones, speakers, computers, mobiles, etc.).
  • the system will also display Web API.
  • Figure 32 shows an example user interface that includes applications/events that were launched.
  • an additional display can be provided to the caregiver or to another third party.
  • This additional display could be implemented on a tablet or smartphone and would provide information to the caregiver about what the patient is up to and activities he or she did in the past, as well as feedback about the patient's medical condition. This feedback loop provides reassuring information to the caregiver.
  • this additional display presents information that is different from the information displayed on the device used by the patient.
  • the information presented to the caregiver is derived from information stored in the database system, which may be supported by a server associated with a nursing home or healthcare provider, or which may be supported by a service provider offering the services using Internet-based or cloud computing resources.
  • the patient device 50 with display 200 is configured generally as described above to provide services to a person of reduced cognitive and/or physical ability.
  • the patient device has a communication port, such as a WiFi wireless communication port or cellular data communication port, allowing the device to communicate over the Internet.
  • the information displayed on the device can be sent to a TV 52 in the patient's room.
  • the TV 52 can be provided with computer network communication capability, allowing it to directly communicate with the portal computer 55.
  • the TV 52 can be configured to act as a display that mirrors the information shown on the patient device 50. In this later case, the TV 52 is in communication with the patient device 50 and device 50 handles communication with the portal computer 55.
  • Figure 35 shows an exemplary screen as displayed on display 200 of the patient device.
  • the exemplary screen is generated by a calendar application running on the patient device.
  • the patient device may be implemented using a variety of different hardware and software architectures.
  • the tablet form factor is presently preferred.
  • the patient device may be implemented using a tablet computer using the iOS operating system, the Android operating system, the Windows Surface operating system or other systems.
  • the patient device is implemented as an application (App) running on the native device's operating system (e.g., iOS, Android, etc.).
  • the patient device is implemented using a tablet computer running the patient device software as a standalone application which includes the hardware interface layers allowing the standalone application to send images to the display 200 and to respond to touch interface commands through a touch screen device associated with the display.
  • the patient device employs the software architecture shown in Fig. 43 by which the processor of the patient device is programmed to perform the functions described here. Of course, different architectures may also be employed. As illustrated, the software architecture may be built upon the native device operating system 231 .
  • the updater 213 is a dedicated software component that connects to the portal 55 at a pre-defined time frequency looking for a new wrapper 215 software release.
  • the wrapper 215 is a dedicated middleware connecting the low- level native device operating system 21 1 to a Ul and application-rendering package.
  • the wrapper 215 implements the device APIs for synchronizing the local buffer to the remote portal 55. These APIs include device authentication; activity, message, picture and video download; user settings and customization download; and check and download the latest bundle 217, including Ul design and device-specific applications.
  • the wrapper 215 executes the code and information provisioned through the bundle 217.
  • the final Ul shown on the device is completely customizable depending on the bundle 217 information loaded and the device-specific settings like: zoom level, enabled/disabled apps, volume level, Text-to- Speech enable/disabled; favorite applications; new event prompt options, etc.
  • the bundle 217 contains all the Ul and application-specific code.
  • the system enables very targeted distribution of bundles for each display. With this, it is possible to customize every user interface aspect such as: fonts, creation of new applications, resizing and redistribution of all graphical components.
  • the bundle contains all event querying and display logic.
  • the wrapper starts bundles, updates bundles and supplies operating system-related functions to the bundle (e.g., text-to-speech, reboot, WiFi restart).
  • the updater allows wrapper updates and starts wrappers.
  • the updater is manually installed on the patient device.
  • the updater provides no visible user interface and normally does not require updates.
  • the updater is started or launched upon booting of the patient device, or it may be started by a user via a suitable touchable icon displayed on the screen of the patient device. Once started, the updater starts the wrapper, if available, and checks for wrapper updates every predetermined time interval, such as every 10 minutes. In case of network failure with no wrapper installed, the updater rechecks at more frequent intervals, such as every minute.
  • the updater installs new wrappers.
  • the native operating system is the Android operating system
  • the updater may be implemented using the Android APK.
  • An Android service is scheduled to run every 10 minutes through its alarm manager.
  • the Android APK is also used to schedule service run and to start wrapper activity.
  • the APK is also used to broadcast receiver handling onboot message and starting activity.
  • the wrapper is automatically installed by the updater. It has no visible icon and is always started through the updater. The wrapper checks and installs new bundles. The wrapper periodically performs system restart and also periodically performs WiFi restart. These steps are performed to ensure the patient device is forced to reestablish connectivity in the event the network connection is lost for some reason. The wrapper is also responsible for sending crash reporting messages to the portal computer, as shown in Figure 45.
  • Bundle The bundle is automatically installed by the wrapper.
  • the bundle supports protocols such as HTML5, CSS, JS App. Unlike the updater and wrapper, the bundle is actually visible on the patient device, such as in the form of the calendar.
  • the bundle uses the event API and handles display of all events. The bundle also notifies the wrapper of any display setting changes.
  • the bundle is implemented in HTML5, CSS, and as a java script (JS) compressed tar file.
  • the bundle may be configured to use and support protocols and technology such as:
  • JQuery e.g., Internet Explorer only
  • HTML5 APIs are supported:
  • Figure 46 shows the updater broadcast receiver operation that runs after the patient device finishes its boot-up sequence. The procedure is straightforward: a start Updater Activity procedure is launched, causing the updater to be loaded and run, whereupon the updater broadcast receiver procedure finishes.
  • the wrapper is started next, as shown in Figure 47.
  • the procedure first sends crash reports, if any, testing to ensure the report transmission was transmitted okay, and then deleting the local copy of the crash report.
  • the procedure then tests to see if root access to the underlying hardware is operational, and if not, a report to the user is generated and the wrapper activity start sequence is terminated. This report to the user is provided for troubleshooting purposes; the ordinary user of the patient device is not expected to understand this message.
  • wrapper activity start procedure then disables the device auto update, schedules a WiFi reboot, restarts and then runs the downloaded Changed settings will also initiate a reboot, as illustrated.
  • the downloader activity of the wrapper is illustrated in Fig. 48. If there is no new bundle to be downloaded, the wrapper activity merely sleeps for a predetermined time interval and then tests again to see if a new bundle is present. If so, the bundle is downloaded and the activity downloader checks to see if the file was properly downloaded, including checking the file's MD5 checksum to ensure that the downloaded code was not corrupted during transit. The code is then installed and the procedure again enters a sleep state for the predetermined time interval. If any of the downloading procedures or subsequent testing and loading procedures fail, the activity downloader enters a sleep state for a different predetermined interval (the sleep-upon-failure interval).
  • the crash handling wrapper activity procedure is shown in Fig. 49.
  • the wrapper collects relevant exception (error) information, including a timestamp when the crash occurred.
  • This exception information is saved to local storage and an alarm is scheduled to cause the device to restart after a predetermined time, such as 10 seconds.
  • a predetermined time such as 10 seconds.
  • a crash or exception may be due to an intermittent loss of network connectivity.
  • a scheduled reset is programmed into the crash-handling procedure to address this by rebooting.
  • the procedure used by the bundle mechanism is illustrated in Fig. 50.
  • This procedure is run by the bundle on the patient device when settings are to be updated.
  • the procedure fetches the new settings from the portal computer (server), tests to ensure that the portal computer has associated proper credentials with the new settings (to ensure these new settings are intended for this device), and then a check is made to determine if any settings have actually been changed. If so, the settings are updated and the wrapper code is notified.
  • checking credentials if the patient device has previously been provided with the necessary credentials, nothing needs to be done by the user of the device. If, however, the credentials are not present, or if they do not match the designation from the portal computer, the user is prompted with a login screen to enter the necessary credentials.
  • the bundle settings update procedure determines that no changes to the settings have been made, the procedure enters a sleep mode for a predetermined refresh rate interval.
  • refresh rates may be established. All of these may be programmatically changed by interaction through the portal.
  • TTS Enabled enable/disable application (e.g., calendar) text-to-speech
  • WiFi restart rate periodic WiFi restart
  • the portal computer is a networked computer or collection of integrated computers that communicate over a suitable network connection with the patient device 50.
  • the portal computer and the patient device may be programmed to communicate over a secure channel via the Internet, such as using a virtual private network (VPN) connection.
  • a data storage system 56 that is programmed to function as a database into which is stored the data used to implement the patient device- portal integration.
  • the portal computer can be implemented using the computer system 12 (or server 150 and database 160) described above. Alternatively, a separate computer may be used to implement the portal computer 55.
  • Family members interact with the portal computer to customize the user interface of the patient device 50, to select specific content or specific applications, and to directly control the patient device by remote control. Information entered through the portal computer is pushed to the patient device 50, and feedback about how the device is being used by the patient is sent back to the portal computer.
  • the portal computer 55 stores within its data storage system 56 a collection or kit of Ul components as well as digital content (e.g., pictures, video, music) and application programs that can be pushed to the patient device 50 where the application programs are then run. To do this, the portal computer 55 thus functions to allow family members or caregiving facility staff to customize the way the patient device functions. Such customization may be classified into three categories: Ul component customization, shown diagrammatically at 60, content and application customization, shown diagrammatically at 68, and remote control capability, shown diagrammatically at 70.
  • Customizations including selected Ul components, applications and content, are pushed from the portal computer to the patient device.
  • the portal computer assembles a package containing the data structures that store all Ul component setting information and certain application selection information.
  • the package may be compressed using any of a variety of different data compression algorithms and then sent via the secure channel to the patient device.
  • the patient device decompresses the package and extracts the included component setting information and application selection information.
  • These data are then placed into a buffer in memory of the patient device while the current settings of the device are saved in the memory of the patient device for backup. Then the data placed in the buffer is swapped for the current component settings and application selection settings, and any executing applications are commanded to reboot or otherwise reload the new settings.
  • the data package can also contain actual copies of executable applications to be run on the patient device, as well as content such as pictures, music, video and other multimedia content supplied by the family member. Because the portal computer saves the state of the patient device, the portal computer does not resend copies of executable applications that, according to the saved state information, are already resident on the patient device.
  • an application selection setting is stored in the package delivered from the portal computer to the patient device. This application selection setting is operated on by the patient device by suppressing visibility of the executable application, and without actually deleting the application from the patient device unless it is deemed necessary to reclaim storage space from the patient device. Thus if the family member or caregiver later decides to re-enable the application, it can simply be switched on via the application selection setting and does not have to be pushed again to the patient device.
  • the portal computer may be implemented by a networked computer that is coupled to communicate over the Internet.
  • the portal computer has at least one processor and associated non-transient memory into which the program instructions are stored to cause the portal computer to implement the functions described here.
  • the portal computer is equipped with display monitor and suitable input device(s) such as keyboard, mouse and/or touch screen.
  • the portal computer employs the following software architecture by which the processor of the portal computer is programmed to perform the functions described here. Of course, different architectures may also be employed.
  • FIG. 44 The portal communication software architecture is shown in Fig. 44. As illustrated, the networked computer acting as the portal computer sends and receives messages to and from the patient device. Fig. 44 shows how the APIs of the portal computer and patient device are configured to interact through this messaging. A further description of the portal software API will now be described.
  • the portal computer is configured to include a wrapper update API 219 that communicates with the updater 213 on the patient device, and mediates the "check new wrapper version" message used to signify when a new wrapper is to be delivered and installed on the patient device.
  • the portal computer also includes an event API 221 that mediates events performed on the patient device.
  • the event API responds to messages including "login,” “get event list,” and “get calendar preferences.”
  • the event API also handles similar messages with other applications pushed to the patient device.
  • the portal computer further includes a bundle update API 223 that mediates the updating of bundles on the patient device, through the "check new bundle version" message.
  • the portal computer includes additional portal software API's 225 by which the portal computer can be interfaced with other systems, such as with cloud services available on the Internet.
  • the portal computer also includes a sentry crash reporting module 227 that logs crash reports sent from the patient device.
  • crash reports may be generated, for example, if the previously described updater-wrapper-bundle mechanism does not properly operate, such as due to an intermittent loss of network connectivity.
  • Crash reports may also be sent from applications running on the patient device, such as the calendar application, for example.
  • Login - Provide a way to authenticate the client on the server.
  • uhid A UNIQUE hardware identifier. The system accepts only one device per user and vice-versa.
  • apikey The APIKEY obtained with the /login method. [0241] get_update_version - Returns the latest available bundle version for this client.
  • apikey The APIKEY obtained with the /login method.
  • uhid A UNIQUE hardware identifier. The system accepts only one device per user and vice-versa.
  • Sync_Api - Provides all data needed by the client. This includes settings, events, messages and other media resources (photos, music and videos). Every request to this method returns a SHA1 Hash in a header field named E-Tag.
  • This hash shall be used on the next request.
  • the server uses this information to determine if there was any change in the previously served content. If positive, new data will be returned with a 200 status code. If negative, a 304 will be returned. This not only reduces the server CPU usage but also saves bandwidth.
  • the media resources (images, music and videos) shall be stored locally. Each media has a unique name across the system.
  • On4-Token The APIKEY obtained with the /login method.
  • On4- Version The application version.
  • On4-TzOffset The time difference between UTC time and local time, in seconds. A minus sign must be used if local time is behind UTC.
  • On4-Position The device latitude and longitude. This information is used to provide accurate weather information to the device.
  • On4-Time The current UTC timestamp. The value 0 (zero) is accepted for NOW. The server will return events from this time up to 3 days ahead.
  • If-None-Match The E-Tag value obtained from the last request, or empty if first call or none has been provided. Ul Component Customization and Ability Metrics
  • the user interface with which the patient interacts when using the patient device 50 is comprised of a kit of Ul components 62 that can be selected by the family member (or staff member) and then pushed to the patient device.
  • the Ul components are stored in the data storage system 56 and comprise a specially chosen set of user interface components, each graded to match a certain level of cognitive and/or physical ability, to provide the patent with the information he or she needs on a daily basis and to allow family members and the patient to stay in touch.
  • each of the individual Ul components 64 within the kit 62 has at least one associated cognitive and/or physical ability metric 66.
  • the ability metric assigns a numerical score to each Ul component based on the degree of difficulty a user will have in using that Ul component.
  • the kit 62 includes a variety of redundant or function-overlapping Ul components, ranging from automatic, to extremely easy to use, to sophisticated to use, so that the appropriate one can be selected for the particular patient's abilities.
  • plural metrics can be associated with each Ul component.
  • physical ability metrics (vision, hearing, manual dexterity) may also be associated.
  • Cognitive ability and other ability metrics may also be applied to rank the degree of difficulty of different applications or even content, where applicable.
  • a user of high cognitive ability and high physical dexterity might have no difficulty understanding how to use a dropdown menu Ul component to select which photos he or she wants to view in a photo-viewing application.
  • a person of lower ability might not be able to navigate a dropdown menu, but might be able to understand how to use forward and backward buttons to browse through photos once they have been automatically selected for the patient.
  • a person of still lower ability might be unable to operate any Ul components, in which case the application would perform an automatic slideshow, where photos are selected at a predefined rate not controlled by the patient.
  • Figures 36a and 36b which feature a rudimentary photo-viewing application that has been pushed to the patient device.
  • Figure 36a represents a display that might be customized for a patient of average eyesight and moderate cognitive ability.
  • the day and time are displayed in "normal” sized letters and the "next photo,” "previous photo” and “go back” buttons 80 are displayed and active.
  • the prospect of pushing a left-pointing button to "go back” and a right-pointing button to "go forward” is not a daunting task.
  • the user interface in Figure 36b has been customized using the portal for a person with below average eyesight and limited cognitive ability (or perhaps limited manual dexterity).
  • the day and time are displayed in a "large” size and the "next photo", “previous photo” and “go back” buttons have been suppressed.
  • the slideshow feature 82 of the photo-viewing application has been switched to "auto” which causes the pictures to cycle automatically from a first picture, to a next picture, and so forth. In this case the patient has only to watch the screen. There are no buttons to push.
  • the portal computer 55 can selectively assemble a highly refined and highly customized user experience for each patient.
  • having this large number of different Ul components at different ability metric levels is not without its difficulties.
  • the typical family member or nursing staff person may have little or no experience in user interface design.
  • the portal computer sorts and ranks Ul components according to cognitive ability metric (and other metrics), so that the appropriate ones are offered first to the person seeking to customize the user interface of the patient device.
  • Figures 42a and 42b show two alternative embodiments for how this is accomplished.
  • the display 57 presents an image of what appears in the display of the patient device, in this case the calendar application, as at 81 , with clock-date-weather header, as at 83.
  • an application selector 85 is an application selector 85, where icons corresponding to the applications are available to be pushed to the patient device.
  • the display 57 also includes a ranking window 87 where the ability metric for each application is displayed.
  • the clock-date-weather header has a metric of "2" indicating that this interface component is relatively easy to use, as compared with the pictures application, which has a metric of 4 and is thus above average in difficulty.
  • the ranking window shows metrics as they are assigned by the system, which can either be based on actual usage statistics of the particular patient, or based on aggregate statistics generated by the aggregation server computer 74 (Fig. 34). If plural different metrics are implemented (e.g., cognitive ability and physical ability metrics), the user of the portal computer allows the family member or nursing staff to switch between metrics by selecting the appropriate radio button at 89. Selecting either of these buttons will switch the ranking of the displayed applications to reflect the selected choice.
  • metrics e.g., cognitive ability and physical ability metrics
  • Figure 42b shows a different manner of offering applications for selection to be pushed to the patient device.
  • an application selector window 91 lists all available applications, sorted in order of ability metric, based on the radio button selection of the desired metric at 89.
  • the family member or nursing staff can then drag selected applications into the push window 93. Doing this marks the applications in the push window as applications that will be pushed to the patient device. In this case three applications have been selected.
  • the portal computer displays the average difficulty and maximum difficulty based on the applications that have been dragged into the push window. This allows the family member or nursing staff an easy-to-understand grasp of the degree of difficulty the selected pallet of applications will present to the patient.
  • the portal computer 55 is programmed to automatically select an appropriate configuration, based on previously obtained knowledge of the patient's abilities. This a priori knowledge is obtained through feedback of usage statistics from the patient device, as will be described below.
  • Figures 38 and 39 show exemplary screens whereby the basic display and user interaction features are set.
  • Figure 39 includes an application selection structure 91 and 93, similar to that of Figure 42b.
  • Figure 40 shows another exemplary screen that is used to add music and video content to a list of content pushed to the patient device.
  • Figure 41 shows an exemplary screen used by the family member or nursing staff to preset a messaging application, such as Skype, with persons with whom the patent wants to regularly communicate.
  • a family member may want to share an application or content with the patent, but the user interface for that application or content may be above the patient's cognitive ability.
  • the portal provides a remote control capability, allowing the family member, operating through the portal, to directly control what is presented on the patient device.
  • the family member may want to show the patient some pictures that were previously pushed as content to the patient device.
  • the family member could, for example, directly launch a slide presentation application, allowing the patient to see the pictures.
  • Feedback is important in several respects. First, feedback alerts the family member when a particular user interface component, content or application is not being used, or is being used incorrectly by the patient. This allows the family member to make customizations so as to better match the user interface, content and applications to the patient's cognitive ability. As shown in Figures 37a and 37b, the feedback provided from the patient device can be presented to the family member in a variety of ways. Two ways have been illustrated.
  • Figure 37a the display 57 attached to the portal computer supplies an image of the screen, as it appears on the patient device. This display is updated in real time and shows what the patient is currently doing. In this case, the patient last pushed the "calendar" button and is thus viewing the calendar application.
  • Figure 37b shows an alternate "heat map” view, where each Ul component that may be actuated by the user of the patient device is shown with an overlay, showing a heat map or graphical comparative view of how frequently each of the Ul components has been used during the last measuring interval.
  • graphical indications in the form of different sized circles are used to show relative usage statistics. The larger the circle, the more often that Ul component has been used.
  • numerical statistics can be displayed along with the graphical indications or in place of the graphical indications. Of course, other ways of graphically representing a heat map may also be employed.
  • feedback may also be sent to an Internet-based aggregation server computer 74 (Fig. 34) in the cloud 72 where usage metric statistics are aggregated across many users.
  • the aggregated data is then used to update the cognitive ability metrics for each of the Ul components. In this way, the degree of difficulty of each Ul component is fine-tuned based on usage statistics gathered over time. If desired, the aggregate data can be weighted and combined with usage data from the specific patient, to provide a blended cognitive ability metric.
  • These aggregated data are also used by the portal computer 55 when automatically configuring a "recommended" user interface configuration for a particular patient.
  • the portal computer receives from the aggregation server computer a set of Ul component templates, representing recommended customization configurations for each of the applications stored on the portal computer that can be pushed to a patient device.
  • Ul component templates representing recommended customization configurations for each of the applications stored on the portal computer that can be pushed to a patient device. These templates are constructed using the aggregate usage statistics so that Ul components are selected to be consistent with each of a plurality of different ability levels.
  • the aggregation server computer is a networked computer or collection of integrated computers that communicate over a suitable network connection with the portal computer, or alternatively, directly with the patient device(s).
  • the aggregation server computer has at least one processor and associated non-transient memory into which the program instructions are stored to cause the aggregation server computer to implement the functions described here.
  • An exemplary aggregation computer employs the following software architecture by which the processor of the portal computer is programmed to perform the functions described here. Of course, different architectures may also be employed.
  • a "Data Collection” software agent kept always running inside the patient device.
  • the Data Collection software logs on a local buffer all system announcements such as message, picture, video, activity, and also all user interaction such as touch, application selection, and scroll. Every log data point is time-stamped to reflect the exact time it took place. According to a customizable frequency rate, the new log data points are uploaded to the portal using a Log_API.
  • Log_API This method allows clients to send information logs to the server. This can be used to send any important/relevant information.
  • apikey The APIKEY obtained with the /login method
  • Event_time 1380566627, //A unix timestamp representing the message creation time "level”: "INFO” //Log level.
  • Valid options are: INFO, DEBUG, WARNING and ERROR ⁇
  • the aggregation server computer 74 is programmed to compute aggregate ability metrics across all applications and Ul components, for each different ability metric employed by the system (e.g., cognitive ability metrics, physical ability metrics, etc.). To do this, the aggregation server is programmed to receive usage data packets sent from each portal computer participating in the aggregation service. The portal computers are each programmed to periodically send usage data packets containing information extracted from the feedback received from the patient device(s) communicating with that portal computer.
  • the usage data packet consists of one or more records each containing the following information:
  • the data packet does not contain any information disclosing the identity of the patient. All that is known about the patient is his or her currently assigned ability level.
  • the aggregation server computer extracts and stores the data from the data packets received.
  • the data are thus accumulated for a plurality of portal computers, each potentially serving a plurality of different patients.
  • the data collected by the aggregation server represents a population of patients, indicating the degree of difficulty the population had in using each of the different Ul components and applications. From the aggregated data the aggregation server computes usage statistics that are then sent back to the respective portal computers.
  • the usage statistics include, for example, aggregate (population- wide) metrics for each Ul component and application.
  • aggregate metrics might show, for example, that the Ul component having ID number 512 was used by 90% of the population of patients who reported an ability level of 3, but by only 20% of the population of patients who reported an ability level of 2.
  • the statistics generated in this fashion can be sent back to the participating portal computers where they are used in ranking the Ul components and applications for presentation to the family members or nursing staff when designing a Ul configuration for a particular patient.
  • the usage statistics generated by the aggregation server computer represent how the Ul components are each received and understood by a potentially wide population of varying ability levels.
  • Each individual patient may have specific ability levels that differ from the population. For example, a particular patient may generally track fairly consistently with the ability level 3 population, but may idiosyncratically have a level 4 ability in using a particular Ul component or application.
  • the portal computer can take this fact into account, by also using the usage data collected from that patient and overriding specific Ul component rankings from the population to match the specific patient's abilities. This may be accomplished by storing a table of Ul component rankings for each patient within the data storage system 56 of the portal computer. Initially, all Ul component rankings are assigned the values received from the aggregate population.
  • the Ul component rankings generated by the aggregate population data may also be used by the portal computer in establishing the ability level of a particular patient. This is done by systematically pushing Ul components and applications to the patient device in gradually increasing levels of difficulty (based on aggregate ranking metrics) over time until it can be established which Ul components or applications the patient is not making good use of (based on the feedback provided from the patient device). The patient's ability level may then be established as being equal to the maximum ability level at which the patient was consistently able to perform. The goal here is not to challenge the patient, but rather to establish a baseline where the patient is comfortable using the Ul components and applications that have been selected.
  • the kit of Ul components 62 stored in the data storage system 56 of portal computer 55, provide overlapping, and somewhat redundant, functionality.
  • the most simple gestural command made by the patient might be "touch anywhere on the screen.”
  • a more advanced one to do the same thing might be to press a single button labeled with the function the button performs. Higher still would be a menu bar of tabbed choices, with no sub-levels (essentially a row of labeled buttons); higher still would be a menu that provides one layer down of submenu choices, etc.
  • Figures 36a and 36b show how the portal may be used to customize the user experience on the patient device.
  • Figures 36a and 36b represent an exemplary photo-viewing application that has been configured for different levels of ability.
  • each application in this case the photo-viewing application includes a data structure, defined in the local memory of the patient device, that stores the state of all configurable user interface components. Examples of these components are shown in Figures 36a and 36b and include: "previous photo,” “next photo,” “go back,” “slide show state,” “header bar date-time presentation size,” “reminders” and the like. It will be understood that these are just some examples of the user interface components that may be configured.
  • the states of these user interface configuration variables are changed by interaction with the application via the portal. Using the portal, the states of these variables may be changed, and when changed, they affect how the application performs when run on the patient device.
  • the data structure associated with each application also stores data, captured in real time, recording each instance when a particular user interface feature is used by the patient while operating the patient device.
  • a counter is incremented and stored in the usage history portion of the data structure. Date and timestamp information may also be stored in the data structure, recording exactly when the user interface feature was used by the patient.
  • the incremented number represents the number of times a particular feature was accessed within a predefined (customizable) interval of time. As shown, a 30-day interval has been chosen. Thus "231 .30" indicates that the "previous photo” button was activated 231 times in the last 30 days.
  • the data captured in the usage history is sent as feedback information to the portal, and also to the cloud server which collects aggregate statistics, as will be more fully described. Collecting usage history information allows the portal to display to the family member or caregiver which features of each application the patient is able to use. Features that are never used can be switched off, as shown in Figure 36b.
  • Exemplary computer program code run on the portal computer to retrieve settings from the portal computer database and send those settings to the patient device to configure the display of the patient device.
  • private function api_settings($etag FALSE)
  • $display_update empty($this->getDisplay()->update_at) ? NULL : $this->getDisplay()-
  • $calendar_update empty($this->getCalendar()->update_at) ? NULL : $this->getCalendar()- >update_at->format('U');

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Business, Economics & Management (AREA)
  • Biomedical Technology (AREA)
  • General Business, Economics & Management (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Epidemiology (AREA)
  • Software Systems (AREA)
  • Public Health (AREA)
  • Primary Health Care (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Entrepreneurship & Innovation (AREA)
  • General Physics & Mathematics (AREA)
  • Strategic Management (AREA)
  • Human Resources & Organizations (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Data Mining & Analysis (AREA)
  • Tourism & Hospitality (AREA)
  • Quality & Reliability (AREA)
  • Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Operations Research (AREA)
  • General Engineering & Computer Science (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

La présente invention concerne un dispositif patient destiné à être utilisé par des patients dont les aptitudes cognitives ou physiques sont amoindries et qui s'intègre avec un portail réseau conçu pour être utilisé par les membres de la famille ou par le personnel soignant pour personnaliser l'expérience utilisateur du patient. En utilisant le portail, un membre de la famille envoie des personnalisations d'interface utilisateur ainsi que du contenu et des applications à exécuter sur le dispositif patient. Le membre de la famille peut également commander le dispositif patient à distance par l'intermédiaire du portail. Le dispositif patient collecte des données d'utilisation concernant l'ensemble des composants d'interface personnalisables ainsi que le contenu et les applications envoyés. Ces données sont envoyées vers le portail pour servir de rétroaction, permettant ainsi au portail d'indiquer au membre de la famille quelles fonctionnalités de l'interface utilisateur, contenus ou applications sont utilisés ou non. Les données agrégées collectées par l'intermédiaire du portail génèrent des chiffres de classement utilisés pour aider à la sélection des fonctionnalités de l'interface utilisateur.
PCT/US2013/077400 2012-12-28 2013-12-23 Activation d'événement/application dépendant du contexte WO2014105782A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2015550716A JP2016512623A (ja) 2012-12-28 2013-12-23 さまざまな認知能力レベルの人々のためのコンテクスト依存型アプリケーション／イベント起動

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US13/729,960 2012-12-28
US13/730,327 2012-12-28
US13/729,960 US9208661B2 (en) 2012-01-06 2012-12-28 Context dependent application/event activation for people with various cognitive ability levels
US13/730,327 US8803690B2 (en) 2012-01-06 2012-12-28 Context dependent application/event activation for people with various cognitive ability levels
US14/096,475 US9335904B2 (en) 2012-01-06 2013-12-04 Context dependent application/event activation for people with various cognitive ability levels
US14/096,475 2013-12-04

Publications (1)

Publication Number Publication Date
WO2014105782A1 true WO2014105782A1 (fr) 2014-07-03

Family

ID=51022015

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2013/077400 WO2014105782A1 (fr) 2012-12-28 2013-12-23 Activation d'événement/application dépendant du contexte

Country Status (1)

Country Link
WO (1) WO2014105782A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017146958A (ja) * 2016-02-12 2017-08-24 日本電信電話株式会社 ユーザインタフェース情報提供装置、ユーザインタフェース変更装置、ユーザインタフェース情報提供方法、ユーザインタフェース変更方法、及びプログラム
US10892907B2 (en) 2017-12-07 2021-01-12 K4Connect Inc. Home automation system including user interface operation according to user cognitive level and related methods

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030036683A1 (en) * 2000-05-01 2003-02-20 Kehr Bruce A. Method, system and computer program product for internet-enabled, patient monitoring system
US20060066448A1 (en) * 2004-08-04 2006-03-30 Kimberco, Inc. Computer-automated system and method of assessing the orientation, awareness and responses of a person with reduced capacity
US20070197881A1 (en) * 2006-02-22 2007-08-23 Wolf James L Wireless Health Monitor Device and System with Cognition

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030036683A1 (en) * 2000-05-01 2003-02-20 Kehr Bruce A. Method, system and computer program product for internet-enabled, patient monitoring system
US20060066448A1 (en) * 2004-08-04 2006-03-30 Kimberco, Inc. Computer-automated system and method of assessing the orientation, awareness and responses of a person with reduced capacity
US20070197881A1 (en) * 2006-02-22 2007-08-23 Wolf James L Wireless Health Monitor Device and System with Cognition

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017146958A (ja) * 2016-02-12 2017-08-24 日本電信電話株式会社 ユーザインタフェース情報提供装置、ユーザインタフェース変更装置、ユーザインタフェース情報提供方法、ユーザインタフェース変更方法、及びプログラム
US10892907B2 (en) 2017-12-07 2021-01-12 K4Connect Inc. Home automation system including user interface operation according to user cognitive level and related methods

Similar Documents

Publication Publication Date Title
US9335904B2 (en) Context dependent application/event activation for people with various cognitive ability levels
US8803690B2 (en) Context dependent application/event activation for people with various cognitive ability levels
US9208661B2 (en) Context dependent application/event activation for people with various cognitive ability levels
US10652504B2 (en) Simple video communication platform
AU2021215191B2 (en) Systems and methods for remote and host monitoring communications
CN113785295A (zh) 为计算设备配置基于背景的限制
CN108351697A (zh) 包括多个显示器的电子设备和用于操作其的方法
KR20180024628A (ko) 콜 백 알림 서비스 방법 및 전자 장치
TWI695312B (zh) 用於執行功能的方法及裝置
US20170329865A1 (en) Electronic device and method for providing content
JP2016512623A (ja) さまざまな認知能力レベルの人々のためのコンテクスト依存型アプリケーション／イベント起動
US20170099248A1 (en) Systems and methods for generating a queue of messages for tramsission via a messaging protocol
KR102507536B1 (ko) 콘텐츠 정보를 제공하기 위한 방법 및 그 전자 장치
WO2014105782A1 (fr) Activation d'événement/application dépendant du contexte
WO2014106294A1 (fr) Procédé de communication informatique destiné à être utilisé par les personnes âgées
CA3052732C (fr) Moteur de flux de travail pour la gestion de soins de sante d'un patient
KR102288995B1 (ko) 설문 조사 서비스를 제공하는 전자 장치 및 그 동작 방법
WO2018057544A1 (fr) Systèmes et procédés de génération d'affects de conscience à l'aide d'une ou de plusieurs entrées non biologiques
US10516762B2 (en) System for remotely running a service program
TWI474276B (zh) 具關懷快遞功能之遠距居家照護系統及其方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13868966

Country of ref document: EP

Kind code of ref document: A1

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
ENP Entry into the national phase

Ref document number: 2015550716

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13868966

Country of ref document: EP

Kind code of ref document: A1