JP2016512623A - Context-sensitive application / event launch for people with various cognitive levels - Google Patents

Abstract

Patient devices designed for use by patients with reduced cognitive or motor skills integrate with a network-based portal designed for use by family members or nursing staff to customize the patient's user experience. The family uses the portal to push user interface customizations and content and applications running on the patient device. Families can also control patient devices remotely via a portal. The patient device collects usage data for a set of customizable interface elements as well as usage data for pushed content and applications. These data are sent as feedback to the portal and the portal indicates to the family which user interface features, content and applications are being used, and which user interface features, content and applications are not being used Make it possible. The total data collected through the portal generates ranking scores that are used to help select user interface features. [Selection] Figure 1

Description

  The present disclosure relates to a computer-implemented system for assisting a person with reduced cognitive or motor skills in managing upcoming events and maintaining ties with close relatives.

  People with early to mild Alzheimer's disease or dementia suffer from both memory loss and inability to operate complex equipment. These people are always worried about missing events or activities or forgetting other time-sensitive matters. As a result, these people always write a large amount of notes addressed to themselves. The accumulation of those notes results in another confusion. You forget which notes are important and when they are important.

  As mobile devices such as smartphones and tablet devices become widespread, it may be possible to use these devices as an aid for those who have reduced cognitive or athletic ability to remember important events. But this is not as simple as it might seem. Persons with reduced cognitive ability can be very difficult to interact with these mobile devices. First, because cognitive ability is reduced, it may not be understood how to use a particular user interface function or application that runs on the device. Second, as is often the case, people with reduced cognitive abilities also have physical disabilities and may be difficult to operate user interface functions on the device.

  To make matters worse, trying to design a user interface that is well suited to the cognitive or athletic ability of a particular person will not provide a solution that is applicable to everyone. In addition, unfortunately, as often happens, a person's ability declines over time. For example, even a fully adapted user interface at the present time may not be able to meet the user's needs after six months.

  The family hopes to help, but they may also struggle with the complexity of the user interface. You may think that your family has given you a portable device that is useful to an aging loved one, but there is little way to know if the device is actually helping. As is often the case, mobile devices are often unfamiliar to the elderly and are too complex and useful, so they are often left unused in the elderly's room. Even if the family eventually knows that the device is not in use, the family has no effective means to change the situation. This is because typical families are not experts in designing user interfaces for people with cognitive or motor disabilities.

  This section provides an overview of the disclosure and does not provide a comprehensive scope of the disclosure or all of its features.

  The present disclosure provides a solution in the form of a combination of an integrated patient device (for example, a mobile device) and a network-based portal for the above problem. Patient devices are designed for use by patients with reduced cognitive or motor skills. Patient equipment is integrated with a network-based portal designed to be used by family members or nursing staff to customize the patient's user experience. Family or nursing staff use portals to push user interface customizations and content and applications that run on patient devices. Family or nursing staff can also control patient devices remotely via a portal.

  The patient device collects usage data for a set of customizable interface elements and usage data for pushed content and applications. These data are sent as feedback to the portal, which allows the portal to show which user interface functions, content and applications were used or not used to the family or nursing staff. The total data collected through the portal generates ranking metrics that are used to help select user interface features.

  The portal includes an integrated database or kit of user interface (UI) elements, from which a family or nursing staff selects and customizes a customized user interface and user specifically designed for a particular patient You may create an experience. When stored in a consolidated database, each UI element is associated with at least one cognitive or motor ability score that ranks the intelligibility and usability of that UI element for that particular patient. ing.

  If a family wants to customize the user interface for a particular application or even the global behavior of the patient device itself, the family can compare these cognitive and motor skills scores of each UI element for that particular patient. The most suitable one can be selected.

  As the patient uses the device, actual usage data regarding each UI element used and when it was used is collected. From these data, a feedback signal is sent to the portal, which allows the portal computer to take into account this usage data and how various UI elements were actually used (or not used) by the patient. It is possible to decide whether to give the number of used points. The portal computer is also programmed to share usage statistics with a cloud-based aggregation computer that generates total complexity statistics and scores that represent the entire population of patients using the patient equipment. Preferably, these usage statistics are shared with the cloud-based aggregation computer without revealing patient-specific information to ensure patient privacy.

  If desired, the consolidated database may be configured to store similar fitness scores in association with various different selections of content pushed to the device. In this case, if a 60 minute long video was pushed to the device but the patient saw only 2 minutes, the usage data reflects a low level of content suitability for this particular patient Thus, a low fitness score is assigned to the content. This low fitness score indicates to the family or nursing staff that the content selected and pushed to the patient device may be too “difficult” or not interesting to the patient.

  Similarly, the consolidated database may be configured to store similar fitness scores in association with various different selections of applications that are pushed to the device for execution on the device. For example, if an instant messaging application is pushed to a patient device, but the usage score indicates that the application has never been used, a low fitness score is assigned. This low fitness score indicates to family members or nursing staff that the instant messaging application that is selected and pushed to the patient device may be too “difficult” or not interesting to this particular patient. Show.

  Thus, according to one aspect, the present disclosure describes a computer-implemented system for assisting a person with reduced cognitive or motor skills. The system includes a patient device having a display, a processor connected to the display, and a communication port for communicating with a portal computer. The processor of the patient device is programmed to provide feedback data to the portal computer regarding use by the patient of selected user interface elements pushed to the patient device.

  The portal computer has a processor and associated memory, the associated memory storing a plurality of user interface elements according to a predefined data structure that associates a capability score with each user interface element. . The portal computer is programmed to present a plurality of user interface elements to the portal computer user in a presentation arrangement based on the ability score. The portal computer is further programmed to allow the first user to select at least one interface element from the user interface elements presented in the arrangement and then push the selected user interface element to the patient device. Has been.

  Further areas of applicability will become apparent from the description herein. This summary description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure.

  The drawings described herein are merely illustrative of selected embodiments and do not describe all possible embodiments and are intended to limit the scope of the present disclosure. Absent.

FIG. 1 is a diagram showing an outline of a system architecture, and shows a user, a display device, a server type system, and a network. FIG. 2 is a diagram illustrating an example of a message screen displayed on the display device. FIG. 3 is a diagram illustrating an example of a message screen displayed on the display device after confirming receipt of the reminder. FIG. 4 is a diagram illustrating an example of another approach as to where system logic should be placed in the overall system. FIG. 5 is a diagram illustrating exemplary database elements used by the system. FIG. 6 is a diagram illustrating an example of a simplified logic flow for a display device. FIG. 7 is a diagram illustrating the relationship between group messages and different types of users for a particular display device. FIG. 8 is a diagram illustrating an example of a process for setting up a relationship between a group user and a master user for a display device. FIG. 9 is a diagram showing a subset of the system in order to explain the receipt confirmation of the reminder. FIG. 10 is a diagram illustrating an exemplary hardware block diagram for a display device. FIG. 11 is a diagram illustrating an example of a user interface for creating and / or editing a reminder message. FIG. 12 is a diagram illustrating an example of a user interface that a master user sees to review all active reminders and messages for a particular display device. FIG. 13 is a diagram illustrating an example of a user interface that a regular user sees to review all active reminders and messages for a particular display device. FIG. 14 is a diagram illustrating an example of a similar user interface formatted for smartphones and other mobile devices. FIG. 15 is a diagram illustrating an example of a user interface viewed by a group user to view group reminders and messages. FIG. 16 is a diagram illustrating an example of a user interface for creating or editing an instant message. FIG. 17 is a diagram illustrating an example of a user interface for managing parameters for a specific display device. FIG. 18 is a diagram illustrating an example of a user interface indicating an existing preset reminder. FIG. 19 is a diagram illustrating an example of a user interface for selecting a preset reminder. FIG. 20 is a diagram illustrating how the state of the display device can be monitored. FIG. 21 is a diagram showing the configuration of the display and the master user. FIG. 22 is a diagram illustrating an example of a display device displaying several examples of reminder messages. FIG. 23 is an entity diagram showing the basic elements of how an event or application is automatically launched using the disclosed system. FIG. 24 is a high-level flowchart illustrating how cognitive ability is a factor in event or application activation. FIG. 25 is a flowchart illustrating how context is collected and used by various elements in the system. FIG. 26 is a flowchart for explaining a start event flow executed by the system. FIG. 27 is a flowchart for explaining an event activation flow executed by the system. FIG. 28 is a use case diagram illustrating an exemplary use of the system. FIG. 29 is an interaction diagram showing how the interaction level of the system is customized based on cognitive abilities and based on preference and technical context information. FIG. 30 is a diagram illustrating how cognitive ability is modeled by the system when the data structure maintained in the computer memory by the system is reflected in the cognitive ability. FIG. 31 is a block diagram illustrating one embodiment of a tablet-based web-enabled system. FIG. 32 is a diagram illustrating an example of a screen display in which some exemplary applications / events are activated. FIG. 33 is a block diagram illustrating a computer-implemented system and its associated database and data structure. FIG. 34 is a block diagram illustrating how patient devices and network-based portal computers are integrated. FIG. 35 is a diagram illustrating an exemplary display screen of a patient device with an exemplary calendar application. FIG. 2 is a user interface and data structure diagram showing how customizations are pushed to patient equipment and how usage data is collected. FIG. 2 is a user interface and data structure diagram showing how customizations are pushed to patient equipment and how usage data is collected. FIG. 6 shows a comparison display generated by a portal computer showing how usage feedback from patient equipment is displayed. FIG. 6 shows a comparison display generated by a portal computer showing how usage feedback from patient equipment is displayed. FIG. 38 is a diagram illustrating an exemplary display screen generated by a portal computer in which patient device settings are customized. FIG. 39 is a diagram illustrating an exemplary display screen generated by a portal computer where patient device settings are customized and an application to push to the patient device is selected. FIG. 40 illustrates an exemplary display screen generated by a portal computer in which settings for applications for patient devices (eg, music and video applications) are customized so that selected digital content is added. It is. FIG. 41 is a diagram illustrating an exemplary display screen generated by a portal computer in which settings for an audio-video chat application (eg, Skype) for a patient device are customized from the portal computer. FIG. 2 illustrates two different embodiments for displaying capability scores in association with an application or UI element, the display being used by a portal computer for use in customizing a patient's experience when using patient equipment. Presented on the display. FIG. 2 illustrates two different embodiments for displaying capability scores in association with an application or UI element, the display being used by a portal computer for use in customizing a patient's experience when using patient equipment. Presented on the display. FIG. 43 is a diagram for explaining the software architecture of a patient device. FIG. 44 is a diagram illustrating the portal device software architecture showing how the portal device communicates with the patient device API via messaging. FIG. 45 is a diagram illustrating the software architecture of the portal device showing how crash reports are processed. FIG. 46 is a diagram for explaining the software architecture of the portal device showing the updater broadcast receiver. FIG. 47 is a diagram for explaining the software architecture of the portal device showing the wrapper activity start. FIG. 48 is a diagram illustrating the software architecture of the portal device, showing the wrapper activity downloader. FIG. 49 is a diagram for explaining the software architecture of the portal device, showing a wrapper activity crash. FIG. 50 is a diagram for explaining the software architecture of the portal device showing the update of the bundle setting.

Overview The system of the present disclosure provides a reminder message that is displayed by a person at a remote location or a local location (eg, a friend, family member, administrator) at an appropriate time by appropriate messaging on a relatively simple display device. Allows you to create. This display device does not require the viewer to interact with any control elements, so that Alzheimer's people do not need to learn how to operate it. The only interaction required by this display device takes place in a single initialization step and upon receipt of an optional reminder that requires only pressing one button.

  The system operates over a network, such as the Internet and / or a local area network (LAN). People (friends, family, administrators) connect to the system via any modern browser. The system then interacts with the display device via the network.

  The system includes multiple display devices and multiple accounts, and can give multiple people the ability to create reminder messages. Master accounts are also given the ability to edit messages from other accounts and other privileges. In the case of a situation such as a helping home, a group manager account can send a message to a group of display devices or to a single display device. However, an account directly associated with a particular display device can hide such group messages if desired.

  Account owners associated with a particular display device can see each other's reminders, including group messages, so friends and family members can see the scheduled or current activity of the person to whom the reminder is addressed To know about. However, to protect privacy, group account owners can only see their own group messages unless they have permission to view them.

  Messaging can be set up in advance and can be displayed at the appropriate time relative to the events they mention. The content and level of detail of messaging, including voice, varies depending on how close it is to the event of interest. Once the event begins, messaging continues until the event ends, after which the content of this messaging changes depending on what time it is currently in relation to the event end time.

  Reminders can be programmed to automatically repeat at specific intervals (every day to every year) to include various situations and events.

  The reminder may require the viewer to confirm receipt as needed. Multiple acknowledgment requests can be active at the same time. If such reminders are not acknowledged, remote users (friends, family and administrators) may check status and / or receive alerts via short message service or email.

  A preset reminder is provided to reduce typing. Account owners can use system defined preset messages or create their own messages for future use. Account users can customize preset messages.

  The message may be an “instant message” that is not tied to any particular event. Such instant messages are displayed for a relatively short time and do not require any action by the viewer to view the message.

  In order to avoid potential failure situations such as equipment failure, power loss, or loss of communication, the system monitors the health of each display device and, for such failures, the appropriate account owner and / or administrator. Can be warned.

  In one aspect, the system can provide hybrid care depending on the patient's cognitive ability, ranging from fully managed by a third party, to shared control, and to a patient in autonomous activity. The aim is to provide assistance in an automatic and natural manner. Third party control of the system can be local or remote. In addition, the system itself adapts the level of interaction provided to the patient based on further improvements or declines in cognitive ability.

Thus, the system operates to automatically and naturally adapt event triggering (eg, application / event activation on the display) based on the following core functions:
A. Prepare / set events / apps by a third party (eg family).
B. Estimating event context. The context can take several forms (medical form, contextual form, etc.) depending on the event to be initiated.
C. Triggering events based on matching the context's current situation and the person's cognitive ability.

In addition to the core functionality described above, the system further provides a number of additional benefits, including:
The application can start automatically in an unobtrusive manner (for example, a voice message if the patient is not sleeping).
• Activation and application interaction can be customized to the patient's cognitive abilities.
The patient can enjoy a number of services (eg family images, video conferencing or video reminders, music appreciation) without having to know how to launch the application / event.
• System services can be customized to take into account patient personal preferences.
• The solution includes an implicit interaction (from the patient's perspective) to an explicit interaction (from the perspective of the third party preparing the event / application).
Our solution relates to when to launch an application / event and how to launch the application / event.

With reference to the detailed description accompanying drawings, an example embodiment in more detail.

  FIG. 1 is a diagram showing an outline of a system architecture, which is a set of different types of remote users, a server system, and a set of display devices (hereinafter also simply referred to as “displays” or “multiple displays”). It shows. Each user 100, 110 interacts with the system via the network 130. The network 130 may be a combination of multiple wide area networks (such as the Internet) or a local area network. Each user is associated with a particular display 140. In the figure, master user A 100 and associated normal user 110 interact with display A. In addition, there is another set of users associated with the display B or the like.

The user account is centered on the display. There is at least one master user 100 associated with each display. The master user has ultimate control over how the display will look. The master user can do the following:
o Create new event reminders and messages.
o Edit event reminders and messages created by you, or event reminders and messages created by other master or normal users belonging to the managed display.
o Create a new master or normal user for the display.
o Control whether group events are enabled on the display (see group users).
o Hide or show certain group event reminders sent to the display. If this event conflicts with other events planned by the master user (or normal user), it may be necessary to hide the group event reminder.
o Event reminders and messages created for the display can be viewed by anyone who created them.
o Create and edit preset event reminders that others can use.
o Change display details (name, location, time zone, etc.).
o Change your own user details (name, username, email address and password, etc.).

A normal or regular user 110 can place messages on this display, but has less authority:
• Edit event reminders and messages that you create yourself.
• Hide or show certain group event reminders sent to the display.
-Event reminders and messages created for the display can be viewed by anyone who created them.
• Create and edit preset event reminders that others can use.
• Change your own user details (name, username, email address, password, etc.).

Group user 120 may be associated with multiple displays. FIG. 1 shows only one group user, and describes a situation where three displays (A, B, C) to which this group user has access are arranged in a specific facility. The master user can do the following:
• Invite a master user to join this group by sending an email invitation. In order to send such an invitation, the group user needs to ask for the “user name” of the master user. A master user who has received an email invitation to join a group must click on the link to receive an invitation. The master user still has the authority to disable any or all of the group events from being displayed on the display.
Create and edit group event reminders and instant messages that are sent to all displays enabled to accept group events from this group user.
Specify that event reminders or instant messages are sent to only one display (not all displays).
• Group users cannot see event reminders and messages created by master and normal users on either display unless they are given permission to publish specific items.
• Change your own user details (name, username, email address, password, etc.).

  The server 150 manages the system including access to the system by each user and update of each display. Since a single server 150 can manage a large set of users and displays scattered throughout the world, FIG. 1 shows only a subset of what is possible. Database 160 stores all information about all users, displays and messages. Information that requires attention in handling such as passwords and e-mail addresses is kept encrypted.

  In typical operation, a user interacts with the system via a web browser or a dedicated application to create a reminder. This reminder is stored in a database, after which the server determines which reminders should be sent to each particular display at the appropriate time. The user can view the status of all reminders and messages, including editing as appropriate and hiding messages.

  The display only displays a message that they will be sent. They can manage some of these messages as needed to minimize the amount of traffic required during operation. These displays may optionally provide viewers with a simple means of responding to messages when requested (eg, touching the display, replying in words, etc.). Sent back to the server.

  FIG. 2 shows a typical set of messages that can be displayed on a display. If a person has Alzheimer's disease, messaging needs to be simple, direct and appropriate to the situation, since complex messages and graphics can lead to confusion.

  The top row of the display 200 shows only the current date and time. The time zone of the day such as “morning” or “evening” is also displayed. Time and date are automatically obtained from the network. Since it can happen that the user is in another time zone while the display is in one time zone, the time zone of the display is determined by a selection 1710 made by the master user of the display.

  In this drawing, it is assumed that a person is reading from left to right, so the event or reminder “title” 210 is displayed on the left side in the sample display. Of course, this rule can be different for different cultures, and adjustments to how the display is arranged can be adapted to each country.

  The title of the message is intentionally shortened by limiting the length of the title in the input menu 1100.

  The size and font color used to display the message title (and other parts of the messaging) will vary depending on how close it is to the event of interest. As the start of the event (and the end of the event if the event has an arbitrary length) is approached, the font gradually increases and the font color changes to a more urgent color.

  Since the message title may be too short, the second line 210 can add a message or instruction. This second line is optional and can be hidden until the time is closer to the event. The late display of the second line is based on the assumption that displaying too much information too quickly only confuses the reader.

  Additional messaging 240, 250 is added to the reminder to give a clue as to when the event is to take place. Supplemental timing message text is created in a simple colloquial style. It would be too confusing for the reader to say that the event is scheduled to start "11:30 am on April 10, 2011" if the same meaning can be communicated with a phrase such as "about 2 hours later" It will be a thing. The algorithm for how such timing messaging works can be quite complex and needs to be tailored to the viewer's cultural and language style. As always, messaging needs to be kept to a minimum. But if too little information is given, it can also be a problem.

  FIG. 22 shows how more messaging examples are displayed on the active display.

  The sample message “morning medicine” is asking for a response (in this case, pressing the “OK button” 220). Instructions about how to respond can be given in words or other means. In this figure, the OK button is simply a graphic on the display, and the system uses the touch sensitive display 1015 system to sense that this button has been pressed. As will be described later, the status of the response can be monitored.

  FIG. 3 shows a sample display similar to FIG. However, the OK button is different in that it is replaced with a checked box icon 320. This icon (or similar type of indicator) tells the viewer that the message has been executed. Sometimes people forget that they have already done something that they regularly do, such as taking medication. The checked box icon serves as another form of reminder.

  FIG. 4 shows two ways to distribute the logic of the system.

  In the upper version, almost all logic is arranged on the server side 400, 410, and the display 430 is comparable to a thin client such as a browser connected to the Internet 420. Such an arrangement means that a commercial product such as a new tablet computer can be used for the display.

  In this arrangement, the tablet computer is basically used as a browser display. HTML and PHP commands on various web pages determine what to display and when to display it.

  By reading the network time and calculating the next auto-redirect command (header ('Location: page_url.php')), a display refresh that takes place immediately after the start of every minute or at some other selected time is within the web page To be programmed. With each refresh, the display can update the displayed time, get a new message, and update the wording and font of the currently active reminder message. If requested, the voice may be played by a command found in HTML5 or an alternative.

  The lower version places part of the messaging logic on the client side 440. Information about future events may be stored in the display's local database 460. An algorithm located in the display system can then determine what to display at a given time without having to communicate with the server system. The display still needs to communicate with the server periodically to get message updates, but the frequency of such communications can be lower. Most of the logic of the system, particularly the master, normal, group interfacing, account management and general system management parts, is still located in the server 450.

  Implementation can be done in a number of ways. In one version, the software code can be effectively downloaded to a display browser using a language such as AJAX.

  As another version, the display may include a software application that, if present, resides in non-volatile memory. This software can be made to run automatically when the display is first turned on. This means that power interruption and communication interruption can be handled automatically.

  FIG. 5 shows typical database 500 table categories used in the system. During operation, the algorithm stored on the server accesses the following database tables to determine how to handle each display, user and situation.

  The table for display 505 includes information about each single display, such as the name and time zone associated with the display. The table for user 510 contains information about each user, including name, contact information, password, and user account type. Users appearing in this table are associated with a display or set of displays (if they are group users). Message table 515 holds all messages, including information about when and how each individual message should be displayed, information about who created the message, and information about the type of message. . These three tables contain the core of the database used by the system.

  In addition to the core table, there are a number of important supplemental tables. The display check table 520 is used to store the tone of each display. The preset table stores predefined messages that can be used to reduce typing to some extent. These preset messages contain most of the same information as the regular messages stored in the message table. The group request table is used for storing a request for joining a group made by a group user to a master user. The group hide table is used to store information that determines whether a particular group message should be displayed on a particular display. The OK button table stores the response status for each message requesting such a response. The instruction table is used to store localized (different language) instructions and language for the user interface. The image table is used to store images that can be associated with a particular message. The audio table is used to store an appropriately formatted audio file that can be associated with a particular message or situation.

  FIG. 6 first shows how the algorithm and database table work together to manage all displays.

  Periodically, the database 600 is consulted to determine which message is currently active (610). A message is active if an entry in the message table indicates that the message should be displayed based on the current time zone, date and time 650.

  Next, if a particular message comes from a group user, the group hide table is accessed (615) to determine if this message should be displayed on this particular display.

  Next, the OK button table is accessed (620) to determine if a response is requested at this particular time. The message may be displayed without requesting a response until a predetermined time before the event begins. Therefore, for example, the viewer can know that the event is approaching. However, no response from the viewer is required until the event is about to take place.

  Next, based on the parameters stored in the message table and other tables, the exact wording and font selection is compiled (625). Whether messaging is tailored to suit each situation is more of an art domain than science, but the key element of this disclosure is that such coordination is integrated with the system. is there.

  Next, if there is any audio associated with the message or status, the audio table is accessed and compiled into the message as appropriate (630). Similar to the wording and font previously selected in the previous step, the audio can also be adjusted.

  Similarly, if there are any graphics or images associated with the message, these are also integrated into 635. Again, adjustments according to the situation can be made.

  Finally, the completed compiled message is represented on the display (640). This includes any text, audio and / or images that were determined to be part of the message in previous steps. The display and messages are then refreshed based on a refresh timer as needed.

  FIG. 7 shows a portion of the relationship between group user 700, master user B 715, and regular user B 720 when the message is placed on a particular display.

  The simplest situation is when the master user wants to place message B1730 on display B760. This message is allowed because display B is managed by this user. Other displays on the network, such as display A, display C770, and display D780, ignore message B1. Similarly, user B can place message B2735 on the same display B. This is because this user has been given authority to do so by Master User B.

  Master user B also has the ability to edit or delete message B2 created by regular user B. Regular user B can also edit message B2, but this user cannot edit message B1 created by master user B.

  Master user B and regular user B can see all messages addressed to display B, regardless of whether they are currently displayed on this display.

  In this figure, the group user is shown as creating two group messages 705, 710. These group messages may be sent to all displays 760, 770 belonging to this group, but for displays 780 that are not part of this group, the display 780 is on the same network. Even if it is not sent.

  If the group message is addressed to any display, the master user and regular user associated with that display can see this message. If either a master user or a regular user determines that a group message conflicts with a scheduled event, the user has the ability to hide this group message. Since individual group messages may be allowed to be displayed or hidden, group message 1705 may be hidden (745) separately from group message 2710 (750) in a hidden state. The determination by the master user B and the regular user B does not affect what is displayed on the other displays 770 and 780.

  FIG. 8 shows a process flow for determining whether a particular display is part of a particular group. Since control of the display belongs to the master user of the display, the group user must first request the user name of the master user (800). If the master user agrees (810), the group user can send an invitation to join the group by email to the master user (820). The email contains a special link with an encrypted key that, when clicked, brings the master user to a web page that displays the group that the display has just subscribed to (830). In this regard, the group user's group message is displayed on the display (840) unless the master user decides to remove the display from the group (850) or hide the particular group message (860). ). Normal users can also hide individual group messages (as in step 860), but cannot exclude displays from the group.

  FIG. 9 shows a variation of FIG. 1 and is used to explain how the OK button or receipt confirmation system operates. First, a reminder message is created by the master user 900, regular user 910 or group user 915 specifying the need for confirmation of receipt by the viewer of the display. Messages are stored in the database 980 and presented to the display 940 at the appropriate time (970). Thereafter, at a particular time, an OK button is displayed (950) with other linguistic or visual prompts. Alternatively, the external device 960 may be activated to request some kind of action. The requested receipt confirmation is then made by the viewer and logged in the database 980. Various users can then see whether the receipt has been confirmed via the web page. Alternatively, the server may send a short message (SMS) or email or make a phone call.

  FIG. 10 shows a typical hardware block diagram of the display. Display includes processor 1000, memory 1040 for instruction operations and variables, non-volatile memory 1045 for BIOS, operating system 1050 and application 1055, power supply 1060 and optional battery 1065, display 1010 and optional touch panel system 1015, and , Consisting of a typical set of elements including networking (wired and / or wireless) 1030 to connect to WAN / LAN 1035.

  This display can be a stand-alone product or part of another product. For example, the display can be built into a television. In that case, the touch panel user interface may be replaced with a remote control configuration. Since almost all of the other elements are already part of today's television, these elements can be shared and exploited.

  FIG. 11 shows an example user interface screenshot for entering or editing a reminder message. This can be part of a web page or part of an application.

  This example begins at 1100 where a message title or headline is entered. There is also a place 1105 for inputting the second line of the description. Although the number of lines can be increased, this drawing limits the description to only these two parts and keeps the message to the viewer simple.

  It may be more useful to change the message once the event has started. For example, the message may be “Your Birthday soon” on the day until the birthday, but may be “Happy Birthday” on the day of the birthday. To provide this option, a second set of message titles and notes is provided (1110).

  Each reminder message is then given a start date and time 1120. Certain events, such as holidays and birthdays, are actually events for the day itself, so the event may be designated as “all day” (1125).

  If the event is not an all day event, the next thing to specify is how long the event will last (1130). If the event lasts less than a day, the length of the event can be specified in minutes or hours. If the event occurs over multiple days, the end of the event may be defined by specifying a specific date and time 1140.

  Next, it may be determined (1150) when the event begins to appear on the display. The timing to start displaying an event is highly dependent on the type of event and user and viewer preferences and is not tied to the length of the event.

  If necessary, an audio reminder may be played to draw attention to the event. When to start playing such voice messaging (1160) can be specified independently of when to start displaying events. However, voice messaging should not begin until the message is visually displayed. The type of voice messaging may be selected independently (1165).

  A check box is provided that can be checked by the user when confirmation of receipt of the reminder message is requested (1170). In addition, if the user wishes to be alerted if no acknowledgment has been made after a certain period (by the end of the event), another check box 1175 is provided for that purpose. .

  If the event repeats to some degree predictably, the system allows the user to specify how to repeat this event (1180). Multiple recurring options from daily to yearly and several options in between can be provided. Unlike the calendar system used in PC, PDA and phone systems, only one repeated event is shown at a time to avoid confusion by the viewer of the display.

  FIG. 12 shows a typical user interface that a master user sees to review and manage all messages that are scheduled to be displayed on the display. This can be seen on a web page or can be part of an application.

  The interface is displaying which display is being displayed (1200), and other supplemental information such as the time at which the display is located, and whether this display has been enabled to accept group messages (1205). ) Is displayed.

  A button (1210) for displaying information in a format that is easier to use on a mobile device is provided (it is also possible to automatically switch to this mode). A button (1215) for adding a new reminder message or a button (1220) for adding a new instant message is provided. A button (1225) is provided for confirming how the display itself looks at that moment. In addition, a button (1230) for displaying help and a management function with low usage frequency is provided.

  The main table shows a summary of all active events currently listed for this display. The columns of the table are the title and notes (1240, 1245), when the event begins, and information about what the display should do at various times (1250), when the event ends, and Information about whether or how to repeat the event (1255), information about when to start displaying the event, or whether the event is currently displayed on the display (1260) Is shown. Information is also shown regarding whether an acknowledgment is required, whether an acknowledgment has been made, and whether a warning should be issued if no acknowledgment is made (1270). The last column shows who created the message (1275) and an edit button if the message is a message that this user can edit (1280). Since FIG. 12 is for a master user, this user has edit authority for any message created by another master user or a regular user. If a particular message was created by someone else, the edit button may be configured to look a little different (1280).

  A flag that is displayed when two or more events overlap or conflict is not shown here. Since different users can try to place event reminders on the same display, it is possible for one user to accidentally create an event that conflicts with another, so that such a conflict may be displayed in some way. is important.

  For group messages, instead of an edit button, a button is provided that is used to hide or display the group message. In this case, an open eye 1285 is displayed if the message can be viewed, and a closed eye is displayed otherwise. The specific implementation of this functionality may vary depending on user interface preferences.

  FIG. 13 shows a similar drawing for managing reminder messages. Only this figure shows how it looks to normal users. Since the normal user can edit only the message entered by himself / herself, the edit button is displayed only for a subset of the enumerated messages (1300), and the edit button is not displayed for a message created by another person (1310). . Since the normal user can hide and display the group message, a button for doing so is also displayed (1320).

  FIG. 14 shows how these interfaces look to the mobile device. Only a portion of the entire interface is displayed and the rest can be viewed by scrolling or paging. Means for viewing the “full version” interface are also provided (1400).

  FIG. 15 shows how the interface looks to the group user. Since the example used had only one group message, only one message 1500 is shown in this table. Unlike the master user and the normal user, the group user can edit the group message, so an edit button is displayed.

  FIG. 16 shows an exemplary interface for sending instant messages to the display. A place is provided for the message title 1600 and the second line 1610 for details. Also, means for specifying how long the message is displayed is provided (1620).

  Unlike typical instant messaging used on telephones or PCs, the display type viewers described in this disclosure are more passive viewers. No action is required by the viewer to display the message on the display, but there is no guarantee that this person will always notice the message. An audio notification may be specified (1630) to call attention to the message. Alternatively (although not shown here), a message may be created to request receipt confirmation, similar to the message described above.

  FIG. 17 shows a part of the management function of the system (in this case, management of a display (name in the figure is “frame”)). Each display can be given a name 1700. The name is usually the name of the person viewing the display. Next is a means for designating the time zone of the display (1710). Alternatively, time zone information can be obtained via a network to which a display is connected.

  Next, a location description in some form, such as a city or room number, may be specified (1720). The combination of display name and location helps to uniquely identify each display.

  If this display is OK to accept a group message, a check box 1730 is provided for that purpose. This check box is automatically checked when the master user clicks on an email invitation (830), but can be unchecked or checked again at any time thereafter.

  FIG. 18 shows an interface for managing preset reminders. Several preset reminders have been defined by the system and are shown here as coming from the administrator (1820). Some presets may be defined by a master user, normal user or group user. If the user has editing authority (editing authority according to the same rule as a normal reminder message), an editing button is displayed (1810).

  FIG. 19 shows a simple means for selecting a preset. Once a preset has been defined, when creating a new reminder message, the preset can be selected by clicking on a preset button (not shown but will appear on the same interface as shown in FIG. 11) (1900). Once a preset is selected, it can be subsequently modified or customized. Thus, the user is not tied to a specific set of parameters, date, time, etc.

  FIG. 20 illustrates low level operation designed to monitor the tone of a given display set. The display can be accidentally turned off, lose power or communication, or have a hardware failure. Since the display usually does not exist near the person who manages it, it is necessary to provide a means for obtaining some indication of the tone of the display.

  To that end, the display first sends a periodic “keep-alive” signal 2000 to the server via the network 2020. The period of the survival signal can be preset (2010) and need not be too frequent depending on the needs.

  The server ("system" in this figure) accepts the survival signal from all displays that the server is monitoring. If one or more of the displays did not send a survival signal (2040), a warning may be sent (2050). Alternatively, the web page may be updated to include the name and location of the suspicious display.

  Meanwhile, the user 2060 can view the status of the display and / or receive alerts even though they are not near the display.

  FIG. 21 shows a low level management function, display and other setup of the master user.

  You need to create a unique account for each display. This account may be created (2100) by the system administrator. This administrator may be a service provider or a person at the factory. If the display is a unique device specifically configured to operate with this system, a unique account code (possibly generated algorithmically) may be stored in the display's non-volatile memory. If the service provider is creating a display account, various means may be used to create a unique code. Once created, these unique account codes are also stored in the system databases 160, 505.

  Next, the system administrator creates a new master user account. This account consists of a unique username and a pointer to a specific display. A password (preferably a unique password) is also generated. Also, if a display dedicated to this system is manufactured, the setup of the master user can be done at the display factory. Alternatively, the service provider may create master user account details. Either way, once created, this information is also stored in the same database 160, 510.

  The new master user is then given a new username and password. The master user then logs into the system (2120). Once logged in, this person can create a new reminder message (2130) and can create other users, etc., as described above.

  Before or after this step, the master user installs the display intended for use (eg, Alzheimer's). Installation includes logging the display into the system (2140). Logging in to the system involves two steps. The first step is to establish a network connection. This connection can be accomplished in many ways, depending on the specific type of network connectivity used. Connectivity can be achieved via various wired (eg, LAN over cable, modem over telephone) or wireless (eg, Wi-Fi, cellular, Bluetooth) means. For example, if there is an existing Internet service available via a Wi-Fi connection, the display must first establish a link to this Wi-Fi.

  The second step for logging the display into the system is to have the system recognize the previously established display unique identification code (2100). This step can be performed manually or automatically by the display.

  When executed manually, the screen on the display requests account login information for the display, such as a username and password. A user may use any of a variety of input devices (eg, touch screen, remote control or keyboard) to enter the requested information.

  When executed automatically, the display reads its unique identification information from non-volatile memory and provides this information to the system. Once the requested information is provided to the display's non-volatile memory, automatic login of the display can be done by either the factory or the master user.

  The password is specifically encrypted before being given to the server. Encryption is necessary for privacy protection.

  FIG. 22 shows a prototype display device. This display device is installed in a stand so that it can be placed on a table. Alternatively, such a display can be built into the wall or part of another device such as a television.

  Note that, similar to FIGS. 2 and 3, messaging is adjusted to match the current time for each event. For example, “Jerry and dinner” is displayed as “2 days later” (2210). This is 6pm on Tuesday (2215). My birthday is “3 days later”. Since this birthday is an all day event, no time is given and only "Wednesday" is displayed (2220).

  The “Visit Alice” event displays a little more detail (2230). This event is a multi-day event, which starts on “December 17”, “after about one week”, and lasts for three days (2235).

  Each of these messages changes automatically over time, depending on how close it is to the event and whether the event has started or just ended.

  Since the picture was taken during the day, the illustrated sample display has a white background. To reduce the possibility of disturbing someone sleeping, at night the display background will be black and the font color will be adjusted accordingly to make it easier to read. The timing at which the display goes into night mode can be arbitrary, can be set by options selected by the master user, or the display can be globally determined as determined by the geolocation of the IP address detected by the display. It can be adjusted automatically depending on where it is located.

  Referring now to FIG. 33, a computer-implemented system for assisting a person with reduced cognitive ability in managing upcoming events will be described. A computer-implemented system is indicated generally by the reference numeral 10 and includes a computer system 12. Computer system 12 may be implemented using a single computer or a group of computers networked to handle the various functions described in a categorical manner herein. The computer system 12 manages the electronic database 14 and, if necessary, an analysis system used to analyze data stored in the database 14. Database 14 functions as a data storage device configured to store a plurality of information items related to time-dependent events (and other context-dependent events) for the patient. The analysis system, for example, analyzes trends in the cognitive ability of a particular patient, adjusts the system's performance to fit the patient's cognitive ability, and provides feedback information about the patient to the patient's caregiver It may be programmed to be provided to a person.

  Several different presentation devices may be used for a single patient, if desired. For example, one device may be a tablet computer operated by a patient, and the other device may be a wall-mounted television display in the patient's room. The system can dynamically control which device is used to interact with the patient. In some cases, both devices may be used simultaneously. The system can customize the presentation sent to each device individually. Thus, the level of complexity for a television display may be different from that used for a tablet computer in a given situation. The system can use the context information and even the patient's cognitive ability to adapt each display as appropriate to the patient's needs.

  The computer system 12 may also be programmed to generate a memory game that is provided to the patient. Accordingly, the memory game generator 16 is illustrated as being connected to a computer system. It will be appreciated that the generator may also be implemented to program and make available an appropriate memory game based on the patient's cognitive abilities by programming the computer system 12. Memory games can greatly help to make a patient's memory work and can slow the progression of the patient's illness. In addition, information about the patient's current cognitive ability is collected using feedback information automatically acquired when the patient plays the game, and this collected information is used by other systems. This will be described in more detail below.

  Computer system 12 preferably allows third party application developers to develop software applications and physical devices that interact with system 10 (eg, reading data from or writing data to a database). , Further including an application program interface (API) presenting a set of standardized inputs / outputs and their associated communication protocols.

  Computer system 12 includes a web server 22 through which caregivers 26 and patients 28 interact with computer system 12. In this regard, web pages are provided for viewing and interaction by computer devices such as tablets, laptop computers, desktop computers, smartphones, and the like. The computer system 12 may be connected to a local area network (LAN) 24. The LAN 24 allows other computer equipment to communicate with the computer system 12, such as, for example, a workstation computer terminal utilized by a nursing home staff 30.

  Database 14 is configured to store data constructed according to a predefined data structure that facilitates the provision of services executed by the computer system. The database includes a data structure 32 that stores a plurality of information items (information content), each associated with a set of relevant context attributes and associated triggers. For example, one item of information content may be a reminder message that the patient is making an appointment with an optometrist. Associated with the message may be trigger data indicating when the reservation is scheduled. Also associated with the message may be other context attributes such as how large the message should be displayed based on what device the message is being viewed on. See FIG. 32 as an example of the display of this message.

  If the reservation is still in time, the information content stored for the event may include a very general text reminder and is stored in the data structure 32 as one record. As the time of the event approaches, the system may provide more detailed information about the event (such as a reminder to “take your old glasses”). This is stored as a second record in the data structure. The system selects the appropriate item of information by selecting the one that fits the current context.

  In this regard, the system also stores the current context for the patient, such as the patient's location and any relevant medical condition attributes, in other data structures. These are illustrated as context data structure 34. Further details of context attributes are described below. Computer system 12 uses the current context attributes in structure 34 in determining which information content to obtain from structure 32.

  In addition to the patient's current context, the computer system further maintains a cognitive ability data structure 36 that stores data indicative of the patient's cognitive ability. This may be quantified, for example, as a relative value suitable for expression as a sliding scale (eg, a scale of 1-10). A patient's cognitive ability may be assessed by explicit input by a caregiver or nursing home staff. Alternatively, the system can establish the cognitive performance data itself via feedback from the memory game generator 18 or by analyzing how well a patient can generally interact with the system. .

  In certain embodiments, the system automatically routes specific applications and events based on set parameters set by third parties taking into account specific information such as patient context, technical context, and contextual context. to start. FIG. 23 illustrates the key considerations that are considered in the process of determining which application / event to launch, when to launch it, and how to launch it. If the context information satisfies the parameter setting, execution of the application and / or event is started. This provides some information or interaction that an individual sees or uses on a computer terminal, such as a tablet computer. The system also adjusts the level of dialogue ability based on the patient's cognitive ability. The goal is to provide a patient or user with an unobtrusive automatic means of obtaining relevant and sometimes necessary information and services.

  In FIG. 23, a third party is an individual or group that is generally involved in nursing care in at least some form. Such third parties may have authority for entering reminders, initiating video conferences, uploading images, setting appointments, and other functions of remote care. As used herein, the term “caregiver” refers to such a third party and may include family members, doctors, nursing staff, and the like.

As shown in FIG. 23, the context also provides useful information. The system can initiate some of the events / applications with knowledge of other factors that the caregiver may not be aware of. These include current status at the nursing home or other patient center (current status detectable by camera, microphone, nurse / doctor input, medical sensors, etc.), active / available technical information (eg, human Not to send a reminder to the wristwatch and display it on the TV), as well as medical information (medical sensors, current physician reports, data obtained from user status reports).

Cognitive Ability As shown in FIG. 23, the patient's cognitive ability also forms an important aspect of the system. The patient's cognitive ability is the current level (on a rating scale) of the ability of the patient to interact with the electronic system, tablet or other device in the system. If the score is high, the patient is probably able to interact with the device himself and may not require much assistance from any context or third party assistance. If the score is low, the system and third parties may provide more assistance. The cognitive ability scale and how it is determined is further described below with reference to FIGS. 29 and 30.

  A computer-implemented system obtains and stores electronic data records that indicate the patient's cognitive ability. In one embodiment, the electronic data corresponds to a collection of individual skill measurements or evaluation values (skill variables) each represented by a numerical value in an appropriate range, such as a range of 0-10. If desired, an overall cognitive ability score or total rating may be calculated and stored based on individual measurements or ratings.

  The dynamic rendering system uses these skill variables to represent facts in the most appropriate manner based on the patient's skill set. In this embodiment, the collection of skill variables stored in the computer's memory corresponds to the patient's overall “cognitive ability”.

Skill variables include sets that can be static or dynamic. Some variables are measured or evaluated by a human operator, and other variables are automatically evaluated by the system based on historical observations and sensor data. The following is a list of skill variables used by the system. In this regard, those skilled in the art will appreciate that the system may not require all of these variables, and that there are other variables that fall within the scope of this disclosure but are not listed here.
・ Concern level ・ Visual impairment or skill ・ Short-term memory skill ・ Long-term memory skill ・ Recognizing and remembering names / faces of familiar people ・ Reading skills ・ Attention skills ・ Time and space detection ・ Speech skills ・Listening skills • Ability to solve simple logical problems • Reasoning skills (ability to understand the usual implied consequences of movement and facts)

If desired, these skill variables may be combined algorithmically by a computer system to derive a single value “cognitive ability” score. A suitable scoring mechanism may be based on the clinically recognized stages of Alzheimer's disease shown below.
Stage 1: No obstacle Stage 2: Decline very slowly Stage 3: Decrease moderately Stage 4: Decline moderately Stage 5: Decrease moderately serious Stage 6: Serious decline Stage 7: Extremely serious decline

In addition to cognitive ability, the context system also considers context information relevant to the patient. FIG. 24 shows a high-level flow chart of this context-dependent application / event launch for people with different cognitive abilities. (Step 1) A third party (eg family, friend, caregiver) enters information and sets parameters for starting events and applications. (Step 2) Once the system is prepared, the system collects and stores context information. Such context information relating to events or the like preferably consists of three sub-contexts: patient-related information, status / external information, and event / application / device information. (Step 3) If the context meets the prepared settings of the system, an event can be started. (Step 4) If started, the system launches the application while customizing the interaction level for the patient.

In some embodiments, the context of an event (an event such as an application or task) may consist of three sub-contexts: a patient-related context, a situation or external condition context, and a technical context. The state of these contexts is stored in a context data structure in the memory of a computer that forms part of the system.

Patient-related context The patient-related context includes (but is not limited to) all information available from the patient. This information is stored as data in the context data structure. Examples of patient-related context data include:
・ Medical context obtained from sensors (eg vital signs)
・ Digital medical records (history)
• Patient behavior (eg sleeping or not sleeping)
The patient's location (eg, watching the display indoors)
• Patient preference (eg voice start / notification preference, sound, video, TV program, image preference)
・ Request of family / caregiver

Situation or external condition context A situation / external context includes (but is not limited to) all information available to the patient from an external source. This information is similarly stored as data in the context data structure. Examples of situational or external condition context data include:
・ Weather information ・ Time ・ Third party information (eg identity)
-TV viewing and broadcast programs-Other people in the patient's room

Technical Context The event application / equipment context contains all information available from the equipment that makes up the system. This information collected by communicating with the device itself is likewise stored as data in the context data structure. Examples of technical context data include:
• Tablet display device status • Amount of available bandwidth • Display type (eg size)
-Network type-Other available devices (smartwatch, in-room TV, other system components)

  The technology context is useful because different devices can be added to the network in the future to add additional functionality. For example, if a patient or patient caregiver purchases a “help me” necklace, a new TV, or a digital picture frame, the system may recognize a context that includes these new technologies (eg, instead of a master tablet device) Allows the system to modify its behavior, such as displaying an image on an image frame).

Each event / application is started with a specific context (the most common subset of contexts).

Polling Context Information FIG. 25 shows the polling context information that the system collects from step 2 of FIG. As described above, information is classified into three categories. Patient-related information, such as medical records, must be manually entered into the system database by a third party. However, current medical information is constantly collected by the system via fixed and portable sensors.

  Once all context information has been collected, the system analyzes the data and determines whether an event is initiated based on parameters set by a third party (FIG. 26). If it is determined that the start context is satisfied, the system evaluates whether the event is triggered. For example, if the schedule reminder is off at a particular time, the system needs to determine whether the patient is in the room. The system will not trigger an alert if the patient is not in the room. However, if it is determined that the patient is in the room, the system initiates the event. Similarly, if the family wants to make a video call, but the context indicates that it is the current nap time (based on priority ranking) or the doctor is currently being treated, the system will not trigger a warning or event . Similarly, if the patient is still supper and the medication is to be made after dinner, the system will not trigger a reminder for the medication.

Other examples of start events include:
-A person falls.-Timed events (eg medication / reminders)
・ Sleep, meal, blood pressure ・ Initiated by a third party ・ Entering another person ・ Voice command

  As shown in FIG. 27, depending on the patient's cognitive ability, the method of obtaining the patient's attention may be more pressing and obvious. A warning attracts the patient towards the tablet display screen, at which time the camera detects the presence of the patient. If the presence is recognized as a patient (identification can be performed by facial recognition, voice or other electronic identification system), the system launches the application / content seamlessly. Upon triggering the event, the system again considers the patient's cognitive level and appropriately adjusts the level of interaction ability and the need for interpretation. For example, if the patient is receiving a video call, the system alerts the patient. If the patient's function is complete, the system displays an option for the patient to accept or reject the video call. For patients with low cognitive levels, the system authenticates the patient's identity before automatically initiating the video call, or, as shown in FIG. It can start automatically.

  In customizing the patient interface, the system takes into account several factors (FIG. 29). First of all, the patient's cognitive ability needs to be entered manually by a third party. However, as the patient continues to use the system, the system will record and adapt to the patient's skill history from previous interactions. The system can also provide a minigame specifically designed to test current cognitive abilities, so the system can automatically update how it should interact with the user .

  In addition, the system applies the preferences of patients and third party individuals (eg, doctors, caregivers, family members, friends, etc.). Thus, the patient is not difficult to use or interpret system events. The system interface also varies depending on patient preference and cognitive level. Ranging from simple and automatic interfaces for cognitively (or technically) incapacitated people to more complex and manual interfaces for independent and cognitive patients (Figure 30).

  Adjusting the actual use of the application (including not only launching the application, but also changing the user interface, buttons and / or interactive mode) based on the determined cognitive ability factors makes the application useful to the patient Is important to do.

  Some embodiments require manual input of the patient's cognitive level, but the system adapts to the patient by having the patient perform a test in the system. This embodiment may be configured for a cognitive level that changes daily. System parameters can be routinely adjusted by a third party. Alternatively, the system can be configured to perform tests daily or weekly or bi-weekly to automatically respond to patient needs.

  System customization is not limited to events that are triggered when a context is started. The system may trigger an event when it is dormant. For example, if there is no event set to start but the patient is observed to be in the room, the system tablet display may go dark, display an image show, or become a reflector Good.

  FIG. 31 shows that the system is connected to a tablet screen, an event database, a plurality of web applications, RF communication (eg, Bluetooth), and a large number of devices (eg, a camera, a microphone, a speaker, a computer, a portable device, etc.). 1 illustrates an embodiment including a connection. The system also displays a web API. FIG. 32 shows an example of a user interface including activated applications / events.

Additional displays may be provided to the caregiver or other third party as needed. This additional display, which can be implemented on a tablet or smartphone, provides the caregiver with information about what the patient is trying to do and information about what the patient has done in the past, as well as feedback about the patient's medical condition . This feedback loop provides caregivers with reassuring information. In general use, this additional display presents information that is different from the information displayed on the device used by the patient. Like the information presented to the patient, the information presented to the caregiver is derived from information stored in the database system. The database system may be supported by a server associated with a nursing home or healthcare provider, or may be supported by a service provider that provides services using internet-based or cloud computing resources.

Additional Descriptive Use Cases To further illustrate some of the possible uses of the disclosed system, consider the following use cases enabled by the disclosed computer-implemented system:
• Video communication initiated by a remote third party that is activated when the patient is viewing the display. An audible signal can alert the patient and direct the patient to the display.
An application initiated by a medical condition based on data read from a device connected via Bluetooth communication (for example, to a patient display that is sent when the read data indicates that the patient is dehydrated) s message).
An application initiated by an understanding of patient behavior (eg, patient is in the room and not sleeping).
• Operation based on a predetermined time of the day.
• Messages that appear on the display in response to external conditions (for example, advise the patient to wear a warm jacket based on what is known to be cold outdoors, or emergency messages and what to do in case of fire) Display what to do).
An image that appears on the display when the display does not display other information (ie, the technical context does not indicate a conflict and the patient's context indicates the time of day that the patient may enjoy the image).

Priority and conflict resolution If several applications are prepared, a conflict can occur if several contexts are met simultaneously. To solve this problem, each application / event is assigned a priority level. In the case of the same priority, the first registered in the queue is executed.

With reference to FIG. 34, the outline | summary of integration with the patient apparatus 50 and the network-based portal computer 55 is demonstrated here.

Patient Device The patient device 50 having the display 200 is generally configured as described above and provides services to persons with reduced cognitive and / or athletic capabilities. The patient device has a communication port, such as a WiFi wireless communication port or a cellular data communication port, thereby allowing the device to communicate over the Internet. Alternatively, information displayed on the device can be transmitted to the TV 52 in the patient's room. The TV 52 may be equipped with computer network communication capabilities so that the TV 52 can communicate directly with the portal computer 55. Alternatively, the television 52 may be configured to operate as a display that displays the information displayed on the patient device 50 as it is. In the latter case, the TV 52 is in communication with the patient device 50, and the device 50 processes communication with the portal computer 55.

FIG. 35 shows an exemplary screen displayed on the display 200 of the patient device. In this case, an exemplary screen is generated by a calendar application running on the patient device.

Method for Programming Patient Equipment Patient equipment may be implemented using a variety of different hardware and software architectures. A tablet form factor is currently preferred.

  In certain embodiments, the patient device may be implemented using a tablet computer that uses an iOS operating system, Android operating system, Windows Surface operating system, or other systems. In such an implementation, the patient device may be implemented as an application (app) that runs on a native device operating system (e.g., iOS, Android, etc.).

  In another embodiment, the patient device executes the patient device software as a stand-alone application that includes a hardware interface layer, where the stand-alone application sends an image to the display 200 via a touch screen device associated with the display. Implemented using a tablet computer that will be able to respond to touch interface commands.

  The patient equipment uses the software architecture shown in FIG. With this software architecture, the processor of the patient device is programmed to perform the functions described herein. Of course, different architectures may be used. As shown, the software architecture may be built based on a native device operating system 231. Updater 213 is a dedicated software component that connects to portal 55 at a predefined time frequency to look for new wrapper 215 software releases.

  The wrapper 215 is dedicated middleware that connects the low-level native device operating system 211 to the UI and application rendering packages. The wrapper 215 implements a device API for synchronizing the local buffer with the remote portal 55. These APIs include device authentication, activity, message, image and video downloads, user settings and customization downloads, and latest bundle 217 inspection and downloads (including UI design and device specific applications) including.

  The wrapper 215 executes code and information provided via the bundle 217. The final UI displayed on the device is the loaded bundle 217 information, as well as zoom level, enabled / disabled apps, volume level, enabled / disabled speech synthesis (Text-to-Speech) Fully customizable according to device-specific settings such as favorite applications, new event prompt options, etc.

  The bundle 217 contains all UI and application specific code. The system allows very targeted bundle distribution for each display. This allows customization of all user interface aspects such as fonts, creation of new applications, resizing and redistribution of all graphic components.

  Thus, the bundle includes all event queries and display logic. The wrapper initiates the bundle, updates the bundle, and provides functions related to the operating system to the bundle (eg, speech synthesis, reboot, WiFi restart). The updater allows the wrapper to be updated and starts the wrapper.

When implementing this updater-bundle-wrapper configuration, an embodiment based on the Android operating system may be configured as follows.
Updater: Updater is manually installed on patient equipment. Updaters do not provide a visible user interface and usually do not require updates. The updater may be initiated or activated when the patient device is booted, or may be initiated by the user via an appropriate touchable icon displayed on the patient device screen. Once started, the updater starts the wrapper (if available) and checks for wrapper updates every predetermined time interval (eg every 10 minutes). If a network failure occurs due to the absence of an installed wrapper, the updater rechecks at more frequent intervals (eg every minute). The updater installs a new wrapper.

If the native operating system is an Android operating system, the updater may be implemented using Android APK. The Android service is scheduled to run every 10 minutes by its alarm manager. Also, the execution of the service is scheduled and a wrapper activity is started using Android APK. In addition, using APK, a receiver handling on boot message and a starting activity are broadcast.
Wrapper: The wrapper is automatically installed by the updater. It has no visible icons and is always started by an updater. The wrapper checks and installs the new bundle. The wrapper periodically performs system restarts and periodically performs WiFi restarts. Performing these steps ensures that the patient device is forced to re-establish the connection if the network connection is lost for any reason. The wrapper is also responsible for sending a crash report message to the portal computer, as shown in FIG.
Bundle: The bundle is automatically installed by the wrapper. The bundle supports protocols such as HTML5, CSS, JS App. Unlike updaters and wrappers, bundles can be viewed on patient devices in the form of calendars, for example. The bundle uses an event API and handles the display of all events. The bundle also notifies the wrapper of any display setting changes.

The bundle is implemented as a tar file compressed in Javascript (JS) by HTML5 and CSS. The bundle
AJAX request,
Zepto. js
JQuery (for example, Internet Explorer only)
It may be configured to use and support protocols and technologies such as CustomJS communication with TTS and configuration plug-ins.
The following HTML5 APIs are supported:
File system
Webstorage
Audio
File Writer

  For further explanation of how the updater-wrapper-bundle mechanism works, reference is now made to FIGS. FIG. 46 illustrates the updater broadcast receiver operation performed after the patient device has completed its bootup sequence. The procedure is simple. The updater is loaded and executed by starting the updater activity start procedure. Then, the updater broadcast receiver procedure ends.

  As shown in FIG. 47, the wrapper is then started. The procedure first sends a crash report (if any), tests that the report transmission was sent successfully, and then deletes the local copy of the crash report. The procedure then tests to see if root access to the underlying hardware is operational. If not, a report to the user is generated and the wrapper activity start sequence is terminated. This report to the user is provided for troubleshooting purposes. The normal user of the patient device is not expected to understand this message.

  If the root access is successfully established, the wrapper activity start procedure invalidates the automatic device update, sets the WiFi reboot schedule, restarts the downloader, and then executes it. The changed setting also initiates a reboot as shown.

  The wrapper downloader activity is illustrated in FIG. If there are no new bundles to be downloaded, the wrapper activity is simply asleep for a predetermined period of time and then tested again to see if there are any new bundles. If a new bundle exists, the bundle is downloaded and the activity downloader checks to see if the file was downloaded properly. This involves checking the MD5 checksum of the file to ensure that the downloaded code has not been altered during the transfer. The code is then installed and the procedure goes back to sleep again for a predetermined period. If either the download procedure or the next test and loading procedure fails, the activity downloader goes to sleep for a different predetermined period (sleep period after failure).

  The crash handling wrapper activity procedure is shown in FIG. When a crash occurs, the wrapper collects relevant exception (error) information, including the time stamp of the crash. This exception information is stored in the local storage, and an alarm schedule is set so that the device is restarted after a predetermined time (for example, 10 seconds). Often, crashes or exceptions can be due to intermittent loss of network connectivity. Therefore, a scheduled reset is programmed into the crash handling procedure to address this issue by rebooting.

  The procedure used by the bundle mechanism is shown in FIG. This procedure is performed by the bundle on the patient device when the settings are to be updated. The procedure retrieves the new settings from the portal computer (server) and ensures that the portal computer has associated the appropriate credentials with the new settings (these new settings are directed to this device). A test is performed (to ensure that), and then a check is made to determine if any of the settings have actually changed. When any setting is actually changed, the setting is updated and the wrapper code is notified. When checking the authentication information, if the patient device has previously been given the required authentication information, the device user does not need to do anything. However, if the authentication information does not exist or if they do not match the designation from the portal computer, the user is prompted to enter the required authentication information on the login screen. When checking the details of the update, if the bundle settings update procedure determines that the settings have not changed, the procedure enters sleep mode for a predetermined refresh rate period.

As a non-limiting example, the following refresh rate may be established. All of these may be changed programmatically by interaction through the portal.

Settings Management In the above table, the following settings may be changed or set using a portal computer.
-Refresh rate: Calendar event refresh from portal computer (server)-Preferences refresh rate: Setting refresh from portal computer-Enabled TTS: Enable / disable application (eg calendar) speech synthesis-Software update Rate: Bundle update check rate-Software update failure rate: Bundle update check rate in case of failure and no calendar-Reboot rate: Periodic reboot rate-WiFi restart rate: Periodic WiFi restart

Portal Computer A portal computer is a networked computer or a collection of integrated computers that communicate with the patient device 50 through an appropriate network connection. For example, the portal computer and patient device may be programmed to communicate over a secure channel over the Internet, such as using a virtual private network (VPN) connection. Associated with the portal computer is a data storage system 56 that is programmed to function as a database in which data used to implement the integration of patient equipment and portals is stored. If desired, the portal computer can be implemented using the computer system 12 (or server 150 and database 160) described above. Alternatively, the portal computer 55 may be implemented using a separate computer.

  The family (or care facility staff) interacts with the portal computer to customize the user interface of the patient device 50, select specific content or specific applications, and directly control the patient device by remote control. Information entered by the portal computer is pushed to the patient device 50, and feedback regarding how the device is being used by the patient is sent back to the portal computer.

  In general, portal computer 55 includes within its data storage system 56 a collection or kit of UI elements, as well as application programs that can be pushed to digital content (eg, images, videos, music) and patient equipment 50 (its applications). The program is then executed on the patient device 50). To do this, the portal computer 55 functions to allow the family or care facility staff to customize the way the patient device functions. Such customization may be divided into three categories: UI element customization (schematically indicated by reference numeral 60), content and application customization (schematically indicated by reference numeral 68), remote Control capability (schematically indicated by reference numeral 70).

  Customizations including selected UI elements, applications and content are pushed from the portal computer to the patient device. The portal computer assembles a package that includes a data structure that stores all UI element configuration information and given application selection information. The package may be compressed using any of a variety of different data compression algorithms and then transmitted to the patient device via a secure channel. The patient device restores the package and extracts element setting information and application selection information included therein. These data are then placed in a buffer in the patient device's memory, while the current settings of the device are saved in the patient device's memory for backup. The data placed in the buffer is then swapped with the current element settings and application selection settings, and any running application is instructed to reboot or otherwise reload the new settings.

The data package may also include actual copies of executable applications that run on the patient device, as well as content such as images, music, video and other multimedia content provided by the family. Since the portal computer saves the state of the patient device, the portal computer does not retransmit a copy of the executable application (already present on the patient device according to the saved state information). If a family or other caregiver chooses to hide or remove a particular executable application from the patient device, the application selection settings are stored in a package delivered from the portal computer to the patient device. This application selection setting is implemented by the patient device by not showing the executable application without actually deleting the application from the patient device unless it is deemed necessary to reclaim storage from the patient device. . Thus, if the family or caregiver later decides to re-enable the application, it can simply be switched on via the application selection settings and does not need to be pushed back to the patient device.

Method for Programming a Portal Computer A portal computer may be implemented by a networked computer connected to communicate over the Internet. The portal computer has at least one processor and associated non-transient memory in which program instructions that cause the portal computer to implement the functions described herein are stored. Preferably, the portal computer comprises a display monitor and a suitable input device such as a keyboard, mouse and / or touch screen.

  The portal computer uses the following software architecture. With this software architecture, the processor of the portal computer is programmed to perform the functions described herein. Of course, different architectures may be used.

  The portal communication software architecture is shown in FIG. As shown, a networked computer operating as a portal computer sends and receives messages to and from patient equipment. FIG. 44 shows how the portal computer and patient device APIs are configured to interact via this messaging. A further description of the portal software API will now be described.

  As shown in FIG. 44, the portal computer is configured to include a wrapper update API 219. The wrapper update API 219 communicates with the updater 213 on the patient device and mediates a “check for new version wrapper” message used to indicate when a new wrapper is delivered and installed on the patient device. . The portal computer also includes an event API 221 that mediates events executed on the patient device. The event API responds to messages including "Login", "Get Event List" and "Get Calendar Preferences". The event API also handles similar messages in other applications pushed to the patient device. The portal computer further includes a bundle update API 223 that mediates bundle updates in the patient device via a “check for new version bundle” message. Finally, the portal computer includes additional portal software API 225. With these additional portal software APIs 225, the portal computer can be connected to other systems such as cloud services available on the Internet.

Referring to FIG. 45, the portal computer also includes a sentry crash reporting module 227 that logs crash reports sent from patient devices. Such a crash report may be generated when the updater-wrapper-bundle mechanism described above does not operate properly, for example, due to intermittent loss of network connectivity. The crash report may also be sent from an application running on the patient device, such as, for example, a calendar application.

Description of portal software API
Login-Provides a way to authenticate clients on the server. Once authenticated, APIKEY is returned. This key is used as an authentication for almost all other methods.
Params:
username: Resident Display Login, obtained on the resident's display tablet;
token: Resident Display Password, obtained on the resident's display tablet;
uhid: UNIQUE hardware identifier. The system only accepts one device per user. The reverse is also true.

Log out-This invalidates a previously acquired APIKEY.
Params:
apikey: APIKEY obtained by / login method.

get_update_version-returns the latest available version bundle for this client.
Params:
apikey: APIKEY obtained by / login method.

get_wrapper_update-Return and download the latest version of the wrapper. In this case, the client UHID is used instead of APIKEY as identification. At this point, the client has not yet logged in.
Params:
uhid: UNIQUE hardware identifier. The system only accepts one device per user. The reverse is also true.

  Sync_Api-provides all the data needed by the client. This includes settings, events, messages and other media resources (photos, music and videos). Each request to this method returns a SHA1 hash in a header field named E-Tag.

  This hash is used for the next request. The server uses this information to determine whether there has been a change in previously provided content. If there is a change, new data is returned with a 200 status code. If there is no change, 304 is returned. This not only reduces server CPU usage, but also saves bandwidth.

Media resources (images, music and video) are to be stored locally. Each media has a unique name throughout the system.
Accept: can be used to indicate the desired response format.
On4-Token: APIKEY obtained by / login method.
On4-Version: Application version.
On4-TzOffset: Time difference (in seconds) between UTC time and local time. If the local time is later than the UTC time, a minus sign must be used.
On4-Position: The latitude and longitude of the device. This information is used to provide accurate weather information to the equipment.
On4-Time: Current UTC timestamp. For NOW, the value 0 (zero) is accepted. The server returns events from this time up to 3 days ahead.
If-None-Match: E-Tag value from the last request, or empty if the first call is provided or nothing is provided.

UI Element Customization and Capability Points The user interface with which the patient interacts when using the patient device 50 consists of a kit of UI elements 62 that can be pushed to the patient device after being selected by a family (or staff member). UI elements are stored in the data storage system 56 and constitute a specially selected set of user interface elements. Each of these user interface elements matches a given level of cognitive and / or motor skills, provides the patient with the information the patient needs every day, and allows the family and patient to communicate As is graded.

  In order to facilitate the selection of appropriate UI elements for the needs of a given patient by a family or nursing staff, each individual UI element 64 in kit 62 includes cognitive and / or motor skills. At least one score 66 is associated. The ability score assigns a numerical score to each UI element based on the degree of difficulty that the user has when using the UI element. Kit 62 includes a variety of redundant and functionally overlapping UI elements, ranging from automatic UI elements to UI elements that are very easy to use and those that are sophisticated to use. One that is appropriate for the patient's capabilities can be selected.

  Multiple points may be associated with each UI element as needed. Therefore, in addition to cognitive ability scores, motor ability scores (sight, hearing, hand dexterity) may be associated. Cognitive abilities and other ability scores may be applied where applicable to rank different applications or content difficulty levels.

  For example, users with high cognitive skills and high motor skills will have difficulty understanding how to use the drop-down menu UI elements to select which photos they want to see in an image viewing application. It may not be. Less capable people may not be able to navigate the drop-down menu, but how to use the forward and backward buttons to browse photos that were automatically selected for the patient once May be understandable. A less capable person may not be able to operate any UI element. In that case, the application executes an automatic slide show. In an automatic slide show, photos are selected at a predefined rate that is not controlled by the patient.

  This is illustrated in FIGS. 36a and 36b and includes a basic photo viewing application pushed to the patient device. FIG. 36a shows a display customized for a patient with average vision and moderate cognitive ability. Thus, in FIG. 36a, the date and time are displayed in “normal” size text, and the “next photo”, “previous photo” and “back” buttons 80 are displayed and active. For this patient, the prospect of pushing the button facing left to “return” and pushing the button facing right to “go forward” is not a difficult task.

In contrast, the user interface of FIG. 36b has been customized with a portal for people with low vision and limited cognitive ability (or perhaps limited hand dexterity). Thus, the date and time are displayed in “large” size, and the “next photo”, “previous photo” and “back” buttons are hidden. In order to explain the removal of the button 80, the slide show function 82 of the photo browsing application has been switched to “automatic”, so that the images from Automatically circulated. In this case, the patient only has to look at the screen. There is no button to press.

Using cognitive and other ability scores By providing a full kit of UI elements, the portal computer 55 can selectively assemble a highly sophisticated and highly customized user experience for each patient. it can. However, having such many different UI elements at different capability score levels is not without difficulty. A typical family member or nursing staff may have little or no experience in user interface design.

  Therefore, to simplify the process of customizing the UI, the portal computer will sort and rank UI elements according to cognitive ability scores (and other scores), thereby customizing the user interface of the patient device. Appropriate ones will be provided first to those who are trying. Figures 42a and 42b show two alternative embodiments of how this is achieved. In FIG. 42a, the display 57 presents an image of what is displayed on the patient equipment display (in this case, the calendar application (reference number 81)), along with a clock-date-weather header (reference number 83). An application selector 85 is provided along the right side of the display of the patient device. In application selector 85, an icon corresponding to the application is available to be pushed to the patient device. The display 57 also includes a ranking window 87 in which the ability score of each application is displayed. Note that the clock-date-weather header has a score of “2” indicating that this interface element is relatively easy to use compared to the image application (having a score of 4, ie, the difficulty level exceeds the average). Note that It will be appreciated that the ranking window indicates the number of points assigned by the system that may be based on either the actual usage statistics for a particular patient or the total statistics generated by the aggregation server computer 74 (FIG. 34). If multiple different scores are implemented (eg, cognitive and athletic scores), the portal computer user can score by the family or nursing staff selecting the appropriate radio button (reference number 89). Allow switching. By selecting one of these buttons, the ranking of the displayed application is switched to reflect the selected option.

  FIG. 42b shows a different way of presenting an application to select to push against the patient device. In this embodiment, the application selector window 91 lists all available applications. The listed applications are sorted in order of ability score based on the selection of a desired number of radio buttons (reference number 89). The family or nursing staff can then drag the selected application to the push window 93. By doing this, the application in the push window is marked as being pushed to the patient device. In this case, three applications are selected. The portal computer displays the average difficulty level and the maximum difficulty level based on the application dragged to the push window. This allows the family or nursing staff to easily understand the level of difficulty that the selected application palette imposes on the patient.

  Alternatively, the portal computer 55 is programmed to automatically select the appropriate configuration based on previously acquired knowledge of the patient's capabilities. This a priori knowledge is obtained from feedback of usage statistics from patient equipment, as described below.

Once basic selections are made regarding application selection, the portal also provides a more detailed screen. This screen allows the family or nursing staff to set further details specific to a particular application. In this regard, FIGS. 38 and 39 show exemplary screens for setting up basic display and user interaction features. FIG. 39 includes application selection structures 91 and 93 similar to those of FIG. FIG. 40 shows another exemplary screen used to add music and video content to the list of content pushed to the patient device. FIG. 41 shows an exemplary screen used by a family or nursing staff to preset a messaging application such as Skype with a person that a patient wants to communicate with periodically.

Remote control In some cases, a family may wish to share an application or content with a patient, but the user interface for that application or content may exceed the patient's cognitive capabilities. In such cases, the portal provides remote control capabilities so that the family can operate through the portal and directly control what is presented to the patient device.

For example, during a call or video chat through a patient device, the family may want to show the patient several images that were previously pushed as content to the patient device. Using the remote control capability, the family can, for example, launch a slide presentation application directly to show the patient an image.

Usage data collection, feedback and summing The concept of usage data collection and feedback is important for patient device-portal computer integration. When a patient uses a patient device, data is collected in real time that indicates which applications and content has been viewed and when, and which UI elements have been used and when. These collected data are then communicated to the portal computer as feedback.

  Feedback is important in several ways. First, the feedback alerts the family if a particular user interface element, content or application is not being used, or if it is used incorrectly by the patient. This allows the family to customize the user interface, content and applications to better match the patient's cognitive abilities. As shown in FIGS. 37a and 37b, feedback provided from patient equipment can be presented to the family in various ways. Two methods have been described.

  In FIG. 37a, a display 57 attached to the portal computer provides an image of the screen displayed on the patient device. This display is updated in real time to show what the patient is currently doing. In this case, the patient finally browses the calendar application by pressing the “Calendar” button. FIG. 37b shows an alternative “heat map” diagram. Here, each UI element that can be actuated by the user of the patient device is shown with an overlay and a heat map or graphical comparison of how often each UI element was used during the last measurement period. Show. In FIG. 37b, graphical representations of different sized circle shapes are used to show relative usage statistics. The larger the circle is, the more frequently the UI element is used. If desired, numerical statistics may be displayed with or instead of the graphic display. Of course, other methods for representing the heat map graphically may be used.

  Second, the feedback may also be sent to an internet-based aggregation server computer 74 (FIG. 34) in the cloud 72 where the usage statistics are summed across multiple users. Thereafter, the cognitive ability score is updated for each UI element using the summed data. In this way, the difficulty level of each UI element is finely adjusted based on usage statistics collected over time. If desired, the combined data can be weighted and combined with usage data from a particular patient to provide a mixed cognitive performance score.

  These combined data are also used by the portal computer 55 when automatically configuring a “recommended” user interface configuration for a particular patient. To accomplish this, the portal computer receives from the aggregation server computer a set of UI element templates that indicate recommended customization configurations for each of the applications stored on the portal computer that can be pushed to the patient device. These templates are created with total usage statistics so that UI elements are selected to match each of a plurality of different capability levels.

In order to compile the aggregated statistical data, personal information about the patient other than the patient's current cognitive ability score is not transferred to the cloud. Therefore, it is inevitably discovered that UI element A was not used by persons with cognitive ability less than level 3, UI element B was not used by persons with cognitive ability less than level 5, etc. Is done.

Method of Programming an Aggregation Server Computer An aggregation server computer is a networked computer or a collection of integrated computers that communicate with a portal computer or directly with a patient device through an appropriate network connection. The aggregation server computer has at least one processor and associated non-transient memory in which program instructions that cause the aggregation server computer to implement the functions described herein are stored.

The exemplary aggregation computer uses the following software architecture for the portal computer: With this software architecture, the processor of the portal computer is programmed to perform the functions described herein. Of course, different architectures may be used.

Data Collection Software Agent The “Data Collection” software agent is maintained to always run inside the patient device. The data collection software logs all system announcements such as messages, images, videos, activities, etc., and all user interactions such as touch, application selection and scrolling to a local buffer. All log data points are time stamped to reflect the exact time at which it was done. Depending on the customizable frequency rate, new log data points are uploaded to the portal using Log_API.

患者機器上で実行されているデータ収集のためのソースコードの例

Log_API-This method allows the client to send an information log to the server. This can be used to transmit any important / relevant information.
Params:
apikey: APIKEY obtained by / login method.
data JSON: An array containing one or more objects in the following format:

["Message": "Sample message", // Message to send "Event time": 1380566627, // Unix timestamp indicating message creation time "Level": "Information" // Log level. Valid options are : Info, debug, warnings and errors]

Log example:

Example source code for data collection running on patient equipment

  The aggregation server computer 74 is programmed to calculate a total ability score across all applications and UI elements for each different ability score used by the system (eg, cognitive ability score, motor ability score, etc.). To do this, the aggregation server is programmed to receive usage data packets sent from each portal computer participating in the aggregation service. Each of the portal computers is programmed to periodically transmit usage data packets that include information extracted from feedback received from patient devices in communication with the portal computer.

The usage data packet consists of one or more records. Each of these records contains the following information:
・ Portal ID (specific indicator of the sending portal computer)
・ Ability point indication (whether the record represents cognitive ability, motor ability or some other dimension of ability)
• Patient's current capabilities (capability level for the patient for whom data was collected)
UI element ID (specific indicator for the specific user interface element or application reported in this record)
・ Reporting period (period when portal computer collects data)
Usage count (number of times a UI element or application was used by the patient during the reporting period)

  Note that the data packet does not contain any information that discloses the patient's identity. All that is known about the patient is the patient's currently assigned capability level.

  The aggregation server computer extracts data from the received data packet and stores it. Data is thus accumulated for a plurality of portal computers, each potentially serving a plurality of different patients. Thus, the data collected by the aggregation server represents a population of patients and indicates the difficulty experienced by that population when using each of the different UI elements and applications. From the summed data, the aggregation server calculates usage statistics, which are then sent back to the respective portal computer.

  Usage statistics include, for example, a total (over the entire population) score for each UI element and application. For example, these total scores are, for example, 20% of the population of patients who reported competence level 2 while the UI element with ID number 512 was used by 90% of the population of patients who reported competence level 3. % May have been used only. The statistics generated in this way can be sent back to participating portal computers, where the UI elements and applications are ranked and family members are designed in designing the UI configuration for a particular patient. Or used to present to nursing staff.

  The usage statistics generated by the aggregation server computer represent how each of the UI elements is received and understood by a population that may have varying levels of capability and be extensive. Individual patients may have specific ability levels that differ from the population. For example, a particular patient may be fairly consistent with a population of capability level 3 fairly consistently, but have a level 4 capability specifically when using certain UI elements or applications. It may be. The portal computer may account for this fact by using usage data collected from the patient and ignoring specific UI element rankings from the population and adapting to specific patient capabilities. This may be accomplished by storing a UI element ranking table for each patient in the portal computer data storage system 56. Initially, all UI element rankings are assigned values received from the total population. However, if you ignore a particular UI element ranking and fit a particular patient's capabilities, these values will be overwritten and replace the total data stored on the tablet for that patient. In this way, the portal computer can display UI element rankings for each of the UI elements and applications, even if they have not been previously used by the patient. Thereafter, as experience is gained about how the patient reacts to a particular UI element, the table is eventually updated to reflect the patient's unique characteristics.

The UI element ranking generated by the total population data may also be used by the portal computer in determining a particular patient's ability level. This systematically pushes UI elements and applications to the patient equipment with increasing difficulty over time (based on the total ranking score) and which UI elements or applications are not fully available to the patient This is done until it is determined (based on feedback provided by the patient device). Thus, the patient's ability level may be determined to be equal to the maximum ability level that the patient has been able to consistently perform. The goal here is not to challenge the patient, but rather to establish a baseline on which the patient can comfortably use the selected UI elements and applications.

UI Element Kit The UI element kit 62 stored in the data storage system 56 of the portal computer 55 provides redundant and somewhat redundant functionality. For example, the simplest gesture command made by a patient can be “touch anywhere on the screen”. A more advanced way to do the same thing can be to press one button labeled with the function that the button performs. A more advanced menu bar is a menu bar having tab-like options and no lower level (in short, a row of buttons with labels). More advanced is a menu that provides submenu options one level below.

  Choosing from this kit of UI elements, Figures 36a and 36b show how the portal can be used to customize the user experience on patient equipment. As already described, FIGS. 36a and 36b represent exemplary photo browsing applications configured for different levels of capabilities.

  To implement this user interface customization function, each application (in this case, a photo viewing application) includes a data structure defined in the local memory of the patient device that stores the state of all configurable user interface elements. Examples of these elements are shown in FIGS. 36a and 36b and include “Previous Photo”, “Next Photo”, “Back”, “Slideshow Status”, “Header Bar Date-Time Presentation Size”, “Reminder”, etc. . It is understood that these are just a few examples of user interface elements that can be set.

  The state of these user interface setting variables is changed by interaction with the application via the portal. Using the portal, the state of these variables may be changed, and when they are changed, they affect how the application behaves when executed on the patient device.

  In addition to saving user interface state variables, the data structure associated with each application also stores data captured in real time and activates patient equipment to use specific user interface functions by the patient. Record each instance of when. In one embodiment shown in FIGS. 35a and 35b, the counter is incremented and stored in the usage history portion of the data structure. Date and timestamp information may also be stored in the data structure, recording exactly when the user interface function was used on the patient. In the illustrated embodiment, the incremented number represents the number of times a particular function has been accessed within a predefined (customizable) period. As shown, a 30 day period is selected. For example, “231.30” indicates that the “Previous Photo” button has been activated 231 times in the past 30 days. Data captured in the usage history is sent as feedback information to the portal and also sent to a cloud server that collects total statistics (described in more detail later). By collecting usage history information, the portal can display to the family or caregiver which functions of each application can be used by the patient. As shown in FIG. 36b, a function that has never been used can be switched off.

The description of the above embodiments has been provided for the purposes of illustration and description. It is not intended to be exhaustive or to limit the disclosure. Individual elements or features of a particular embodiment are generally not limited to that particular embodiment, but are interchangeable where applicable and selected without being specifically shown or described Can be used in embodiments. The same may be modified in various ways. Such modifications are not considered a departure from the disclosure and all such modifications are intended to be included within the scope of the present disclosure.

APPENDIX Exemplary computer program code is executed on the portal computer to retrieve settings from the portal computer database and send the settings to the patient device to configure the display of the patient device.

Claims (21)

A computer-implemented system for assisting people with reduced cognitive or motor skills,
A patient device having a display, a processor connected to the display, and a communication port;
A system comprising a portal computer for communicating with the patient device via the communication port,
The portal computer has a processor and associated memory, the associated memory storing a plurality of user interface elements according to a predefined data structure that associates a capability score with each user interface element;
The portal computer is programmed to present the plurality of user interface elements to a user of the portal computer in a presentation arrangement based on the capability score;
The portal computer allows a first user to select at least one interface element from the user interface elements presented in the arrangement and then push the selected user interface element to the patient device. A programmed, computer-implemented system.   The processor of the patient device is programmed to provide feedback data to the portal computer regarding use by the patient of the selected user interface element pushed to the patient device. A computer-implemented system as described in 1.   The computer-implemented computer of claim 1, wherein the portal computer is further programmed to report feedback data to an aggregator computer that compiles statistics regarding user interface suitability to different capability levels. system.   The computer of claim 1, wherein the portal computer is programmed to update the capability score associated with a user interface element based on user interface usage data collected and aggregated from a plurality of patients. Implemented system. The portal computer is programmed to present a plurality of applications to a first user of the portal computer;
The portal computer can configure and push the application to the patient device so that the first user selects at least one application and then the application is executed by the processor of the patient device. As programmed,
The computer-implemented system of claim 1. The portal computer is programmed to present a plurality of applications to a first user of the portal computer in a presentation arrangement based on the capability score;
The portal computer selects the application so that the first user selects at least one application from the applications presented in the arrangement, and then the application is executed by the processor of the patient device. Programmed to be pushed to the device,
The computer-implemented system of claim 1.   The computer-implemented system of claim 1, further comprising an aggregator computer that is programmed to communicate with the portal computer and to provide the capability score to the portal computer.   An aggregator programmed to communicate with the portal computer, receive usage data from the portal computer, calculate a capability score based on the usage data, and provide the calculated capability score to the portal computer The computer-implemented system of claim 1, further comprising a computer. The portal computer stores an application program adapted to be executed on the patient device;
The portal computer is programmed to push a selection of the stored applications to the patient device;
The computer-implemented system of claim 1.   The computer-implemented system of claim 9, wherein the portal computer associates a capability score with each of the stored application programs.   The portal computer associates capability scores with each of the stored application programs, and presents the applications to the user of the portal computer in a presentation arrangement based on the capability scores. Computer-implemented system.   The computer-implemented system of claim 1, wherein the ability score is a cognitive ability score.   The computer-implemented system of claim 1, wherein the ability score is a motor ability score.   The computer of claim 1, wherein the portal computer automatically proposes user interface elements to push to the patient device, and uses the knowledge about the patient's capabilities and the capability score in creating the proposal. Implemented system.   The computer-implemented method of claim 1, wherein the portal computer automatically selects and pushes user interface elements to the patient device and uses knowledge about the patient's capabilities and the capability score for the selection. System.   The computer-implemented system of claim 1, wherein the portal computer communicates via the communication port of the patient device to remotely operate the patient device.   The portal computer stores an application program adapted to be executed on the patient device selected from the group consisting of calendar, message, image browsing, video browsing, music, weather, voice chat and voice video chat. The computer-implemented system of claim 1. A computer-implemented system for assisting people with reduced cognitive or motor skills,
A patient device having a display, a processor connected to the display, and a communication port;
A system comprising a portal computer for communicating with the patient device via the communication port,
The computer has a processor and associated memory for storing information regarding a plurality of different user interface elements;
The portal computer defines a data structure in the associated memory for storing information identifying a subset of the plurality of different user interface elements in association with information identifying the patient device;
The portal computer includes information identifying a subset selected from the plurality of different user interface elements that the user of the portal computer determines that the user of the portal computer is suitable for the user of the patient device. Programmed to be embedded in the data structure;
The portal computer communicates via the communication port based on a data structure in which the information is embedded, and causes the patient device to set the display of the patient device using a subset of the user interface elements. , A computer-implemented system.   The computer-implemented system of claim 18, wherein information based on a data structure in which the information is embedded is pushed to the patient device.   The computer-implemented system of claim 18, wherein information based on a data structure in which the information is embedded is pulled from the portal computer by the patient device.   The patient device executes an updater function that periodically communicates with the portal computer via the communication port, and information in the data structure in which the information is embedded is used for setting the display of the patient device. The computer-implemented system of claim 18, wherein the computer-implemented system determines whether the information differs from the information being stored.

Images