AU2021106898A4 - Network-based smart alert system for hospitals and aged care facilities - Google Patents

Abstract

AIBUILD has developed an innovative real-time monitoring and smart alert system (CaptureLiveA), which can prevent accidents and assist healthcare professionals in responding better and quicker towards the elderly and those who have disabilities and dementia, especially with high fall risk and low mobility. It helps ease the pressure on healthcare professionals and vulnerable residents since it empowers healthcare staff to reduce unattended falls and incidents through real-time monitoring in healthcare and associated industries. The invented easy-to-deploy and sophisticated smart alert system eliminates false alarms and reduces nurse fatigue. CaptureLiveAl aims to achieve zero falls and many motion predictions and detections, including but not limited to: Bed Exit, Chair Exit, Wheelchair transition, Fall Prediction, Call for help, Mobile Patient, Mask Wearing and so on. More importantly, many institutions are suffering from depletion of nursing and caring staff. Thus, there is an urgent demand for a more advanced smart alert system to predict/detect incidents and deliver better care while relieving the nursing staffs burden. As such, we present our Al-powered smart alert system, which has a high accuracy of motion and incident detection and nurse triage dashboard. 26 Drawings Local Network Report ReceiveVideoData UPS PowerSuppy Handle Events AlIBUILD Dashboard Accesfor Operation Maintenance and Upgrade Receive Notification Devices(PagerSmartphone) Handle Events Cloud Server Remote Server Main Server Backup Server o 1 (sync with main server) Send Alert Push Video Data Nurse Call Capture Live Camera Remote Client SendAskfor Help Events Relocatable Button Report Operation Fig. 1

Description

Drawings

Local Network

Report ReceiveVideoData UPS PowerSuppy Handle Events AlIBUILD Dashboard Accesfor Operation Maintenance and Upgrade

Receive Notification Devices(PagerSmartphone) Handle Events Cloud Server Remote Server Main Server Backup Server o 1 (sync with main server) Send Alert Push Video Data Nurse Call Capture Live Camera

Remote Client

SendAskfor Help Events Relocatable Button

Report Operation

Fig. 1

AUSTRALIA Patents Act 1990

INNOVATION PATENT SPECIFICATION AI-POWERED SMART ALERT SYSTEM FOR HEALTHCARE AND ASSOCIATED INDUSTRIES

The invention is described in the following statement.

NETWORK-BASED SMART ALERT SYSTEM FOR HOSPITALS AND AGED CARE FACILITIES BACKGROUND

[0001] Falls are one of the main contributors to death and healthcare-related costs in older people. Older adults' falls can result in tissue damage, bone fracture and even death. In addition, when patients/residents are left undiscovered on the ground for an extended time following a fall, "long-lie" consequences for the patient are amplified. Fall-related mortality in the elderly is strongly linked to the amount of time before a fall is discovered. Tsai's study found that the early detection of falls reduces the risk of death by 80%. Unfortunately, current methods of fall prevention and detection are grossly inadequate. The unintelligent falls systems lead to high instances of false alarms, nurse burnout, fatigue, and desensitisation to safety alarms leading to a delayed response.

[0002] During the COVID-19 pandemic or the outbreak of any infectious disease, staffing levels are negatively impacted. Reduced staffing levels lead to less monitoring time for residents/patients and an increased workload on carers/nurses. The ongoing impact of these staff shortages leads to increased falls and even more pressure across the entire health care system.

[0003] For patient well being, the recent Royal Commission into Aged Care recommended an increase in patient contact time. A smart system that reduces the work burden of a fall and triages nurses to attend to a patient as needed will reduce falls, improve the quality of a patient's life, and increase the amount of time carers have available to spend with their patients. Thus, there is an urgent demand for a more advanced smart alert system to predict/detect incidents and deliver better care while relieving the nursing staffs burden.

[0004] The patent describes a people management and nurse call system that utilises state-of-the-art machine learning algorithms. The peer-to-peer (P2P) video transmission communication provides a reliable nurse call system that predicts patients' needs. As such, we present our Al-powered smart alert system, which has a high accuracy of motion and incident detection with a nurse triage dashboard.

EXISTING TECHNOLOGY

[0005] At present, a nurse call system installed in a hospital, residential aged care facilities, or similar institutions enables the communication between the patient and the nurse (the person being cared for and the caregiver) mainly includes the following features:

* Call points

[0006] Call points is a catch or string found in hospital and residential aged care facilities, at places where patients are generally powerless, for example, next to their bed and in the washroom. It permits patients in medical services settings to alarm care workers or other medical staff of their requirement for help.

* Voice assistant from call points

[0007] Voice assistants from the call points can help patients, and residents overcome common dangers. Such as falling, forgetting or being isolated. Voice assistants in care homes help with tasks like playing music, the weather forecast, reading news, and much more for people, especially for patients and residents with limited mobility. Voice assistants technology that is available is accessible for elderly users. However, WiFi coverage is primarily necessary for the voice assistants to keep functioning.

* Traditional Fall detection

[0008] Traditional fall detection includes wearable sensors, ambient sensors and sensor mats. It is designed to detect and send alerts to the carers when falling occurred.

* Pagers

[0009] Pagers serve a critical function in hospitals and residential aged care facilities. It is a type of wireless telecommunications device that receives and displays alphanumeric and voice messages. There are two models of pagers: one-way pagers and two-way pagers. One-way pager can only receive notifications. As two-way pagers, they can acknowledge, reply and initiate messages through an internal transmitter.

* DECT phone integration

[0010] Digital Enhanced Cordless Telecommunications (DECT) is an acronym for Digital Enhanced Cordless Telecommunications. It is a wireless standard that's commonly used with landlines. The adoption of the wireless standard has provided a significant boost to wireless communication.

. Staff duress cards

[0011] Staff duress cards are generally assigned to both aged care/medical personnel and care workers. This card combines an ID badge for storing the staff information, access control for ensuring security and a duress button for enabling staff to make a call if they need any help.

[0012] The nurse call system is commonly open to integrate and support specific sensors, which can be broadly split into three categories:

. Bed, Chair and floor sensor mats

[0013] Staff can place the sensor mat on the bed, chair and floor to alert personnel when at-risk patients or residents leave a bed, chair or step into some restricted areas without any assistance.

* Infrared Alarms

[0014] Infrared alarms can sense the movements in the room and then automatically trigger the alarm to notify the nurse or care workers to minimise potential risks. It can be quickly activated, configured and operated. The main challenges of using infrared sensors are that the configurations may become complex due to different room settings and their inability to differentiate between animals, humans and objects.

. Pendants

[0015] Pendants work as an SOS alarm, and it is perfect for people who can wear it in their daily life. When the button is pressed, it can immediately notify and call the nursing staff and emergency contact persons.

SUMMARY

[0016] The proposed Al-powered smart alert system, CaptureLiveA, consists of a server, a

web nurse call dashboard, and depth sensors. Each sensor is connected with a

micro-computer with an artificial intelligence algorithm to execute object recognition, human

recognition, motion tracking and risk analysis. When an incident is predicted or detected,

staff are alerted via the dashboard. CaptureLiveAl then sends an alert to the server with a

short recording of the incident. As a result, the burden on healthcare professionals is reduced

by reducing false alarms, freeing up more time and enabling staff to react in time when

needed. There is also a continual learning process enabled by reviewing falls recordings and

planning for improvements.

[0017] The patient has usually exited the chair and is already mobile before the mat alarm

sends an alert. CaptureLiveAl will send an alert when the patient vacates the chair, giving

more lead time to respond. A 10-second headway will allow an in-time response to lessen the

severity of any fall-related harm. This headway time will increase as CaptureLiveAl learns.

[0018] To mitigate false alarms, CaptureLiveAl enables the healthcare professional to report the false alarm with details and a short comment with time and location recorded - data to be collected fortnightly from the health care facility and analysed for accuracy to allow continual machine learning.

BRIEF DESCRIPTION OF THE DRAWINGS

Fig. 1 shows the overall architecture of the smart alert system. Fig. 2 shows the flow chart of the process of the alert system when a fall is detected. Fig. 3 shows the sequence diagram of the process when a fall is detected. Fig. 4 shows the sequence diagram of establishing the peer connection between the smart camera and a viewing device.

DESCRIPTION OF EMBODIMENTS

Falls Prediction - monitor patients' stability and gait

[0019] The patient's body skeleton information can be extracted from a depth camera stream. The data collected allows us to analyse the person's fall risk concerning the gait and dynamic stability of patients' movements. The Falls Prediction element of this project will roll over the entire trial period and will continuously learn and improve with progressive abilities and higher accuracy levels.

Falls Detection - instantly detect patient fall.

[0020] Through our on-chip system, patients' skeleton data can be processed in real-time. For example, when a person is falling, the machine-learning-based fall detection method will instantly detect the falling behaviour by analysing the movement recorded. When a fall is detected, a notification will immediately be sent to the server and trigger an alarm.

Falls Detection - review, analyse and adjust procedures.

[0021] When a fall is detected, a video recording of the immediate pre and post period will be recorded and stored in the Remote AWS Cloud or a specified local server. This video can be reviewed to determine how the fall occurred and the possible injury sites determining the need for further investigations. Access to Falls data and video review will be determined with permissions.

Bed Exit - instantly predict and detect bed exit.

[0022] By applying computer vision methods, our system can automatically detect different pieces of furniture in the room. For example, when the patient is trying to get up from the bed, our system will detect the bed exit behaviour by analysing the patient's movement. A notification will also be sent to nurses to attend to the patient in time and provide help.

Chair Exit - instantly predict and detect chair exit.

[0023] Our system can distinguish whether the patient is lying down on a bed or sitting on a chair. Then, when the patient is trying to exit from the chair, the system will detect this behaviour and send a notification like the bed exit case.

Patient on the move - detect and track patients moving around the room.

[0024] Once the person's foot hits the floor and becomes mobile, our system can identify when the patient is on the move.

Mobile and relocatable virtual nurse call button

[0025] By using a depth camera, our system can implement a virtual nurse call button. If patients put their hands on the virtual button, our system will send an alarm to nurses. In addition, our system can automatically detect the virtual button's position based on computer vision methods by recognising the button sticker. It means that we can put the button sticker anywhere within sight of the camera.

Identify if the person is wearing a mask.

[0026] By integrating the computer vision method into the system, we can now identify whether the person in the sight of the camera is wearing a mask or not. If someone is not wearing a mask, the system could alert or log the incident to help with epidemic prevention.

Reassign alert to another staff member

[0027] When an alert is triggered, it will be automatically assigned to the nurse in charge of that patient. However, if the nurse is busy, it is easy to reassign this task to another staff member by clicking the "hand over" button.

Room Occupancy

[0028] By fusion skeleton tracking methods and computer vision methods, we can now detect the number of people in the room. When there are multiple people in the room, we can set the detection service to silent. The status of the room will be shown in the CCTV stream. When there is no one in the room, a special icon will inform the staff that the room is empty.

Bathroom Occupancy

[0029] We can determine when a patient enters the bathroom and how long they have been in there, raising the alarm after a set period if the patient has not emerged-avoiding'long-lie' condition.

Room Exit - raise a call when a window or door is opened.

[0030] The system can identify when a patient moves towards a specified window or door and raise the alarm to the caregivers.

Occupational Health and Safety

[0031] We can monitor for risks continuously in line with the Occupational Health and Safety legislative requirements for the workplace, including slips, trips and falls. We can also monitor the posture of Nursing/Carers for ergonomic hazards from lifting, supporting and moving people and repetitive tasks, saving millions in Workcover claims. Concerning the patients, we will determine if assistance aids such as walkers and wheelchairs are within reaching distance of a patient, whether the patient has entered a restricted area, or whether there are any trip hazards on the floor.

Nurse Burn Out

[0032] The system can reduce stressful and dangerous work environments by identifying aggressive patients or visitors and streamlining workflow. We can manage and harmonise shifts to avoid drastic changes to start and finish times and work hours. CaptureLiveAl can ease the responsibility of providing high levels of care over long periods by ensuring regular shifts, breaks and staff rotations.

Nurse Call System - Incident reporting alert and summary analytics

[0033] The system can generate incident reports and summary analytics from the incident data, which institutes can use for evaluation.

Set individual patient alert sensitivity and personal needs and set reminders for staff intervention

[0034] Based on the patients' independent risk analysis, alert sensitivity configurations are set explicitly for each patient. Moreover, the system can schedule periodic preventive reminders daily to initiate staff to check on certain high-risk patients more frequently. For instance, the system can set up reminders on a high-risk patient in a 30-minute interval and set a reminder every 3 to 4 hours for lower-risk patients.

[0035] Proper staffing levels and workforce distribution can be scheduled based on the needs analysis of patients on each ward.

Track the number of staff contact hours for each resident in line with the Royal

Commission recommendation of 2 hours

[0036] The CaptureLiveAl can detect people and distinguish between staff and patients. The delineation allows the system to record the total contact hours between the nurse and patient.

Contact Tracing of Infectious Disease

[0037] This has potential usages in terms of contact tracing.

Monitor nurse response times

[0038] Track the increased efficiency of nursing/carer response time to alerts or requests for assistance

Triage attendance based upon alert

[0039] When an incident occurs, it will be classified for urgency and directed to the nominated staff member. In addition, alerts can be redirected to other staff members as required ensuring quicker response times.

Communicate directly with resident/patient via the microphone add Bluetooth speaker

[0040] The preferred embodiment of the CaptureLiveAl would contain a microphone and a speaker built in the device, which allows the nurses or caregivers to communicate with the patient remotely.

Ask-for-help button

[0041] The ask-for-help button can be based anywhere in the room as long as it is within the camera's field of view. There can be several ask-for-help buttons placed around the room within easy reach of the patient. In addition, there will be a secondary button with no wire for any location that is not visible by the camera, for instance, the bathroom in the patient's room. When the button is pressed, an ask-for-help signal will be sent to the paired CaptureLiveAl device or the nurse's devices to respond to the events promptly.

Aggressive behaviour detection

[0042] The system has the functionality of being able to detect the aggressive behaviour of an individual. In addition, the system can record the behaviour and send it to the caregiver's devices.

DETAILED DESCRIPTION OF THE FUNCTIONALITIES

System architecture

[0043] The system comprises the following parts, including: S- A pager interface to receive the latest record info. S- A personal handset for each nurse to communicate and receive the notification. S- A swipe card system to dismiss the alarm, S- A router/switch for local device communication * - The smart camera device and its machine learning processing unit * - Central server, which can be deployed locally in the institution or deployed in a cloud for upgradability

Falls Prediction - monitor patients' stability and gait

[0044] From the skeleton information of the body, we can know the 3D coordinates of the patient's joints. Take the pelvis joint as the start point, draw a normal vector Vp oint to the

headjoint: V, = Jointhead - Joint

Take N as the normal vector of the ground. Calculate the first Fall Index Ias:

I=1 -V -N 1=1

Calculate the average of feet joints to find out the geometric center of the point of support: Pointcenter = (jointieftfoot + jointrightfoot) Draw a vector V 2 from Pointcenterto Jointpevis

V2= Joint - Point pelvis center

Calculate the second Fall Index I2as:

1 1 -V -N 2= 2

Calculate the total Fall Index I as: = + 12

Falls Detection - instantly detect patient fall

Take the coordinate of the pelvis joint of two adjacent frames, take the time period between these two frames as t. Calculate the speed S of the pelvis joint between:

Falls Detection - recording

[0045] When the system is running, a buffer stores video frames of the last several seconds. These frames will be encoded into a video and uploaded to the specified server by the HTTPS method when a fall is detected. The route of the video on the server will also send with the notification so we can access the related video of the notification.

Bed Exit - instantly predict and detect bed exit

[0046] To distinguish the furniture the person is related to, we need to recognise and locate the person and furniture. A trained neural network is applied to detect those objects and provide the object's spatial information on the 2D image. For each image frame, we can label areas that are related to different objects. Finding the maximum overlap area between the person and furniture lets us know which furniture the person is related to. If the person is related to the bed, we take the altitude of the head joint at lay down position Playas the baseline. Take the altitude of the head joint at sit down position H as the target.

Calculate the stroke between these two positions: S =H sit Play

Take the current head altitude of the person as; we can calculate the get-up index as: I = (H - H )/S

Take the get-up index I, for the current frame and I.for the last frame. If I, and I,_,satisfy:

I, < 0. 5, I. 0. 9 I I-1

Then the system will trigger an alert of bed exit. A notification with a video will be sent to the server.

Chair Exit - instantly predict and detect chair exit

[0047] This function works similarly to the Bed Exit function. The differences are the head position's baseline and target. Take the altitude of the head joint at sit down position as Hi and the altitude of the head joint at standup tand, the stroke can be calculated as:

S=H -H stand it

The get-up index should be calculated as: I = (H - H )/S

Other steps are the same with the Bed Exit function.

Patient on the move - detect and track patient moving around the room

[0048] From the extracted skeleton joints, the position of the foot joint could be found. Take the altitude of the footjoint as H . Take P as the horizontal position of the pelvis joint at

the current frame and P .as the horizontal position of the pelvis joint at the last frame.

Calculate the horizontal speed of the pelvis joint V :

V = (P. - P_ )/(t. - t_ )

If H and V satisfy:

Hf 0.lm,V > 0. 2m/s foot p The person's status can be labelled as on the move. Warning of the patient on the move will be printed on the CCTV.

Mobile and relocatable virtual nurse call button

[0049] When setting up a virtual nurse call button, a sticker represents the button location that can be placed anywhere within the camera's field of vision. A pre-trained neural network is applied to the system to detect the location and related area of the button sticker. A set of depth values that represents the distance between the button to the camera could be found as: Darea =dd 2 ..., d

Calculate the average of the distance: dbutton mean(Darea)

Take distances in the current frame:

DCurArea d 1 d2 , d

) Calculate the average of the distance: d cur= mean(DCurArea)

Get the maximum value of the distance: d max= max(DurArea)

If dbutton' dcur and d satisfy:

d d cur- button o 0.003m, dmax - d bbutton 0.06m

Then the button is triggered. A notification will be sent via HTTPS to the server.

Identify if the person is wearing a mask

[0050] A specialised neural network is trained to detect whether a person is wearing a mask. If a person is wearing a mask, a green sign will be rendered on the screen. If someone is not wearing a mask, a red sign will be rendered on the screen to notify the staff of this issue.

Reassign alert to another staff member

[0051] When an alert is triggered, it will be automatically assigned to the nurse in charge of that patient. However, if the nurse is busy, it is easy to reassign this task to another staff member by clicking the hand over the button and choosing the idle one.

Room Occupancy

[0052] By fusion skeleton tracking methods and computer vision methods, we can now detect the number of people in the room. When there are multiple people in the room, we can set the detection service to silent. The status of the room will be shown in the CCTV stream. When there is no one in the room, an icon will be shown to inform the staff that the room is empty.

Bathroom Occupancy

[0053] We can track a person's trajectory by applying the SORT (simple online real-time tracking) algorithm. When the system loses the patient's tracking near the bathroom door, it will label the patient status as 'in the bathroom'. A timer will start to count the period that the patient stays in the bathroom. If the timer hits the time threshold (assigned by staff), it will notify the staff to check if the patient needs help.

Room Exit - raise a call when a window or door is opened.

[0054] Take the depth image pixels set of the specified area as D: D = [d,, d 2 ,..., d

Convert the depth information into a 3D point cloud: D = [d,, d 2,..., d ->P = {(xl, yl, z),( 2 2 , z2 ),..., (xn' Y, z))

Take the horizontal coordinates of the point cloud: S =(x1 , y), (x2 y2 ),..., (n)}

If the patient's skeleton joints are within S, the alert should be triggered.

Occupational Health and Safety

[0055] We can monitor for risks continuously in line with the Occupational Health and Safety legislative requirements AND specific OH&S risks for each patient. We will be able to determine the following aspects:

Dangerous or restricted area entry

[0056] The nurse can set one or more marks to set specific areas or regions that are potentially dangerous and should be restricted for the patients to enter. The nurse can specify the areas in an illustrative embodiment by drawing a rectangle or a polygon on the video. For example, the nurse can draw a rectangle on the door in the video. The information is sent to the database, which can synchronise with the smart camera's computing device. As a result, the smart camera can raise the alarm when the door is opened and when it finds a patient who has passed through the door indicated.

Are walking aids placed within reach of the patient

[0057] The system can calculate the distance between the patient and the walking aide to ensure they are within reaching distance of the patient. An illustrative embodiment can further extend this functionality by detecting the intention of grabbing a walking aid, telling whether the patient is trying to get up from their seats or beds.

Are there any trip hazards

[0058] By comparing the direction of gravity from the device's Inertial Measurement Unit (IMU), the system can detect the ground surface regardless of the orientation of the devices. The surface detection method is done by calculating the relative distance of depth pixels from the camera along the direction of gravity. Since the ground is usually perpendicular to gravity, there will be a group of points that fall the same distance along the direction of gravity. The result can also be updated by directly specifying the height of the smart camera above the ground. The system also can conduct periodical scanning on the ground surface from the depth images.

[0059] The system can detect hazards that may cause a patient to fall by detecting noises from the ground surface. Furthermore, by finding different groups of horizontal surfaces at varying heights, the system can compute height differences and the risk of tripping. CaptureliveAl can also highlight the risky areas on the floor and provide suggestions to the carers to remedy the risk early.

Nurse Call System - Incident reporting alert and summary analytics

[0060] The system can generate incident reports and summary analytics from the incident data, which institutes can use for evaluation purposes. In addition, the system can schedule periodic tasks to summarise the data and records stored in the local-based or cloud-based server and generate analytics by comparing with previous data or other sources.

Set individual patient alert sensitivity for staff intervention according to individual patient needs.

[0061] The nurse can set configurations specific to every patient in the system, including a risk analysis determined by data gathered about the patient's condition, operation, medication etc.

[0062] The system can thus compute the level of sensitivity of the alert from the settings. And the system can schedule periodic reminders on a daily basis, reminding staff to check on certain high-risk patients more frequently. For instance, the system can set up reminders on a high-risk patient in a 30-minute interval and set a reminder every 3 to 4 hours for lower-risk patients.

Track the number of staff contact hours for each resident

[0063] As CaptureLiveAl can detect people, it can also distinguish between staff and patients. As a result, the system can record and track the total duration of contact hours between the patient and nursing staff. This has the potential usages in terms of contact tracing and workforce planning.

Set standards and track progress

[0064] The institution can set target times to attend to a patient once an event has occurred. It may also set standards for the number of checks per patient and a level of alert to indicate the attendance of additional staff.

Triage attendance based upon alert

[0065] When an incident has occurred, or when a record has been created. The nurses can respond in varying ways within the dashboard or phone application.

[0066] Firstly, the nurse can respond to the event by selecting 'On my way', which indicates that the nurse claims that the event will be taken care of by that nurse.

[0067] Secondly, sometimes the nurse may be unavailable at that moment. For instance, they are taking care of other patients or working on other issues. The nurse can triage the events to another available nurse by pressing a 'Handover' button. After clicking the Handover button, the nurse can choose another nurse by clicking on the other nurse's name. Then the system will notify that second nurse about the events.

[0068] In the event of a false alarm, the nurse can report the incident with details for review and learning. The nurse would disarm the alert by choosing a 'Report' button. The system will dismiss the alarm, and the false positive event will be recorded to improve the future machine learning model.

Communicate directly with resident/patient via the microphone add Bluetooth speaker

[0069] The preferred embodiment of the CaptureLiveAl would contain a microphone and a speaker built in the device. This allows the nurses or caregivers to communicate with the patient remotely. A preferred embodiment will have the ability to translate languages.

Ask for help button

[0070] We will have a secondary button with no wire for any location that is not visible by the camera, for instance, the bathroom in the patient's room. The preferred embodiment of the button can connect with the CaptureLiveAl with Bluetooth Low Energy (BLE) technology or other Internet of Things protocols. When the button is pressed, an ask-for-help signal will be sent to the paired CaptureLiveAl device or send out a broadcast so that other devices can pick up the signal. Once the ask for help signal is picked up, the system will create records and notify the nurses through multiple terminals, including the dashboard, pager or smartphone.

Aggressive behaviour detection

[0071] The system has the functionality of being able to detect the aggressive behaviour of an individual. For example, when an individual is standing within the camera's view, the camera can monitor if the joint movement speed surpasses a certain threshold. The system will record the behaviour and send it to the caregiver's devices.

IMPROVEMENTS

[0071] Compared to current nurse call systems, CaptureLiveAl solves many existing problems and provides a more comprehensive and intelligent solution in the following areas.

[0072] CaptureLiveAl is Australia's first innovative Al all-in-one nurse call system that uses a smart camera sensor to identify and eliminate occupational health and safety risks and prevent falls. CaptureLiveAl provides nursing staff and care workers with real-time monitoring of patients, incident reporting and falls detection.

[0073] The current nurse call systems on the market today often only support monolingual communication for the communication needs between patients or residents and nurses or care workers. As a result, the patient and residents might not be able to describe their needs in the way they are most comfortable with. CaptureLiveAl will provide real-time translation to ensure smooth and better communication.

[0074] Unlike the current nurse call system, most of the systems only offer manually activating alerts. For people who may not be able to do so at the moment (e.g. those who are unconscious or severely disabled), the call buttons are useless. CaptureLiveAl has a fully automated system and is more accurate, with gesture recognition and the ability to identify situations where the patient needs help in various scenarios, including " Bed exit * Chair exit * Unstable gait * Patient Confusion and disorientation At the same time, CaptureLiveAl can identify other aspects that facilitate better care for the patient. For example, one of the unique features of CaptureLiveAl is occupational health and safety. For patients or residents who currently get up in the middle of the night but cannot reach the equipment to help them, it would be a time-consuming and repetitive task for nursing staff to check manually and prone to human error. However, CaptureLiveAl can automatically check in real-time that all walking aids are in the correct place within the patient's or resident's reach. This feature can also be applied to detect other trip hazards.

[0075] Compared to traditional circuit versions of call devices, CaptureLiveAl allows multiple virtual buttons to be set simultaneously. In addition, there is no limit to the number of adjustable virtual buttons, according to the patient's preferences, habits, behaviour patterns and convenience, with no construction required.

[0076] Many health care call systems currently have false alarms, but there is nothing that nurses and aged care professionals can do about them. However, all alarms identified by CaptureLiveAl are used for continuous learning. For this reason, CaptureLiveAl provides staff with a report function to provide feedback allowing the nurse call system to become more accurate without any upgrade or maintenance costs. While ensuring high accuracy, the system also has an intelligent triage system that allows alarms of different severity and risk levels to be categorised and rationalised to improve workflow.

[0077] For both patients and residents, the system also assesses risks based on their conditions. Then, it can arrange and notify the nurses and care workers to check with the right patients or residents at the most suitable time, which can genuinely improve efficiency and risk prevention.

[0078] CaptureLiveAl not only interfaces with existing call systems but can also be used stand-alone as a system that allows for customisation, allowing hospitals and nursing homes to adapt to their circumstances, including the following levels.

• Organisation Centric adaptable to organisational requirements * Location Centric adaptable to size and type of room, office or workspace " Patient-Centric Care using neural network training

[0079] More importantly, all of these features are supported by CaptureLiveAl's own vision devices, eliminating the need to fill rooms with several different devices. The system can be set up and operated with ease, and there is no need for system training as CaptureLiveAl is the more straightforward and comprehensive system.

[0080] Nurse Call system points of difference include are based on a continual learning cycle:

CaptureLiveAl is customisable, continual monitoring, prediction, prevention, notification, rectification and continual learnings:

Customisable at the following levels:

* Organisation Centric adaptable to organisational requirements * Location Centric adaptable to size and type of room, office or workspace " Patient-Centric Care using neural network training * Staff members

Continual spectrum risk monitoring using gesture recognition and prediction

Identifying when a patient requires assistance in the following scenarios:

" Bed exit * Chair exit * Unstable gait " Patient Confusion and disorientation * Fall Prediction * Fall Prevention " Fall Detection

Prevention

Occupational Health and Safety - ability to identify items of risk to patients, visitors and staff.

* Are all walking aides continuously within reach of the patient " Are there ample call points for a patient to ask for help * Are there any risks of trips, slips or falls * Are walkways unobstructed " Is there any peculiar or anti-social behaviour predicted * Has the patient been individually assessed for risk and check-ups scheduled at the appropriate intervals for each patient

* Reduced Communication errors - Real-time foreign language translation for communication between patients and staff " Improved workflow through triage, balanced workload, safe rostering, early detection of individual risk of burnout. • Nighttime mode monitoring and notifications

Notification

* Patients who call for help are triaged according to the urgency * Nursing staff can be notified through multiple devices

Rectification " Incidents are recorded and able to be reviewed to assess, rectify and improve processes. • Continually learning - false alarm.

Other " No construction * Integratable with existing nurse-systems * No need for expensive training

USAGE

[0081] CaptureLiveAl can be applied to many healthcare institutions, including residential aged care facilities, dementia villages, acute hospitals and rehabilitation centres.

[0082] CaptureLiveA is in the process of validating several aspects of the technology before widespread use and distribution, including: efficacy, accuracy, safety and cost-benefit analysis. Trials will also determine the camera positioning in each room and whether the system should be deployed throughout the entire organisation or only in high-risk environments.

[0083] In order to eliminate alarm fatigue and staff burnout, CaptureLiveAl has a video real-time monitoring feature that will allow healthcare professionals at nurse stations to watch patient rooms and receive incident alerts from a central location. This enables the distant assessment of prospective falls and the determination of whether action is required or whether a potential false alarm should be dismissed as soon as possible.

[0084] Many nighttime situations, such as hospital falls in isolated places, may go unreported for long periods. Having the CaptureLiveAl system installed in inpatient wards, especially those who had strokes, may also be beneficial if deployed in remote places, allowing nursing staff to notice and respond to such emergencies more quickly.

[0085] Furthermore, the healthcare industry is significantly under-resourced. The aspects of patient triage, staff contact hours, appropriate rosters, personalised patient risk analysis will reduce stress on health care workers and improve patient safety.

Claims (5)

1. A method of fall prediction algorithm based on unstable gaits and environmental information can analyse the potential risk of a patient/resident and predict whether they are going to have a fall. The fall prediction algorithm comprises of: A method of gait analysis to calculate the risk of falling from skeleton joints information A method of computer vision-based self-adaptive human status monitoring

2. A smart alert nurse call system that can collect data and identify and eliminate occupational health and safety risks and prevent falls with real-time video monitoring, comprising: one or more sensors, including depth sensors and cameras, and computing devices configured to monitor the patient in the ward. The device is configured to: detect bed/chair exit; apply fall prediction method according to claim 1 to prevent falls; detect room occupancy; detect ask for help button mounted on the wall and monitor whether the button has been pressed; trigger a notification when an incident has been detected;

At least one central server can be either cloud-based or run in a local area network to store the data of incidents and the care receivers and hospital staff. This central server is configured to: communicate with the sensors and receiving devices through local area network or WiFi; summarise all fall events sent from the local computers; send an alarm alert to caregivers and a short video recording showing the fall event that has taken place;

a dashboard application for nursing staff, which is configured to: receive and manage the notifications; respond and triage the notifications; a mobile application for nursing staff, which is configured to: receive notifications from the server; allow the user to respond to check the patient; receive regular reminders; one or more pagers or receiving devices to receive notifications from the device;

3. A smart alert nurse call system, according to claim 2 that is configured with a holistic approach to cover the entire health service.

From the patient's perspective, by collecting data related to each patient, the nurse call system, according to claim 2, can determine individual needs and deliver the required level of care to the patient at the right time to prevent falls or injury. From the staff members perspective, by collecting data on any limitations to the individual physical abilities, skills and routines the nurse call system according to claim 2, can safely schedule each staff member with the appropriate shifts and working hours to reduce risk and optimise workflow. From the organisation's perspective, the nurse call system, according to claim 2, can match the correct patient load to each staff member. This will reduce the risk of nurse/carer burnout and ensure the patient has a tailored level of care.

4. The nurse call system, according to claim 2, where the process is further capable of identifying potentially dangerous or aggressive behaviour and creating an alert.

5. A smart alert nurse call system, according to claim 2, can record falls and incidents for review by healthcare professionals to assess possible injury sites for investigation and allow continual learning from a risk perspective based on a series of body joint data so they can improve placement of items in the patient's room or adjust individuals medication as needed