WO2021262096A1

WO2021262096A1 - An intelligent networked system for proactively monitoring and predicting various parameters in a defined environment

Info

Publication number: WO2021262096A1
Application number: PCT/SG2021/050356
Authority: WO
Inventors: Jonathan LAU KIAN ENG
Original assignee: Nervotec Pte Ltd.
Priority date: 2020-06-22
Filing date: 2021-06-21
Publication date: 2021-12-30

Abstract

This invention discloses an intelligently networked system for proactively monitoring various parameters,of one or more stochastic behaviours of users,in a defined environment, said system comprising:a defined environment configured in respect of one or more reader mechanisms, installed at pre-defined locations, in said defined environment, each reader mechanism is configured to read a wearable mechanism(WM) tagged to a user; one or more mobile devices (MD), associated to a corresponding user; monitoring a first stochastic behaviour, of said user, in terms of at least a first risk potential level(RPL1); monitoring a second stochastic behaviour, of said user, in terms of at least an instance of social distance non-compliance(SDNC); a communicably coupled processor(P) being configured with a rule engine(RE) and a notification engine(NE), said processor(P) configured to provide notifications in relation to breach of one or more thresholds, along with severity of breach of said thresholds, along with identity of associated user.

Description

AN INTELLIGENT NETWORKED SYSTEM FOR PROACTIVELY MONITORING AND PREDICTING VARIOUS PARAMETERS IN A DEFINED ENVIRONMENT

FIELD OF THE INVENTION:

This invention relates to the field of automation, networked systems and environments, and wearable devices.

Particularly, this invention relates to an intelligent networked system for proactively monitoring various (physiological and social distancing) parameters in a defined environment.

BACKGROUND OF THE INVENTION:

Coronavirus disease (COVID-19) is an infectious disease caused by a newly discovered coronavirus known as Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). COVID-19 has become a global pandemic with numerous public health, socio-economic, and environmental implications.

The following attributes of the virus in part drove COVID-19 to become a global pandemic today:

Firstly, disease symptoms can range from mild (i.e. mild fever, sore throat, loss of smell and test) to severe, thus making it difficult for an individual to decide his/her risk of having COVID-19.

Secondly, the incubation period of the virus ranges from 2 to 14 days. During this period, an infected person is unaware of his/her illness and may interact with others, therefore transmitting the virus.

Thirdly, the virus has multiple modes of transmissions- airborne, contact and droplet, as well as fomite transmission. This significantly increases COVID-19 risk in indoor environments, particularly crowded and inadequately ventilated or sanitized settings.

In order to mitigate the spread of this virus, it is important to: reduce social contact by maintaining a distance of at least one metre between people, otherwise known as ‘social distancing’. regularly screen people for possible symptoms before entry into or while in an indoor environment.

Such practices (known as COVID-19 safety management measures) have been mandated in various defined environments such as schools, workplaces, factories, industries, public transports, construction sites, and the like. However, challenges have been reported in their implementation:

One-metre distance directives are commonly violated due to disruption of processes typically occurring in these environments. In Singapore, more than 300 companies have been fined for violations of safe distancing measures, with close to 140 employers being ordered to cease operations.

Health monitoring is often inconsistent or accurate. In Singapore, migrant workers are required to measure their heart rate and oxygen saturation with a pulse oximeter and input these measurements into FWMOMCare app (a health tracking app developed by the Ministry of Manpower (MOM)). They are then required to report to temperature screening stations to measure their temperature before reporting to work. MOM noted that this manual and fragmented protocol has resulted in erroneous or falsified inputs and low adherence rates.

There is therefore a need to automate and streamline COVID-19 safety management measures to improve compliance and scale implementation. With the advent of video-based vital signs monitoring and Internet-of-Things, a networked, automated system can now be developed for pro-active monitoring of physiological and social distance parameters, thus simplifying COVID- 19 safety management for businesses and governments.

OBJECTS OF THE INVENTION:

An object of the invention is to provide a networked, proactively-monitoring system and method for a defined environment. Another object of the invention is to provide a system and method targeted towards the use of a networked, proactively-monitoring system (for a defined environment) in ensuring safety protocol compliance with regards to a plurality of parameters.

Another object of the invention is to provide a system and method targeted towards the use of a networked, proactively monitoring system (for a defined environment) in enabling individuals to remedy safety protocol breaches.

SUMMARY OF THE INVENTION:

According to this invention, there is provided an intelligently networked system for proactively monitoring various parameters, of one or more stochastic behaviours of users, in a defined environment, said system comprising: a defined environment configured in respect of one or more reader mechanisms, installed at pre-defined locations, in said defined environment, each reader mechanism being configured to read a wearable mechanism tagged to a user; one or more wearable mechanisms, each wearable mechanism being tagged to a corresponding user in said defined environment; one or more mobile devices, each mobile device being associated to a corresponding user in said defined environment; monitoring a first stochastic behaviour, of said user, using sensed data of said mobile device associated with said user, in said defined environment, in terms of at least a first risk potential level; monitoring a second stochastic behaviour, of said user, using sensed data of said wearable mechanism tagged to said user, in said defined environment, in terms of at least an instance of social distance non-compliance; a communicably coupled processor being configured with a rule engine and a notification engine, said processor configured to receive: o sensed data from each of said wearable mechanisms in association with user identity; o sensed data from each of said mobile devices in association with user identity; o sensed data from each of said reader mechanisms in association with defined environment identity; said rule engine configured to define rules in relation to thresholds, along with rules in relation to severity of breach of said defined thresholds, for one or more parameters associated with said first risk potential level and for one or more parameters associated with said social distance non-compliance; and said notification engine configured to provide notifications in relation to breach of one or more thresholds, along with severity of breach of said thresholds, along with identity of associated user.

In at least an embodiment, said defined environment is configured with reader mechanisms, one or more reader mechanisms being located: at entry point(s) of said defined environment in order to tag, a second node based sensed entry data, in terms of person identifier and time; at exit point(s) of said defined environment in order to tag, a second node based sensed exit data, in terms of person identifier and time; and at pre-defined locations throughout said defined environment in order to tag, a second node based person-movement data, in terms of person identifier and time.

In at least an embodiment, said wearable mechanisms are tags, associated with corresponding unique identifiers, which communicate with said reader mechanisms.

In at least an embodiment, said wearable mechanisms comprise a set of first sensors, the sensors are, contact sensors, pulse oximeter sensors, body temperature sensors, skin temperature sensors, and fitness tracking sensors, with sensing being done at pre-determined discrete time intervals.

In at least an embodiment, said wearable mechanisms are configured to collect distance measurements at pre-defined time frequencies and further configured by said notification engine along with said rule engine to send alerts to a corresponding user wearing said wearable mechanism, which are dependent on the severity of threshold breach defined in said rule engine. In at least an embodiment, said wearable mechanisms are Ultra-Wide Band real time locating systems that comprise a location tag for a person, or location anchors.

In at least an embodiment, said reader mechanisms, that are a combination of Ultra-Wide Band enabled mechanisms and Bluetooth enabled mechanisms, are installed near a ceiling of a defined environment.

In at least an embodiment, said reader mechanisms, that are Bluetooth enabled mechanism, are installed more than 6 metres above ground in a defined environment.

In at least an embodiment, said mobile devices are configured with a set of second sensors, that sense one or more physiological parameters of a user; that is selected from a group of sensors, which include heart rate variability sensors, heart rate sensors, body temperature sensors, and skin temperature sensors.

In at least an embodiment, said rule engine is configured to determine rules, and corresponding notifications, in respect of stochastic behaviour, in terms of temperature, of a user, said rule engine being configured to: determine rules relating to temperature spikes as sensed by said mobile devices and / or said wearable mechanisms, allowable temperature ranges as sensed by said mobile devices and / or said wearable mechanisms, disallowable temperature ranges as sensed by said mobile devices and / or said wearable mechanisms, and instances where said rule engine needs to communicate with said notification engine to raise a user-identified flag of alarm in terms of temperature faults.

In at least an embodiment, said rule engine is configured to determine rules, and corresponding notifications, in respect of stochastic behaviour, in terms of space, of a user, said rule engine being configured to: determine rules relating to distances between persons identified by determining readings provided by said reader mechanisms in association with users’ wearable mechanisms, allowable distances in terms of such readings, and instances where said rule engine needs to communicate with said notification engine to raise a user- identified flag of alarm in terms of social distancing faults; and determine rules relating to defining various zones within said defined environment in concurrence with said reader mechanisms per zone.

In at least an embodiment, said rule engine is configured to determine rules, and corresponding notifications, in respect of stochastic behaviour, in terms of patterns, of a user, said rule engine being configured to: determine rules relating to a user’s frequency of use of sanitization stations paired with said reader mechanisms, that is determined by said user’s wearable mechanism over a period of time, allowable thresholds in terms of such readings, and instances where said rule engine needs to communicate with said notification engine to raise a user-identified flag of alarm in terms of sanitization faults.

In at least an embodiment, said system comprising a grouping module engaged in order to group a set of reader mechanisms per zone and rules being configured with respect to such defined zones by said rule engine.

In at least an embodiment, said wearable mechanisms or said mobile devices are configured with a display mechanism to receive notifications from said processor, in terms of determined faults being temperature faults, social distancing faults, and / or sanitization faults.

In at least an embodiment, said processor is configured to receive sensed data from: at least a first input obtained from said wearable mechanisms, said reader mechanisms, and said mobile devices; and at least a second input, obtained from a dashboard, in relation to thresholds for each parameter of sensed data, in order to determine temperature faults, social distancing faults, and / or sanitization faults.

In at least an embodiment, said “first risk potential level” is determined from the sensed physiological parameters obtained from the wearable mechanisms and / or the mobile devices, said “first risk potential level” being determined according to administrator-defined thresholds for each sensed parameter.

In at least an embodiment, said instance of “social distance non-compliance” is determined from said reader mechanisms in communication with said tagged wearable mechanisms, said instance of “social distance non-compliance” being determined according to administrator-defined thresholds of disallowable distance between any two tagged wearable mechanisms in a same defined environment.

In at least an embodiment, said mobile device comprising:

Video Acquisition modules, associated with respective mobile wrappers, to extract video feed from said mobile devices, said mobile wrappers configured to feed frames into a software development kit (SDK) library responsible for processing; a Session Manager within a SDK library for managing processing sessions and for initiating video feed processing in a Session System; said Session System comprising: o a face detection and tracking module, o am image processing module, o a PPG extraction module, and o a PPG signal processing module; an output module configured to output data from said Session System, said output data comprising heart rate, heart rate variability, oxygen saturation, and respiratory rate; and said Session Manager configured to pass said output data to said mobile wrappers that in turn share said outputs with respective mobile devices.

In at least an embodiment, said system comprising an SDK comprising: a PPG signal extraction and processing component further comprising: o a Session Manager containing a face tracking algorithm, which identifies regions of interests from a user’s face (forehead, left and right cheek) and extracts RGB signals from a video feed of said mobile device’s camera; and o a PPG Calculator configured to receive data from said Session Manager, said PPG Calculator containing:

^■ a signal calculation module, configured to average RGB values across all pixels, to give temporal RGB signals;

^■ a signal interpolation module, configured to receive said RGB values, in order to apply mathematical functions to the RGB signals to combine the 3D RGB signals into ID PPG signals and subsequently transform the PPG signal into the frequency domain; and

^■ a signal pre-filtering module, configured to receive signals from said signal interpolation module, in order to remove any outlier signals;

^■ a PPG signals storage bucket to store filtered signals from said signal pre filtering module; and a physiological parameters calculation component, configured to receive filtered signals from said PPG signals storage bucket, said physiological parameters calculation component comprises a multitude of calculation algorithms for each of the sensed physiological parameters, which include an oxygen saturation calculator, a heart rate calculator, a heart rate variability calculator, and respiratory rate calculator.

BRIEF DESCRIPTION OF THE ACCOMPANYING DRAWINGS:

The invention will now be described in relation to the accompanying drawings, in which:

FIGURE 1 schematically illustrates a defined environment where multiple subjects’ physiological and social distancing parameters are monitored in real time in accordance with an embodiment of the invention;

FIGURE 2 illustrates a schematic block diagram of the system of this invention;

FIGURE 3 illustrates the artificial intelligence used by the processor (P) of the system and method of the invention as shown in FIGURE 2;

FIGURE 4 is a flow diagram in accordance with one embodiment of the invention; FIGURE 5 illustrates a network architecture for the monitoring and analysis of various parameters of a user, in accordance with an embodiment of the invention;

FIGURE 6 illustrates a diagram of the architectural content in a mobile device (MD) application of the system, according to one embodiment of this invention; and

FIGURE 7 illustrates a schematic block diagram of the remote photoplethysmography SDK of the system and method of this invention.

DETAILED DESCRIPTION OF THE ACCOMPANYING DRAWINGS:

According to this invention, there is provided an intelligent networked system for proactively monitoring various (physiological parameters and social distancing parameters) in a defined environment.

FIGURE 1 schematically illustrates a defined environment where multiple subjects’ physiological and social distancing parameters are monitored in real time, in accordance with an embodiment of the invention.

Figure 1 illustrates a network implementation of an architecture including a system and method for monitoring physiological parameters and predicting a risk potential to processes, in accordance with an embodiment of this invention.

Figure 1 illustrates a system with one or more mobile devices (MD), wearable mechanisms (WM), and reader mechanisms (RM); all being communicably coupled to a processor (P), over a network, comprising at least a rule engine (RE).

FIGURE 2 illustrates a schematic block diagram of the system of this invention.

In at least an embodiment, there is provided a plurality of first nodes (Nl), with each first node being formed by wearable mechanisms (WM) per person within the defined environment. These wearable mechanisms (WM) can be, but are not limited to, tags, associated with corresponding unique identifiers, which communicate with reader mechanisms (RM). In some embodiments, these tags are associated with an Ultra-Wide Band (UWB) real time locating system, or reader mechanisms (RM). The tags may be worn by persons, within the defined environment, in the form of cards, wrist bracelets, tokens, or the like.

In at least an embodiment, the wearable mechanisms (WM) may comprise a set of first sensors, the sensors being contact sensors, pulse oximeter sensors, body temperature sensors, skin temperature sensors, fitness tracking sensors, and the like sensors. Sensing is done at discrete time intervals. In a preferred embodiment, these may be at once-a-minute frequency of reading.

In at least an embodiment, the wearable mechanisms (WM) are capable of functioning without the reader mechanisms (RM) by communicating with other wearable mechanisms (WM). In other words, distance between wearable mechanisms (WM) is continuously monitored by a processor (P) to determine if protocols relating to minimum set distance are adhered to by persons in a defined environment. Additionally, contact tracing information can be stored in wearable mechanisms (WM) until the wearable mechanisms (WM) synchronises with a reader mechanism (RM).

For each person, data of each wearable mechanism (WM) is combined to form the first node sensed data (Nl). This data is transmitted to a communicably coupled processor (P) with a rule engine (RE) and an associated notification engine (NE).

In at least an embodiment, there is provided a plurality of second nodes (N2), with each second node being formed by reader mechanisms (RM) per defined environment.

Reader mechanisms (RM), which are both Ultra-Wide Band (UWB) enabled and Bluetooth (BLE) enabled anchors, are, typically, cabled and installed near a ceiling of a defined environment. Typically, Ultra-Wide Band (UWB) enabled mechanisms are used in relatively open areas and Bluetooth (BLE) enabled mechanisms are used in relatively enclosed areas where line of sight is difficult to achieve. Typically, these reader mechanism (RM) can be connected either through BLE, Ethernet, Wi-Fi, Wi-Fi mesh, and / or LTE.

Reader mechanisms (RM), which are just Bluetooth (BLE) enabled anchors, are typically on battery and installed more than 6 metres above ground in a defined environment.

For each defined environment, data of each reader mechanism (RM) is combined to form the second node sensed data (N2). This data is transmitted to a communicably coupled processor (P) with a rule engine (RE) and an associated notification engine (NE).

The reader mechanisms (RM), or anchors, are: located at entry point(s) of the defined environment in order to tag a person’s unique identifier and time of entry, creating a second node based sensed entry data; located at exit point(s) of the defined environment in order to tag a person’s unique identifier and time of entry, creating a second node based sensed exit data; located at pre-defined locations throughout the defined environment in order to tag a person’s unique identifier and time of entry, creating a second node based sensed movement data.

In at least an embodiment, there is provided a plurality of third nodes (N3), with each third node (N3) formed by mobile devices (MD) per person within the defined environment. These mobile devices (MD) are configured to sense and collect physiological measurements through remote photoplethysmography (rPPG). Data obtained from the configured mobile devices (MD) forms the second node sensed data (N2).

In at least an embodiment, the mobile devices (MD) may comprise a set of second sensors, the sensors being non-contact sensors selected from a group of sensors consisting of heart rate variability sensors, heart rate sensors, body temperature sensors, skin temperature sensors, and the like sensors. Typically, the mobile device (MD) is configured with sensing and collecting physiological measurements. For each person, data of each mobile device (MD) is combined to form the third node sensed data (N3). This data is transmitted to a communicably coupled processor (P) with a rule engine (RE) and an associated notification engine (NE).

In at least an embodiment, there is provided a processor (P) communicably coupled with a rule engine (RE) to receive data from the first set of nodes (Nl), the second set of nodes (N2), and the third set of nodes (N3) to process this data in accordance with a rule engine (RE), which effectively and intelligently, learns / updates its rules based on input data.

In at least an embodiment, the rule engine (RE) is configured to determine rules relating to temperature spikes, allowable temperature ranges, disallowable temperature ranges, and instances where the rule engine (RE) needs to communicate with the notification engine (NE) to raise a person-identified flag of alarm in terms of temperature faults (TF).

In at least an embodiment, the rule engine (RE) is configured to determine rules relating to distances between persons identified through readings from reader mechanisms (RM) in association with persons’ wearable mechanisms (WM), allowable distances in terms of such readings, and instances where the rule engine needs to communicate with the notification engine (NE) to raise a person(s)-identified flag of alarm in terms of social distancing faults (DF).

In at least an embodiment, the rule engine (RE) is configured to determine rules relating to defining various zones within the defined environment in concurrence with the reader mechanisms (RM) per zone. A grouping module (GM) is engaged in order to group a set of reader mechanisms (RM) per zone and rules may be configured with respect to such defined zones. For example, the cafeteria, work area, and toilet may be labeled first, second, and third zones respectively. Each zone would have different rules for fault determination- In terms of social distancing, the cafeteria may have the highest fault determination thresholds due to its people density, followed by the work area and toilet.

In at least an embodiment, the rule engine (RE) is configured to determine rules relating to reader mechanisms (RM) paired with sanitization stations for frequency of use of a person, in turn determined by the person’s wearable mechanism (WM). This rule engine (RE) is configured to determine allowable frequencies of use, and instances where the rule engine (RE) needs to communicate with the notification engine (NE) to raise a person(s)-identified flag of alarm in terms of sanitization faults (SF).

In at least an embodiment, there is provided a display mechanism configured to receive a person’s notifications from the processor (P) in terms of determined faults such as temperature faults (TF), social distancing faults (DF), and / or sanitization faults (SF).

In at least an embodiment, there is provided a logging mechanism configured to log information pertaining to history of a user’s movements. This log is important for contact tracing.

FIGURE 3 illustrates the artificial intelligence used by the processor (P) of the system and method of the invention as shown in FIGURE 2.

Using data from the mobile devices (MD) and the wearable mechanisms (WM) communicating with the reader mechanisms (RM), the processor (P), of this invention, is configured to perform predictive analysis in order to deduce deviation in health parameters before actual deviation takes place.

In at least an embodiment, a first input to the processor (P) is the first node based sensed data (obtained from the wearable mechanisms), the second node based sensed data (obtained from the reader mechanisms), and the third node based sensed data (obtained from the mobile devices); all of which form incoming undefined data which is paired with user identity and user profile.

In at least an embodiment, a classifier is configured to classify incoming first input data.

In at least an embodiment, a second input (admin dashboard) to the processor (P) (server) is user-defined data in terms of thresholds of various parameters in order to conclusively define temperature faults, social distancing faults, and / or sanitization faults. In at least an embodiment, the processor (P) (server), preferably, uses a RESTful API as an interface to collate the first input, the second input, and input from the admin dashboard. Data sets are defined as per international norms.

In at least an embodiment of the processor (P) (server), a training module, with training data, is applied to the second input data. With a machine learning module, anomalies are, subsequently, classified within a population using the defined environment over a period of time to determine discrepancies in any one or a multitude of nodes / parameters defined by this system and method.

FIGURE 4 is a flow diagram in accordance with one embodiment of the invention.

Figure 4 illustrates a flow diagram depicting a scenario where a “first risk potential level” and a “second risk potential level” is determined by the system, of Figure 2, which is implemented, in accordance with an embodiment of the invention.

In a first step (STEP 4a), a determination of a user’s physiological parameters is conducted using their mobile device (MD).

In a second step (STEP 4b), a “first risk potential level” (RPL1) is determined from the sensed physiological parameters obtained from the wearable mechanisms (WM) and / or the mobile devices (MD). There are typically three “first risk potential levels” (RPL1)- High (4b.1), Medium (4b.2), and Low (4b.3). The level is ascertained if any sensed parameter crosses the administrator-defined ranges of values for a particular level. For example, for heart rate, values ranging from 0 to 90 bpm qualify the person’s “first risk potential level” (RPL1) as low, values ranging from 90 to 110 bpm as medium; and values above 110 bpm as high. In the event that other physiological parameters qualify the person’s risk potential as medium or high, the person’s risk potential will take the highest possible level among all physiological parameters.

In a third step (STEP 4c), the risk potential is assessed to determine whether the user should be granted entry into the setting.

If the risk potential is high, the process ends at the STEP 4c and the user is requested to leave the setting.

If the risk potential is medium, the user is asked to re-assess his / her physiological parameters in 5 minutes.

Subsequently, if the risk potential remains at medium or high, the process ends at the step and the user is requested to leave the setting.

If the risk potential is low, the user is granted entry into the setting and given a wearable mechanism (WM).

In a fourth step (STEP 4d), a determination of the user’s distance to other users is conducted. An instance of “social distance non-compliance” (SDNC) is determined according to administrator-defined thresholds.

If the user crosses the threshold, the wearable mechanism (WM) vibrates and alerts the user to adhere to distance protocols.

If the user does not cross the threshold, the user will not be alerted by the wearable mechanism (WM).

In a fifth step (STEP 4e), the user is required to perform regular checks of their physiological parameters.

Following the first few steps, the user’s physiological parameters are sensed, collated, and processed to calculate the “first risk potential score” (RPL1), again, which is used to determine whether the user is allowed to remain in the setting or not.

FIGURE 5 illustrates network architecture for the monitoring and analysis of various parameters, of a user, in accordance with an embodiment of the invention.

In at least an embodiment, the mobile device (MD) used by the user comprises a mobile application which contains this invention’s software development kit associated with the system and method of this invention. In some embodiments, this is configured to measure (sense) physiological parameters, through remote photoplethysmography. This sensed data is, then, passed through the artificial intelligence algorithm which resides on the processor (P) and is enabled by the rule engine (RE). The processor (P) is configured to compute at least the “first risk score” (RPS1) and at least the instances of “social distance non-compliance” (SDNC); each of these scores being correlated with administrator-defined thresholds in order to configure the notification engine (NE), accordingly. Third-party API links (API) can be added for additional functionalities; e.g. voice-based emotion analytics, access to telemedicine services, and digital certification of fit-for-work records.

In some embodiments, a dashboard (D) is provided which can be a web interface that allows the system to interact directly with a user or indirectly through their mobile devices (MD) and / or their wearable mechanism (WM). The user of the system may include, but is not limited to, a client using the system to determine the risk potential (RPL1, SDNC) to its processes and an administrator (A) for configuration of the system (typically, in terms of the various thresholds). The dashboard (D) can enable the system to communicate with other computing devices such as web servers and external data servers (shown through API links). The dashboard can facilitate communications within a wide variety of networks and protocol types, including wired and wireless networks. The dashboard (D) may include one or more ports for connecting a number of devices to one another or to another server. In one embodiment, a display module is configured to compile and display all users’ parameters. In another embodiment, a risk scoring module is configured to allow modification of administrator-defined thresholds for risk classification and categorising the risk potential based on physiological parameters received from the mobile device (MD).

FIGURE 6 illustrates a diagram of the architectural content in an application of the mobile devices (MD 1 , MD2) of the system, according to one embodiment of this invention.

The Video Acquisition modules (603, 604), associated with their respective mobile wrappers (601, 602), extract video feed from input devices which are user’s mobile devices (MD1, MD2). The mobile wrappers (601, 602) feed the frames into a software development kit (SDK) library (605) responsible for processing. A Session Manager (SM) within the SDK library (L) manages the processing sessions and initiates video feed processing in the Session System. The Session System (SS) is broadly composed of four modules: (i) face detection and tracking module (606), (ii) image processing module (607), (iii) PPG extraction module (608), and (iv) PPG signal processing module (609). The final outputs of the Session System (SS) are heart rate (HRC), heart rate variability (HRVC), oxygen saturation (02C), and respiratory rate (RRC). These values are passed back to the Session Manager (SM), which communicates with the mobile wrappers (601, 602) that, in turn, share the final outputs with its respective mobile device (MD1, MD2).

The SDK is composed of two main components: i) a PPG signal extraction and processing component (SM, PC), and ii) a physiological parameters calculation component (02C, HRC, HRVC, RRC).

Under the PPG signal extraction and processing component, there are two subcomponents: a Session Manager (SM), and a PPG Calculator (PC). The SM component contains a face tracking algorithm, which identifies regions of interests from the face (forehead, left and right cheek) and extracts RGB signals from video feed of a mobile device (MD)’s camera. These signals are then passed down to the PC component, which contains a signal calculation, a signal interpolation, and a signal pre-filtering module as well as a PPG signals storage bucket. The signal calculation module averages RGB values across all pixels to give temporal RGB signals. These RGB signals are passed down to the signal interpolation module, which applies mathematical functions to the RGB signals to combine the 3D RGB signals into ID PPG signals and subsequently transform the PPG signal into the frequency domain. The PPG signals are passed down to the signal pre filtering module, which applies filters to remove any outlier signals. The resulting signals are stored in the PPG signals storage bucket.

In at least an embodiment of the SDK, the physiological parameters calculation component contains a multitude of calculation algorithms for each of the sensed physiological parameters, which include, but is not limited to, an oxygen saturation calculator (02C), a heart rate calculator (HRC), a heart rate variability calculator (HRVC), and respiratory rate calculator (RRC). The component retrieves PPG signals from the PPG signals storage bucket of the PC module, which are passed down to the calculation algorithms to compute the physiological measurements. In a specific embodiment, the heart calculator (HRC) is coupled with the heart rate variability calculator (HRVC).

In at least an embodiment, the system of this invention is configured to:

1) create a heatmap of real time users who are experiencing influenza like illnesses (ILI); and

2) create a forecast of users who are most likely going to experience illness / symptoms in the next 3 days.

This information can:

1) help hospitals plan and direct resources to address any ILI clusters;

2) aid government in whole-of-government approaches to isolating and eradicating viruses causing such illnesses; and

3) aid in whole-of-society initiatives by providing users with actionable data to self- isolate when the probability is high, and self-quarantine if symptoms are confirmed.

Thus, the system and method of this invention is able to avoid another critical event that blindsided the medical community by actively monitoring confirmed cases of diseases such as Covid-19 from discovery all the way through to recovery.

Thus, the system and method of this invention is able to provide granulized data which can be used for circuit breaker measures to be applied in a very localized context (in estates instead of island-wide / nation-wide) in the future in order to control and eradicate viruses.

Thus, the system and method of this invention is able to provide actionable data which can be presented to users to make informed decisions to stay at home and avoid gatherings regardless of how small the gathering size is - simply by providing information that they are at a higher risk of falling sick within the next 3 days.

The TECHNICAL ADVANCEMENT of the invention lies in providing an intelligent, self learning, self-aware defined environment - enabled by the system and method of this invention - in order to provide an automated compliance determined defined environment to be used by persons in a safe, secure, reliable, and non-intrusive manner.

Because this is a patent document, general broad rules of construction should be applied when reading it. Everything described and shown in this document is an example of subject matter falling within the scope of the claims, appended below. Any specific structural and functional details disclosed herein are merely for purposes of describing how to make and use examples. Several different embodiments and methods not specifically disclosed herein may fall within the claim scope; as such, the claims may be embodied in many alternate forms and should not be construed as limited to only examples set forth herein.

It will be understood that, although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited to any order by these terms. These terms are used only to distinguish one element from another; where there are “second” or higher ordinals, there merely must be that many number of elements, without necessarily any difference or other relationship. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element, without departing from the scope of example embodiments or methods. As used herein, the term “and/or” includes all combinations of one or more of the associated listed items. The use of “etc.” is defined as “et cetera” and indicates the inclusion of all other elements belonging to the same group of the preceding items, in any “and/or” combinations.

In accordance with many embodiments of the present invention, each of the users / parties associated with the system comprise the necessary electronic devices, having platforms and databases where applicable, to execute the methods as set forth by embodiments of the present invention. Alternative system architectures are contemplated by embodiments of the present invention provided such alternative architectures are capable of executing the various methods disclosed herein.

The entire system and method of this invention is enabled on a network having a plurality of nodes, the nodes being configured as end points accessible each of the first user/s, the second user/s, and the third user/s. The network may comprise any network suitable for embodiments of the present invention. For example, the network may be a partial or full deployment of most any communication / computer network or link, including any of, any multiple of, any combination of or any combination of multiples of a public or private, terrestrial wireless or satellite, and wireline networks or links. The network may include, for example, network elements from a Public Switch Telephone Network (PSTN), the Internet, core and proprietary public networks, wireless voice and packet-data networks, such as 1G, 2G, 2.5G, 3G and 4G telecommunication networks, wireless office telephone systems (WOTS) and / or wireless local area networks (WLANs), including, Bluetooth and/ or IEEE 802.11 WLANs, wireless personal area networks (WPANs), wireless metropolitan area networks (WMANs) and the like; virtual local area networks (VLANs) and/ or communication links, such as Universal Serial Bus (USB) links; parallel port links, Firewire links, RS-232 links, RS-485 links, Controller- Area Network (CAN) links, and the like.

The data, in each of the components, means, modules, mechanisms, units, devices of the system and method may be encrypted and suitably decrypted when required.

The systems described herein can be made accessible through a portal or an interface which is a part of, or may be connected to, an internal network or an external network, such as the Internet or any similar portal. The portals or interfaces are accessed by one or more of users through an electronic device, whereby the user may send and receive data to the portal or interface which gets stored in at least one memory device or at least one data storage device or at least one server, and utilizes at least one processing unit.

The portal or interface in combination with one or more of memory device, data storage device, processing unit and serves, form an embedded computing setup, and may be used by, or used in, one or more of a non-transitory, computer readable medium. In at least one embodiment, the embedded computing setup and optionally one or more of a non-transitory, computer readable medium, in relation with, and in combination with the said portal or interface forms one of the systems of the invention. Typical examples of a portal or interface may be selected from but is not limited to a website, an executable software program or a software application.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude or rule out the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

Although a few implementations have been described in detail above, other modifications are possible. In addition, the logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve desirable results. In addition, other actions may be provided, or actions may be eliminated, from the described flows, and other components may be added to, or removed from, the described systems. Accordingly, other implementations are within the scope of the following claims.

Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As one example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In certain implementations, multitasking and parallel processing may be advantageous.

While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

While this detailed description has disclosed certain specific embodiments for illustrative purposes, various modifications will be apparent to those skilled in the art which do not constitute departures from the spirit and scope of the invention as defined in the following claims, and it is to be distinctly understood that the foregoing descriptive matter is to be interpreted merely as illustrative of the invention and not as a limitation.

Claims

CLAIMS,

1. An intelligently networked system for proactively monitoring various parameters, of one or more stochastic behaviours of users, in a defined environment, said system comprising: a defined environment configured in respect of one or more reader mechanisms (RM), installed at pre-defined locations, in said defined environment, each reader mechanism (RM) being configured to read a wearable mechanism (WM) tagged to a user; one or more wearable mechanisms (WM), each wearable mechanism (WM) being tagged to a corresponding user in said defined environment; one or more mobile devices (MD), each mobile device (MD) being associated to a corresponding user in said defined environment; monitoring a first stochastic behaviour, of said user, using sensed data of said mobile device (MD) associated with said user, in said defined environment, in terms of at least a first risk potential level (RPL1); monitoring a second stochastic behaviour, of said user, using sensed data of said wearable mechanism (WM) tagged to said user, in said defined environment, in terms of at least an instance of social distance non-compliance (SDNC); a communicably coupled processor (P) being configured with a rule engine (RE) and a notification engine (NE), said processor (P) configured to receive: o sensed data from each of said wearable mechanisms (WM) in association with user identity; o sensed data from each of said mobile devices (MD) in association with user identity; o sensed data from each of said reader mechanisms (RM) in association with defined environment identity; said rule engine (RE) configured to define rules in relation to thresholds, along with rules in relation to severity of breach of said defined thresholds, for one or more parameters associated with said first risk potential level (RPL1) and for one or more parameters associated with said social distance non-compliance (SDNC); and said notification engine (NE) configured to provide notifications in relation to breach of one or more thresholds, along with severity of breach of said thresholds, along with identity of associated user.

2. The system as claimed in claim 1, wherein said defined environment is configured with reader mechanisms (RM), one or more reader mechanisms being located: at entry point(s) of said defined environment in order to tag, a second node based sensed entry data, in terms of person identifier and time; at exit point(s) of said defined environment in order to tag, a second node based sensed exit data, in terms of person identifier and time; and at pre-defined locations throughout said defined environment in order to tag, a second node based person-movement data, in terms of person identifier and time.

3. The system as claimed in claim 1, wherein said wearable mechanisms (WM) are tags, associated with corresponding unique identifiers, which communicate with said reader mechanisms (RM).

4. The system as claimed in claim 1, wherein said wearable mechanisms (WM) comprise a set of first sensors, the sensors are, contact sensors, pulse oximeter sensors, body temperature sensors, skin temperature sensors, and fitness tracking sensors, with sensing being done at pre-determined discrete time intervals.

5. The system as claimed in claim 1, wherein said wearable mechanisms (WM) are configured to collect distance measurements at pre-defined time frequencies and further configured by said notification engine (NE) along with said rule engine (RE) to send alerts to a corresponding user wearing said wearable mechanism (WM), which are dependent on the severity of threshold breach defined in said rule engine (RE).

6. The system as claimed in claim 1, wherein said wearable mechanisms (WM) are Ultra -Wide Band (UWB) real time locating systems that comprise a location tag for a person, or location anchors.

7. The system as claimed in claim 1, wherein, said reader mechanisms (RM), that are a combination of Ultra-Wide Band (UWB) enabled mechanisms and Bluetooth (BLE) enabled mechanisms, are installed near a ceiling of a defined environment.

8. The system as claimed in claim 1, wherein said reader mechanisms (RM), that are Bluetooth (BLE) enabled mechanism, are installed more than 6 metres above ground in a defined environment.

9. The system as claimed in claim 1, wherein said mobile devices (MD) are configured with a set of second sensors, that sense one or more physiological parameters of a user; that is selected from a group of sensors, which include heart rate variability sensors, heart rate sensors, body temperature sensors, and skin temperature sensors.

10. The system as claimed in claim 1, wherein said rule engine (RE) is configured to determine rules, and corresponding notifications, in respect of stochastic behaviour, in terms of temperature, of a user, said rule engine (RE) being configured to: determine rules relating to temperature spikes as sensed by said mobile devices (MD) and / or said wearable mechanisms (WM), allowable temperature ranges as sensed by said mobile devices (MD) and / or said wearable mechanisms (WM), disallowable temperature ranges as sensed by said mobile devices (MD) and / or said wearable mechanisms (WM), and instances where said rule engine (RE) needs to communicate with said notification engine (NE) to raise a user-identified flag of alarm in terms of temperature faults (TF).

11. The system as claimed in claim 1 , wherein said rule engine (RE) is configured to determine rules, and corresponding notifications, in respect of stochastic behaviour, in terms of space, of a user, said rule engine (RE) being configured to: determine rules relating to distances between persons identified by determining readings provided by said reader mechanisms (RM) in association with users’ wearable mechanisms (WM), allowable distances in terms of such readings, and instances where said rule engine (RE) needs to communicate with said notification engine (NE) to raise a user-identified flag of alarm in terms of social distancing faults (DF); and determine rules relating to defining various zones within said defined environment in concurrence with said reader mechanisms (RM) per zone.

12. The system as claimed in claim 1, wherein said rule engine (RE) is configured to determine rules, and corresponding notifications, in respect of stochastic behaviour, in terms of patterns, of a user, said rule engine (RE) being configured to: determine rules relating to a user’s frequency of use of sanitization stations paired with said reader mechanisms (RM), that is determined by said user’s wearable mechanism (WM) over a period of time, allowable thresholds in terms of such readings, and instances where said rule engine needs to communicate with said notification engine (NE) to raise a user-identified flag of alarm in terms of sanitization faults (SF).

13. The system as claimed in claim 1, wherein said system comprising a grouping module (GM) engaged in order to group a set of reader mechanisms (RM) per zone and rules being configured with respect to such defined zones by said rule engine (RE).

14. The system as claimed in claim 1, wherein said wearable mechanisms (WM) or said mobile devices (MD) are configured with a display mechanism (D) to receive notifications from said processor (P), in terms of determined faults being temperature faults (TF), social distancing faults (DF), and / or sanitization faults (SF).

15. The system as claimed in claim 1, wherein said processor (P) is configured to receive sensed data from: at least a first input obtained from said wearable mechanisms (WM), said reader mechanisms (RM), and said mobile devices (MD); and at least a second input, obtained from a dashboard, in relation to thresholds for each parameter of sensed data, in order to determine temperature faults, social distancing faults, and / or sanitization faults.

16. The system as claimed in claim 1, wherein said “first risk potential level” (RPL1) is determined from the sensed physiological parameters obtained from the wearable mechanisms (WM) and / or the mobile devices (MD), said “first risk potential level” (RPL1) being determined according to administrator-defined thresholds for each sensed parameter.

17. The system as claimed in claim 1, wherein said instance of “social distance non-compliance” (SDNC) is determined from said reader mechanisms (RM) in communication with said tagged wearable mechanisms (WM), said instance of “social distance non-compliance” (SDNC) being determined according to administrator-defined thresholds of disallowable distance between any two tagged wearable mechanisms (WM) in a same defined environment.

18. The system as claimed in claim 1, wherein said mobile device (MD) comprising:

Video Acquisition modules (603, 604), associated with respective mobile wrappers (601, 602), to extract video feed from said mobile devices (MD1, MD2), said mobile wrappers (601, 602) configured to feed frames into a software development kit (SDK) library (605) responsible for processing; a Session Manager (SM) within a SDK library (L) for managing processing sessions and for initiating video feed processing in a Session System; said Session System (SS) comprising: o a face detection and tracking module (606), o am image processing module (607), o a PPG extraction module (608), and o a PPG signal processing module (609); an output module configured to output data from said Session System (SS), said output data comprising heart rate (HRC), heart rate variability (HRVC), oxygen saturation (02C), and respiratory rate (RRC); and said Session Manager (SM) configured to pass said output data to said mobile wrappers (601, 602) that in turn share said outputs with respective mobile devices (MD1, MD2).

19. The system as claimed in claim 16, wherein said system comprising an SDK comprising: a PPG signal extraction and processing component (SM, PC) further comprising: o a Session Manager (SM) containing a face tracking algorithm, which identifies regions of interests from a user’s face (forehead, left and right cheek) and extracts RGB signals from a video feed of said mobile device’s (MD) camera; and o a PPG Calculator (PC) configured to receive data from said Session Manager (SM), said PPG Calculator (PC) containing:

^■ a PPG signals storage bucket to store filtered signals from said signal pre filtering module; and a physiological parameters calculation component (02C, HRC, HRVC, RRC), configured to receive filtered signals from said PPG signals storage bucket, said physiological parameters calculation component comprises a multitude of calculation algorithms for each of the sensed physiological parameters, which include an oxygen saturation calculator (02C), a heart rate calculator (HRC), a heart rate variability calculator (HRVC), and respiratory rate calculator (RRC).