WO2018136402A2

WO2018136402A2 - Non intrusive intelligent elderly monitoring system

Info

Publication number: WO2018136402A2
Application number: PCT/US2018/013824
Authority: WO
Inventors: Aardra Kannan AMBILI
Original assignee: Riot Solutions Inc.
Priority date: 2017-01-18
Filing date: 2018-01-16
Publication date: 2018-07-26
Also published as: WO2018136402A3

Abstract

A non- intrusive intelligent elderly monitoring system configured to automatically detect unusual movements and short and long term behavioral changes of occupants in a room and generate alerts when such an event is observed. The system uses a combination of UWB radar sensor (10) and thermal sensor (11) to monitor the movement patterns as well as vital signs of the person, wherein AI module learns the behavior patterns and classifies and generates datasets, whereupon the system generates alerts if any abnormal deviations in behavior patterns or vital signs is observed.

Description

ITED STATES PATENT AND TRADEMARK OFFICE

Non-Provisional Patent Application Entitled: IVE INTELLIGENT ELDERLY MONITORING SYSTEM

Inventor:

AARDRA KANNAN AMBILI

Attorney:

David Postolski, Esq.

Reg. No. 67,547

Innovation Plaza, Suite 1A

41 River Road, Summit NJ 07901

T: (908) 273-0700 F: (908)-273 0711

Attorney Docket No. 2869RSI03PCT

NON INTRUSIVE INTELLIGENT ELDERLY MONITORING SYSTEM

Inventor:

AARDRA KANNAN AMBILI

CLAIM OF PRIORITY

This application claims priority to Indian Application No. 201741001947 filed January 18, 2017, the contents of which are hereby incorporated by reference in its entirety.

FIELD OF THE EMBODIMENTS

The present invention, in general, relates to systems for monitoring any abnormalities in behavioral patterns of occupants in a room. Particularly, the invention relates to a system capable of automatically detecting abnormal patterns in the behavior of occupants in a room by analyzing their movements, vital signs and time based behavioral patterns and notifying a care taker or control center when such an anomaly is observed. BACKGROUND OF THE EMBODIMENTS

US Pat No. 5905436 discloses a situation based monitoring system that monitors various activities of persons in rooms of a home or residential care facility, determines when the person is in distress and communicates that fact to appropriate personnel. But said system does not have any intelligence, does not mention any sensors, does not have digital signal processing (DSP ) or Artificial intelligence (AI) for determining the status of the occupant. It is less robust in the light of current invention. US20140362213 Al teaches a surveillance system for residential buildings that monitors the status of occupants for their position, and movement. But said system uses digital cameras to monitor the occupants, which makes the surveillance intrusive. Another publication US20060055543 Al details a system and method for detecting a period of unusual inactivity of a person. But it uses a generic motion sensor for its operation and it works by observing periodic activity and non-activity to deduce unusual activity of the person concerned. It does not include any Al or intelligence in analyzing the state of the person. US Pat No. 9036019 discloses a fall detection and reporting technology, in which output from at least one sensor configured to sense, in a room of a building, activity associated with a patient falling is monitored and a determination is made to capture one or more images of the room based on the monitoring. However, in said system, the inactivity of the patient is determined using on- body sensors. Further, said method is rather intrusive on the person's privacy.

US Pat No. 7567200 teaches a method and apparatus for body position monitor and fall detection using radar. Said apparatus uses array / cluster of radars/ transmitters/receivers for fall detection. For its operation, some form of calibration and mapping of premises is also recommended which is cumbersome and can cause errors. Also, the prior art uses only one type of sensor for its functions compared to the presented invention. Present invention incorporates artificial intelligence or self-learning features and is free from calibration and is capable of monitoring abnormal behavior of person based on movements, time and vital signals. From the foregoing, it becomes apparent that there is a need for a non-intrusive patient monitoring system that is capable of learning the behavioral patterns including movement patterns of the person over a period of time, and generate alerts if any abnormality in behavioral pattern is observed.

Current invention proposes a non-intrusive intelligent elderly monitoring system configured to automatically detect any abnormalities in behavioral patterns including unusual movement patterns of occupants in a closed space and notify a caretaker when such an anomaly is observed. The scope of current invention includes classifying different types' movements to detect abnormalities, vital sign monitoring, and time attribute based behavioral analysis.

SUMMARY OF THE EMBODIMENTS

Therefore, a purpose of the present disclosure is to solve the aforementioned problems of prior art, that is, to provide an aligning method for dual camera module, capable of aligning the optical axes of the dual camera module with accuracy.

It is therefore the primary objective of the present invention to propose a non-intrusive intelligent elderly monitoring system capable of monitoring the position, movements and time based behavioral patterns of occupants in a room, and generate alerts if any abnormal pattern in behavior of the person is detected.

It is another object of the invention to combine radar output with thermal sensor data to improve the accuracy of detection of possible abnormal movements. It is yet another object of the invention to propose an elderly monitoring system that is capable of self- learning.

It is a further object of the invention to provide an elderly monitoring system that automatically improves its accuracy of detecting movements over time due to self- learning feature

It is yet another object of the invention to provide a system that that monitors the activities or movements of a person in a non- intrusive manner, without affecting his privacy. It is a further object of the invention to propose a system capable of inherently detecting potential situations involving persons requiring assistance.

Accordingly, the present invention proposes a non-intrusive intelligent elderly monitoring system configured to automatically detect unusual movements and short and long term behavioral changes of occupants in a room and generate alerts when such an event is observed, said system comprising of:

a UWB radar sensor system interfaced with a microprocessor system through serial/ parallel communication interface;

a thermal sensor module interfaced with the microprocessor system through serial/parallel communication interface, wherein said microprocessor system is also configured to communicate with cloud services;

a digital signal processing (DSP) module configured to analyse and extract information from the received signals from UWB radar as well as thermal sensor module; an artificial intelligence module interfaced with the DSP, configured to self-learn the behavior patterns, classify and generate datasets, and to detect change in behavioral patterns; and a means to generate alerts and dispatch it through a communication link to a caretaker.

The invention also proposes a method to detect abnormal medical conditions of a person in a room, and generate alerts, said method comprising the steps of:

receiving data from the UWB radar and thermal sensors;

analyzing the received data by the DSP module for vital signal extraction;

analysis and self-learning the vital signs such as breathing during sleeping or resting continually by the AI module,

generating alerts if any abnormal changes in vital signs is observed.

It further provides a method to detect abnormal time based behavior of a person, said method comprising the steps of:

receiving data from the UWB radar and thermal sensors;

analyzing the received data by the DSP module;

communicating the DSP output corresponding to position, movements, and movement types to the AI module;

learning the behavior patterns of the person with respect to time continually by the AI module, and generating data sets; and

generating alerts if deviation from said behavior pattern with respect to time of activity and duration of activity. The other objectives, features, and advantages of the present invention will become more apparent from the ensuing detailed description of the invention, taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Example embodiments will now be described more fully hereinafter with reference to the accompanying drawings; however, they may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the example embodiments to those skilled in the art.

In the drawing figures, dimensions may be exaggerated for clarity of illustration. It will be understood that when an element is referred to as being "between" two elements, it can be the only element between the two elements, or one or more intervening elements may also be present between two elements. Like reference numerals refer to like elements throughout.

FIG.l illustrates the block diagram of the system.

FIG.2 shows the application scenario of the invention.

FIG.3 illustrates the flow chart of the operations of the system.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Reference will now be made in detail to the description of the present subject matter, one or more examples of which are shown in figures. Each example is provided to explain the subject matter and not a limitation. Various changes and modifications obvious to one skilled in the art to which the invention pertains are deemed to be within the spirit, scope and contemplation of the invention. Hereinbelow, specific embodiments of the present disclosure will be explained in detail with reference to the drawings.

The preferred embodiment of the present invention discloses a non-intrusive intelligent elderly monitoring system configured to automatically detect any abnormalities in behavioral patterns including unusual movement patterns of occupants in a closed space or room, and generate alerts to notify a control center or care taker when such an anomaly is observed. The system features a means to generate alerts and dispatch it to the caretaker via a communication link.

Referring to FIG.l, the system comprises of a UWB radar sensor system (10) that works on the principle of transmitting Radio frequency (RF) burst signal and receiving the reflected RF echo signals. In general, UWB radar has a useful range between 0.5m to 10m. It is further configured to process and analyze echo signals to compute the time of flight of echo pulses and hence the position of various objects in its vicinity. The radar sensor system also incorporates required RF circuits and antennas.

As illustrated in FIG.l, the radar system is interfaced with a Microprocessor system (12) through serial/parallel communication interface. The microprocessor system comprises of a high speed microprocessor, memory components for storing program / data, real time clock, and wireless communication links using technologies such as WIFI, Bluetooth, and GSM etc. The DC power required for the operation of various sub components are generated internally by the power supply sub module. The microprocessor is also configured to communicate with internet cloud services (13) or directly to a handheld device like mobile phone (14) wirelessly or both simultaneously.

The microprocessor system in the present embodiment is also interfaced with additional thermal array sensor (11), which senses human presence or movements by detecting infrared radiation emitted from the human body. The thermal sensor also could be an array thermopile sensor with resolution from 8x8 up to 100x100 pixels, to give spatial information in detecting persons and at same time without affecting the privacy of the person concerned.

FIG.2 shows a private room such as a toilet / bath room with a door (2) wherein the sensor system (1) is mounted on the opposite wall. The function of the Ultra- Wide Band (UWB) Radar (10) and thermal sensor (11) is movement sensing and tracking of persons as well as for vital sign monitoring.

The UWB radar sensor in the present embodiment is configured to detect a person entering the room, moving inside the room, other activities such as standing (3, 5), sitting (4), or exiting the room, as well as abnormalities such as falling down, no activity etc. The thermal sensor output is used to improve the accuracy of the detection and tracking process. In the preferred embodiment of the invention, the radar and thermal sensor outputs are combined to improve the detection process, and to generate more accurate results.

A radar sensor cannot distinguish a human body and other objects. But radar data can be processed to differentiate between stationary objects such as chairs and live persons due to the movements of later, in some manner, such as breathing. Also the radar sensor can precisely detect fine movements like breathing and at same time, measure distance (depth information) of various objects around the subject. The thermal sensors, on the other hand, is capable of detecting thermal radiation from human body, and its spatial position in the field of view, even though it is incapable of reading any information related to depth - distance of the person like the radar sensor.

As per the preferred embodiment of the present invention, the radar and thermal sensors are physically mounted in the same enclosure wherein the field of view of both are fixed so that it is possible to superimpose the two types of data. In other words, the combined sensor system is capable of reading both the distance information as well as confirm the presence of human body spatially in its field of view. In one embodiment, the sensor system is capable of operation with one sensor also.

Referring to FIG.2, once the system is installed in a room, data is collected over several days or weeks' time, wherein said data is used to generate the behavioral pattern and conditions of the persons in the room using AI (artificial intelligence - machine learning algorithms.). During initial training period, the user has to mark days of any abnormal behavior pattern so that the AI system understands and self learns only normal behavioral conditions. The AI algorithm is configured to self- learn based on the data it receives, wherein it is used to detect following situations:

1 . Abnormal movements and falls:

After the self- learning process, the system is capable of detecting abnormal movements of person such as shivering, jerks, and violent movements, and conditions such as falls that may indicate a medical condition requiring immediate medical care. The major difference with prior art is that the system learns and understands normal conditions, movements of the person etc. and then uses it to classify movements and medical conditions. Subsequently if the AI engine detects abnormalities or deviations from the regular patterns, it gives alerts to the caretaker. 2. Abnormal behavior based on time of day and duration:

During normal life of persons (elderly), the day based activities usually follows a regular pattern, like getting up from bed, day activities, time spend for each activity, and time of each activity, as well as night time sleep patterns. The system is configured to learn such patterns, and give an alert based on deviations from said behavior patterns. This could indicate a gradually emerging medical situation that needs attention.

3. Vital sign monitoring

The proposed system is also configured to detect vital signs such as breathing patterns, and heart rate during sleeping or resting. The major difference with prior art is that the system learns and understands normal and abnormal medical conditions. Subsequently, if the AI engine detects abnormalities or deviations from the regular patterns, it gives alerts to the caretaker. The non- intrusive intelligent elderly monitoring system is also configured to receive data from external sensors such as wearable or portable ECG units and wrist bands. The present invention features open communication links such as Bluetooth to communicate with such devices, which makes it also suitable for bedrooms and other similar places. In this case, the AI engine is also configured to collect said additional information and analyze the data, along with regular data, for real time and long term processing, and alarm / medical data generation.

Once an abnormal activity is detected by the system, same information is used locally / remotely to notify the caretaker in a suitable way, using one or more of the communication links including WIFi, BLE, or GSM. In another aspect of the invention, a siren / buzzer also forms a part of the system wherein said siren/buzzer is also sounded to alert nearby care takers.

AI algorithm

As per the preferred embodiment of the present invention, the radar and thermal sensor collects and documents the daily activities of the person monitored and feeds the data into the microprocessor. Said processor is configured to self-learn using artificial intelligence algorithms. These in-built AI algorithms are capable of picking up patterns from the data sets and use it for self-learning purpose and at same time use said data to check for abnormal conditions as well. The attributes in the data sets comprises of time function, movement category, and vital signs, wherein the time function attributes further includes time of event as well as duration of event. The AI algorithm is configured to self-learn from movement, presence and respiration patterns of the person detected through the sensor.

The non-intrusive intelligent elderly monitoring system is configured to learn what the elderly person usually does in his or her daily life, wherein it is capable of figuring out whether the person has deviated significantly from his or her natural patterns with respect to time. For example, the system is capable of issuing a warning if the elderly person has spent more than his usual amount of time sleeping and subsequently alerts a caregiver who could check on the concerned person. Similarly, the system is configured to detect an absence of movement for an unusual duration, wherein the system will alert the caregiver if the anomaly persists. These findings are necessary since people with age often are unable to report their health issues or in the event of an emergency alert a caregiver. The system' s self- learning intelligent algorithms are customized to each individual' s personal needs and patterns, as each person would have their own unique behavioral patterns.

In another embodiment, the system is configured to gather information from external sensors like ECG, wearables also, to get more medical parameters for better analysis, predictions and alert generation.

Referring to FIG.3, the sensor system continuously collects radar data and thermal sensor data in real time. The system at power on initializes the sensors with required parameters for its operation, including, but not limited to, sampling rate, distance range, RF pulse transmission, and reception parameters. It then collects the information over serial/parallel communication interfaces from the sensors. The data is collected at sampling rates from 20 to 50 data sets per second, at distance typically ranging from 1 to 8 meters, which will vary with user installation situation, size of room, etc. Other sensor parameters will also vary according the application conditions described above.

The system then performs various digital signal processing (DSP) operations on the signal or data received as shown in FIG.3. The DSP operations includes removing back ground noise, wherein it removes stationary objects in the room so that the moving objects can be clearly detected. Also the DSP routines extracts amplitude and phase information of reflected radar signals, wherein said information is further used for subsequent analysis to extract meaningful information and data sets to feed into the AI engine. Said DSP operations include, but not limited to, different filtering techniques, spectrum analyses, mathematical and logic operations, and envelop analysis. Output data from DSP module corresponds to position, movement detection, tracking, movement types, and vital signs of person in the room. There will be hundreds of data parameters characterizing this movement patterns.

The thermal sensor data is also processed by the DSP routines to extract signal parameters. Another Program in the present embodiment will correlate the radar data with the thermal information, wherein it is used as additional information to be fed into the AI engine.

The AI engine features machine learning algorithms, and generates datasets during the training process. The self-learning process is done in a continuous mode, during normal operation of the device, even after initial training. The datasets also has information about the real time of the day wherein the behavior pattern is correlated with time as well. The AI engine or module is configured to run on the processor system itself or in the Cloud or partially in both depending on the required functionality of the product model. AI engine also checks for changes in behavioral patterns and marks them depending upon priority, wherein some of said changes is informed to user as an alarm in real time basis like fall detection, lack of vital signs, no movements etc. The system is also configured to flag some gradual behavioral change patterns for long term Medical analysis.

Any alarm condition detected is immediately passed to the care taker depending on the preferred programmed mode of communication. Emergency situations can call for help using mobile phones or sirens / Buzzers or remote notification to hospitals / remote caretakers.

The same principle can be extended to patient monitoring also, for cases when patients require long term treatments lasting months or years.

Although the present invention has been described in connection with the preferred embodiments thereof with reference to the accompanying drawings, it is to be noted that various changes and modifications are possible and are apparent to those skilled in the art. Such changes and modifications are to be understood as included within the scope of the present invention unless they depart there from.

When introducing elements of the present disclosure or the embodiment(s) thereof, the articles "a," "an," and "the" are intended to mean that there are one or more of the elements. Similarly, the adjective "another," when used to introduce an element, is intended to mean one or more elements. The terms "including" and "having" are intended to be inclusive such that there may be additional elements other than the listed elements.

In the drawings and specification, there have been disclosed typical embodiments of the invention, and although specific terms are employed, they are used in a generic and descriptive sense only and not for purposes of limitation. It will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention as defined by the following claims.

Claims

WHAT IS CLAIMED IS:

1. A non- intrusive intelligent elderly monitoring system configured to automatically detect unusual movements and short and long term behavioral changes of occupants in a closed space or room and generate alerts when such an event is observed, said system comprising of:

a UWB radar sensor system (10) interfaced with a microprocessor system (12) through serial/parallel communication interface;

a thermal sensor module (11) interfaced with the microprocessor system through serial/parallel communication interface, wherein said microprocessor system is also configured to communicate with cloud services;

a digital signal processing (DSP) module configured to analyze, process and extract information from the received signals from UWB radar as well as thermal sensor module;

an artificial intelligence (AI) module interfaced with the DSP module, configured to self-learn the behavior patterns, classify and generate datasets, and to detect change in behavioral patterns; and

a means to generate alerts and dispatch it through a communication link to a caretaker.

2. The system as claimed in claim 1, wherein the DSP module extracts information corresponding to position, movements, tracking, movement types, and vital signs from the received data.

3. The system as claimed in claim 1, wherein the AI module is implemented in the cloud (13) or the microprocessor system or partially in both.

4. The system as claimed in claim 1, wherein the data from the radar and thermal sensors is combined to detect the spatial position of the person as well as its depth or range and movements in the field of view.

5. The method as claimed in claim 1, wherein the attributes in the datasets comprises of time function, movement category, and vital signs.

6. The system as claimed in claim 1 further features at least one interface to receive data from external sensors like ECG, medical wristbands, wherein said sensor data is also fed to the AI module.

7. The system as claimed in claim 1, wherein the sensors are configured to collect data in non- intrusive manner.

8. A method to detect abnormal body movements of a person in a fixed field of view, said method comprising the steps of:

receiving data from the UWB radar and the sensors;

analyzing the received data by the DSP module;

communicating the DSP output corresponding to movement detection, tracking, movement types, and vital signal monitoring to the AI module; classifying and generating data sets by the AI module after self-learning the behavior patterns; and

generating alerts if any abnormal behavior pattern or deviation from the regular behavior pattern is observed.

9. The method as claimed in claim 8, wherein abnormal behavior patterns include, but not limited to, shivering, jerks, violent movements and emergency situation like falls.

10. A method to detect abnormal time based behavior of a person, said method comprising the steps of:

receiving data from the UWB radar and thermal sensors;

analyzing the received data by the DSP module;

communicating the DSP output corresponding to position, and activities of the person, to the AI module;

learning the behavior patterns of the person with respect to time continually by the

AI module, and generating data sets; and

generating alerts if deviation from said behavior pattern with respect to time function attribute is observed.

11. The method as claimed in claim 10, wherein the time function attributes comprises of time of event and duration of event.

12. The method as claimed in claim 10, wherein the AI module is configured to self-learn activities such as time of getting up from bed, going to sleep, day activities or similar activities, and time duration spend for each of the said activity of the person, wherein it generates an alert if deviation from said behavior pattern is observed.

13. The method as claimed in claim 11, wherein the time based attributes provide information on gradually deteriorating medical conditions as well.

14. A method to detect abnormal medical conditions of a person in a room, and generate alerts, said method comprising the steps of:

receiving data from the UWB radar and thermal sensors;

analyzing the received data by the DSP module for vital signal extraction;

analysis and self-learning the vital signs such as breathing during sleeping or resting continually by the AI module; and

Generating alerts if any abnormal changes in vital signs is observed.

15. The method as claimed in claim 5, wherein the vital signs include breathing rates, heart rates or other vital medical data such as ECG, heart rate etc. gathered form built in or external sensors.

16. The method as claimed in claim 14, wherein the vital signs include breathing rates, heart rates or other vital medical data such as ECG, heart rate etc. gathered form built in or external sensors.