WO2021137702A1

WO2021137702A1 - Combined medical home device

Info

Publication number: WO2021137702A1
Application number: PCT/NL2020/050827
Authority: WO
Inventors: Juul KOENE
Original assignee: Koene Juul
Priority date: 2020-01-03
Filing date: 2020-12-31
Publication date: 2021-07-08
Also published as: NL2024600B1

Abstract

The present invention is in the field of a combined medical home device for performing measurements. These measurements are considered to represent physical characteristics of a user, such blood pressure and heart beat. The device is a handheld device and is simple to use. The device can contain limited or extended functionality.

Description

Combined medical home device FIELD OF THE INVENTION

The present invention is in the field of a combined med ical home device for performing measurements. These measure ments are considered to represent physical characteristics of a user, such blood pressure and heart beat. The device is a handheld device and is simple to use. The device can con tain limited or extended functionality.

BACKGROUND OF THE INVENTION

Typical prior art devices for performing relatively sim ple tasks have one or more of the following drawbacks. They are inaccessible, relate to high cost of care, are inade quate or no insurance coverage is given. In view of inaccu racy, 1.8 Billion Millenials try to self-diagnose on the in ternet, but the internet contains a lot of misinformation.

In view of time consumption, 20 - 30% of all doctors' visits are by hypochondriacs, who are getting more anxiety from the internet but are not (always) sick; Currently 1 in 8 diagno ses by a general practitioner is a misdiagnosis. This is due to the high workload and too little time with each patient; Healthcare is organized around (acute) events. However, health is a continuous process and begins with healthy living and prevention; Due to the cur rent healthcare situation, Dutch healthcare providers are spending roughly 86 billion euro's in healthcare of which around l/8th is related to mental healthcare.

Some partial solutions are provided. A disadvantage may be that only search queries are provided to rely on. Thereby data is inherently limited in extent. Also, as mentioned above, often mis-diagnosis occurs, especially on the inter net, resulting in mental anxiety. Some devices are not port able, and thereby limited in use. And some devices suffer from lack of privacy, and certainly suffer from lack of con trol of own data once provided.

Some documents may be referred to. US 2015/269825 A1 re cites a wireless system monitoring system including a wire less telephone and a wearable appliance in communication with the wireless telephone, the appliance monitoring activ- ity with an accelerometer and one or more additional sensors to detect movement or vital sign. US 6,705,990 B1 recites a method and apparatus for monitoring a plurality of physio logic parameters associated with a living subject. In one aspect, the invention comprises a monitoring apparatus capa ble of non-invasively and continuously monitoring the blood pressure, heart rate (ECG), and weight of the subject. The apparatus further includes a display, input device, and one or more communications links such that data and other infor mation may be communicated between the apparatus and one or more remote locations. In one exemplary embodiment, the ap paratus comprises a portable monitoring station disposed in the subject's home, which is kept in communication with a remote medical facility via a wireless interface and PSTN or data network. Two-way communications of monitored data, vid eo, audio, and other types of information is provided to fa cilitate remote care of the subject, and obviate the need for an in-home caregiver or frequent trips to the medical facility. A communications system for exchanging data be tween the various locations, and computer program allowing such data exchange, are also disclosed. US 2011/301439 A1 recites a personal monitoring device has a sensor assembly configured to sense physiological signals upon contact with a user's skin. The sensor assembly produces electrical sig nals representing the sensed physiological signals. A con verter assembly, integrated with, and electrically connected to the sensor assembly, converts the electrical signals gen erated by the sensor assembly to a frequency modulated inau dible ultrasonic sound signal. The ultrasonic signal is de modulated from an aliased signal produced by under-sampling.

It is an objective of the present invention to overcome disadvantages of the prior art medical devices without jeop ardizing functionality and advantages.

SUMMARY OF THE INVENTION

The present invention relates in a first aspect to a combined medical home device, by which with early inter vention health costs can be reduced by about 50% of the cur rent expenses. The present medical home device is portable and handheld and combines various possibilities of determin- ing characteristics of a human being. It comprises a sensor for obtaining an electrocardiogram, typically at least one sensor, in particular 1-24 sensors, such as 2-12 sensors, e.g. 4-10 sensors, such that heart rhythm, as well as dis turbances therein, artery blood flow, and in particular cor onary artery blood flow, as well as blood component levels, can be obtained, a switch for activating or deactivating the device, which may be a manual switch or a software switch, such as a switch activated by speech, a transceiver for re ceiving and sending data, preferably a wireless transceiver, the transceiver adapted to communicate with a central de vice, typically located elsewhere, such that input data can be analyzed, such as by comparison with reference data, speech recognition software adapted to receive spoken in structions, such that the present device can perform tasks intended to do, an interactive model manager for controlling the device, for providing an interface between device and user, and adapted for providing questions, at least one mi crophone for receiving spoken instruction, typically an ar ray of microphones, in particular 3-24 microphones, such as 6-12 microphones, at least one speaker for providing audio output, typically an array of speakers, at least two data storages, each data storage being physically and functional ly separated from one and another, at least one first data storage for storing received sensor or camera input, and at least one second data storage for storing output data, a processor for processing input data, typically such that personal and private aspects of the data are separated from technical aspects thereof, providing output data, typically relating to the measured data without any reference or link to a user, and for transceiving data, such that the data can be communicated with a typically external located central device, such as a server, or a cloud application, optionally a power source, such as a battery, typically being present, but which may be sold separately, optionally a loader for loading the power source, typically being present, but which may be sold separately, optionally a display, optionally at least one light source, optionally at least one optical indicator, wherein the device is a handheld device and is user adaptable, that is the device can be adapted by a user and can be adapted in view of continued use.

The present device, referred to as "AIVY", is a novel combination of different existing elements. The combina tion of these elements in one product is new. By using high quality building blocks AIVY improves healthcare as we know it today and provides a brighter and more health ier future. Typical applications are given below. Such as, for the millennial who uses the internet to search for their symptoms, present invention an autonomous medi cal device, may give the millennial accurate information about their health at home before they see a doctor. For elderly who say the time of the doctor is valuable and who are having less access to care on a daily base be cause there is a shortage of elderly care takers. For first time parents who are learning how to be a parent and are in need of more information about their newborns health. For people who have a chronic disease and are in need of weekly or monthly checkups. Different from other companies, the present invention may use Big Data (ex tremely large data sets that may be analyzed computation ally to reveal patterns, trends, and associations, espe cially relating to human behavior and interactions), as well as it uses speech recognition (Voice AI) to comfort the user by listening and asking questions, includes mul tiple (build-in) sensors that can be used to measure all vital organs (like heart, skin, eyes, brain, lungs), is able to use third party (sensor) data, like temperature, weight, and air quality and is completely privacy proof because all of the users personal data is stored on de vice. The combination of Big Data and multiple (build-in) sensors provides the user with useful accurate infor mation based on their actual well-being and not based on their internet search terms or assumptions; The speech interface gives the user a human interaction, as if they're talking with the doctor; By using AIVY everyday enough data is collected to get a good picture of the medical history of the user. No matter where they are in the world the user may always have complete access to their own medical history. The information that is col lected is encrypted/ protected and stored in a way like the device is a data/information vault. The user is in complete control over their own medical information.

Illness can be prevented because it's seen earlier or patterns can be recognized if problems recur more; With AIVY, the doctor gets more time for the patients that re ally need it, diagnosis may be better and the user may have more control over their health at home; All this without the violation of the users privacy and security; Aivy is used wireless and autonomously.

Aivy may consist of multiple computer program layers which interface with each other in a correct manner.

There is a conversational layer where Aivy is able to listen and speak to the user. The measurement layer is where Aivy gets data from the user using its internal measurement hardware. And finally analytics layers where Aivy can analyze and report back to the user. All these subsystems are powered by machine learning using natural language processing and understanding to make communica tion between the user and Aivy seamless.

Aivy may comprise a natural language unit (NLU), typ ically in the above conversational layer, which uses an intent recognition algorithm. By implementing a NLU Aivy is able to quickly recognize the intent of text. By cou pling the intent to an action a user can conversate with Aivy in a natural way, and eventually let the device mon itor the user, such as its heart rate. Aivy's data is presented as a manifest, where stories, intents, entities and responses are stored. In a health check story triages conversation are implemented with entities like pain scale and body part.

Aivy may comprise a Text to speech and likewise a Speech to Text unit, typically also in the conversation layer. To transform spoken words into text the open source speech recognition library Pocketsphinx may be used. The converted audio will then trigger an intent, which will trigger an action. This action can also be a textual response which will be transformed back into ac- tual speech. Aivy's microphone array is adapted to pick up speech after the hot word is spoken to wake up the de vice. The hot word will be recognized as the initial in tent .

Aivy uses an application programming interface (API). Using a custom API build using the Python Flask framework Aivy can communicate with the ECG classification system, typically in the measurement layer. By making a HTTP call from a Aivy action to the API, the heart monitor function may be triggered, starting the sensor. Which may stream data to a file on Aivy. This file may be used to start the ECG classifier.

In a first version the device includes an ECG (elec trocardiogram) sensor, typically at least one sensor, in particular 1-24 sensors, such as 2-12 sensors, e.g. 4-10 sensors, . An electrocardiogram records the electrical signals in one's heart. It's a common test used to detect heart problems and monitor the heart's status in many situations. Electrocardiograms — also called ECGs or EKGs — are often done in a doctor's office, a clinic or a hos pital room. And they've become standard equipment in op erating rooms and ambulances. An ECG is a noninvasive, painless test with quick results. This allows us to un derstand the level of physiological arousal that someone is experiencing, but it can also be used to better under stand someone's psychological state. So Aivy is capable of measuring the heartbeat and classifying underlying conditions using what may be considered as essentially an ECG classification algorithm. This algorithm is trained on thousands of heart rhythms from different diagnoses. Using a multi-layered neural network built in the Tensor- flow framework, Aivy uses a default DNN classifier with three layers. Therein the first comprises ten hidden units, a second one comprises twenty hidden units, and the third ten again. It is trained to return one of five classes. For preprocessing a raw waveform is preprocessed first, of which a baseline of the signal is subtracted. Additionally, some noise removal can be done. Two median filters are applied for this purpose, such as of 200-ms and 600-ms filter respectively. Note that these values may depend on the frequency sampling of the signal. The signal resulting from the second filter operation con tains the baseline variations and can be subtracted from the original signal. The annotations of the MIT-BIH ar rhythmia were used in order to detect the R-peak posi tions. Before training the models, all the input data was standardized with a z-score, i.e., the values of each di mension are divided by its standard deviation and sub tracted by its mean.

In the first version the device includes a microphone array with a high quality speaker. In view of speech recognition it is preferred to use a number of micro phones, in particular at least one array of microphones, in order to obtain a required quality and reliability of this speech recognition software. In particular 2 or 3 arrays may be used, each independently having a number of microphones, such as 2-24 microphones as indicated above. Using voice recognition software that runs locally on the device. In the first version the device may as well in clude an infrared night vision camera module. This camera may in the beginning be used for two purposes. One pur pose was to use the camera application in the context of pupil deviation. This to track brain activity. By using pupil recognition software and computing power the dif ference of environmental factors of light on the pupil may be tracked and calculated. The reaction speed may be calculated with the numbers tracked by this formula: rate= Aconcentration/At ime.

The same camera application may be adapted to make pictures of the skin to detect skin diseases, check blood oxygen, body temperature and fat density. The device may be extended with more (complex) sensor applications. Ex amples of these applications are: humidity sensor, air quality sensor, weight sensor, moist sensor, motion sen sor, ultrasonic sensor, blood pressure etc., typically at least one sensor, in particular 1-24 sensors, such as 2-12 sensors, e.g. 4-10 sensors. These extensions can be ap plied as building blocks to the original center piece. In that way AIVY is a modular system that can be build out as ones at home health hub.

The combination of these build-in sensors, additional sensors and sensor data, a speech application, and certi fied health researches in one device is unique. AIVY may include a wireless back-up docking station where in the device can be charged with power and where all the personal information can be back upped at the same time. Here the user can use the device in an autonomous way. The main device may only have one button. This button can be used to either, back up, delete or activate the device. The device may be able to sense you in the room with a motion sensor and react on ones presence without violating ones privacy and secu rity. With this, the device is able to remind the user about actions that had to be taken. For example, going to the doctor, take in medicine or check double for development of dis ease/feeling of the user. Also the user can address the device by calling AIVY.

The device gives new possibilities to the everyday user and improves the users knowledge on their own health. The composition of the different elements is something new and gives opportunity for the user to get accurate infor mation without having to go on the internet or make an appointment at the general practitioner.

AIVY is a solution for the everyday user, the reha bilitating user, and for the user who has no money or ac cess to health care. AIVY is a new product concept with a new composition of available and new technologies. The combination of the sensor application with the speech on device and the data on device is new to home devices.

The reason the present invention is not obvious is the fact that it gives people who fear going to the doc tor for any reason, and people that feel like they are a burden to their doctor, or people who don't have any ac cess to a doctor, a chance to monitor their health. This device can handle the standard tests a doctor can do, for example ECG and a few other basic things. Not only does the device listen to the call of the user but it can also measure and interpret the data. So instead of having to go to the doctor and pay a lot of money and time for this with having the chance that you get sent home, the user gets a better picture about their health at home. There are currently no viable devices on the market that can make the same deductions as mine. It is not that the old way of solving the problem was not a good idea, but it is essential to have as accurate data as possible to make a possible diagnosis, and for a possible diagnosis more time and understanding is needed. By giving the doctor more time for the patients that are in need of more spe cific care, my device can help the people who are in need of a better understanding of their health before they take the step of going to the doctor.

Another reason why present invention is not obvious is that one's personal data is never leaving the device, you may have full control over one's own medical data and decide what to do with it. Present invention is, more accurate be cause it uses different elements of gathered information. With this it can be more specific and it may only use certified third party information. Present invention is more efficient because the user may know immediately what is going on in their life, or have a better view of an other checkup. So they do not have to go to the doctor and make an appointment only get 10 minutes and need to make another appointment or hear that it's not severe enough which is a complete waste of time for the user. With present invention the user doesn't have to leave their home. Present invention is faster because it gives direct feedback to the user. Also because it collects da ta it can recognize patterns earlier and sickness can be prevented instead of being sick and it's too late. Pre sent invention is also cheaper than the average doctors visit. Especially in the US a simple doctors visit can cost between 300-600 dollar. Present invention may cost underneath 199 euro and one may have as many doctors' visits as you like, available 24/7 when you need them and not limited to 10 minutes per visit. One drawback for the user for present invention could be that the user needs to upgrade the device every once in a while to keep the sensors the highest available quality at that moment. A disadvantage of present invention could be that it may take a longer time for practitioners to trust what comes out of present invention because there is a lot of misin formation already out in the world so my device has to proof longer that its actually accurate and useful in the medical world. This disadvantage can be overcome by show ing practitioners what it can do and show them its trust worthy. Medical practitioners are also part of the advi sory board, this may help with the trust.

In a second aspect the present invention relates to a computer program for a stand-alone and combined medical home device comprising instructions for operating the medical home device according to the invention, the in structions causing the medical home device to carry out the following steps: providing instructions to a user, activating the sensor for obtaining an electrocardiogram measuring and obtain ing said electrocardiogram, storing said electrocardio gram in the first storage, processing input data and providing output data to the second storage, and trans mitting said output data to a central processing unit.

Thereby the present invention provides a solution to one or more of the above mentioned problems.

Advantages of the present description are detailed throughout the description.

DETAILED DESCRIPTION OF THE INVENTION

In an exemplary embodiment the present combined medical home device may comprise an optical camera adapted to meas ure a pupil reflex upon applying light.

In an exemplary embodiment the present combined medical home device may comprise an optical camera adapted to meas ure skin disorders.

In an exemplary embodiment the present combined medical home device may comprise a blood sensor for measuring at least one of oxygen content, and blood constituents, typi cally at least one sensor, in particular 1-24 sensors, such as 2-12 sensors, e.g. 4-10 sensors.

In an exemplary embodiment the present combined medical home device may comprise a temperature sensor, such as an ear sensor, typically at least one sensor, in particular 1- 24 sensors, such as 2-12 sensors, e.g. 4-10 sensors.

In an exemplary embodiment the present combined medical home device may comprise a pressure sensor, typically at least one sensor, in particular 1-24 sensors, such as 2-12 sensors, e.g. 4-10 sensors.

In an exemplary embodiment the present combined medical home device may comprise a motion sensor, typically at least one sensor, in particular 1-24 sensors, such as 2-12 sen sors, e.g. 4-10 sensors.

In an exemplary embodiment the present combined medical home device may comprise at least one air-quality sensor, in particular an oxygen concentration sensor, a CO2 concentra tion sensor, a NO_x concentration sensor, a CO concentration sensor, a humidity sensor, a particle sensor, in particular a <100 pm particle sensor, and a pollen sensor.

In an exemplary embodiment the present combined medical home device may comprise a clock.

In an exemplary embodiment of the present combined medi cal home device output data may be anonymized by removing at least use and device characteristics from input data.

In an exemplary embodiment of the present combined medi cal home device the device may be autonomous and/or stand alone.

In an exemplary embodiment of the present combined medi cal home device the interactive model manager may be adapted to provide instructions to a user.

In an exemplary embodiment of the present combined medi cal home device the optical camera may comprise at least one infrared sensor.

In an exemplary embodiment the present combined medical home device may comprise 1-24 cameras, in particular 2-10 cameras.

In an exemplary embodiment the present combined medical home device may comprise 1-360 light sources, in particular 2-120 light sources.

In an exemplary embodiment the present combined medical home device may comprise 1-360 optical indicators, in par ticular 2-120 optical indicators.

In an exemplary embodiment the present combined medical home device cameras, light sources, sensors, microphones. Microphone arrays, speakers, etc. may be distributed evenly over a surface of the present device, or may be distributed unevenly, such as having a concentration of sensors in a specific direction, or a combination thereof.

In an exemplary embodiment of the present combined medi cal home device the interactive model manager comprises at least two computer program layers which interface with each other, preferably at least three layers.

In an exemplary embodiment of the present combined medi cal home device the interactive model manager comprises a first conversational layer adapted to receive spoken in structions from a user and to provide audio and optical output to the user, a second measurement layer adapted to receive user measurement data obtained from the user us ing the combined medical home device, and an analytics layer adapted to analyze said measurement data and to provide measurement data based output to the user, and an optional visual layer.

So the interface layer of Aivy may Exists of the fol lowing layers:

The conversational layer. This typically contains at least one microphone (array) and at least one speaker ar ray and uses natural languages processing to have a hu manized conversation with the user. This conversational layer records, interprets, deletes, and gives feedback based on the users input. As a result of the conversa tional layer typically follows an action, translating in to the next layer.

The measurement layer. Depending on the action that came forward out of the conversational layer the measure ment layer turns on, one or multiple sensors that will perform the measurements that came forward as an action from the conversational layer. The data that is acquired by these measurements is then taken over by the data pro cessing layer. In this layer the data is processed into accurate and understandable data and then given back to the conversational layer and/or the visible layer. Where the data is translated into human text and turned into speech to explain the output to the user.

Speech is typically the main input and output inter face for the user. Next to the speech input/output, con versational layer there is a secondary layer that in cludes a visible or visual layer. This visible layer will display additional information and reports that are asked by the user. This layer can take part and be a whole of the surface of Aivy. Giving centralized information but not limited by a single flat screen.

The visible layer may only be part of the interface as an additional layer to the conversational layer. The visible layer typically cannot exists without the conver sation layer.

In an exemplary embodiment of the present combined medi cal home device the interactive model manager comprises at least one machine learning routine, preferably a machine learning routing per computer program layer.

In an exemplary embodiment of the present combined medi cal home device the device has a substantially spherical shape, such as comprising a multitude of hexagonal faces and/or a plurality of pentagonal faces.

In an exemplary embodiment of the present combined medi cal home device each face individually comprises at least one of a microphone, a microphone array, a sensor, an opti cal indicator, a light source, a speaker, at least one elec trical contact for providing power and for operating each individual face, and combinations thereof, in particular a light source or light indicator at an outer side of the face, a sensor located there beneath, a speaker and/or mi crophone (array) located under the sensor, and thereunder comprising a centrally sub-surface located computer brain. The sensor may also be located in between speakers and/or microphone (arrays). In an exemplary embodiment of the present combined medi cal home device at least one of a microphone, a microphone array, a sensor, an optical indicator, a light source, a speaker, and further elements may be distributed evenly over the surface of the device, such as 1-6 per 60 degrees in any direction.

In an exemplary embodiment the present computer program may further comprise instructions for comparing input data with reference data.

The invention is further detailed by the accompanying figures and examples, which are exemplary and explanatory of nature and are not limiting the scope of the inven tion. To the person skilled in the art it may be clear that many variants, being obvious or not, may be conceiv able falling within the scope of protection, defined by the present claims.

The invention although described in detailed explana tory context may be best understood in conjunction with the accompanying examples and figures.

SUMMARY OF FIGURES

Fig. la-b, 2a,b,3a,b, 4a,b and 5-15 show views of the present device. Fig. 16 shows an exemplary code part. Fig. 17-19 show an exemplary embodiment.

DETAILED DESCRIPTION OF FIGURES In the figures:

Drawings of the invention explained.

0, a,b,c,d : AIVY complete by itself, Health Hub that speaks, has multiple build-in sensors, and uses additional sensor data and medical researches to give us full control over our health at home.

1, a,b,c,d : Camera application used to make pictures and videos of different body parts.

2, a,b,c,d,e,f : ECG (electrocardiogram) sensor. An electro cardiogram records the electrical signals in your heart.

3, a,b,c : Indication lights

4,a,b : Speaker array

5, a,b,c,d,e,f : Motion sensor 6 : On/Off/Reset button

7, a,b,c,d: (Night vision) Camera for pupil analyze 8, a,b,c,d : Light and infrared application for pupil meas urement

9 : Pupil

10 : Light ring, rainbow spectrum, interactive ring that gets a different color with different actions and can be personalized by the user when in regular conversation.

11a,b : Aluminum surface that can be charged wireless and can be used to transmit data to the backup docking station.

12 : Docking for device to wirelessly charge and transfer data to backup.

13 : Data back up and storage compartment

14 : Encryption

15 : Wireless charging

16 : Eye overlap of contraction and dilation of pupil

17 : Difference between size of pupil with dilation or con traction. This difference will be tracked with the camera application and then can be used to calculate whether the response is good enough or that something is wrong with the brain of the user. The camera will measure while making a video recording.

18 : Difference and change in iris

19a,b,c : Pressure sensor, for the device to check if you hold it right

20 : Movement towards the device. Hold the device with two hand and the device can do an ECG.

21a,b,c : Holding the device, an additional accelerometer and ultrasonic sensor will check the distance and whether you hold the device in the right way.

22 : Sink, context bathroom

23 : Mirror, context bathroom

24 : Human

25 : Lamp, context bedroom

26 : Bed, context bedroom

27 : (Smart) Phone, platform with data that one may include in the device (third party data).

28 : (Smart) Watch, platform with data that one may include in the device (third party data).

29 : (Smart) Temperature measurement device, platform with data that one may include in the device (third party data). 30 : (Smart) Humidifier, platform with data that one may in clude in the device (third party data).

31 : Connected Home, all useful data that you collect in your home with a diversity of smart devices. (Smart Heater, Smart Light, Air Quality sensor, Radiation etc.)

32 : (Smart) Scale, platform with data that one may include in the device (third party data).

33 : Additional sensors that the user will be able to add to AIVY and which are completely private and secure.

34 : Additional platforms from other parties the user is able to add data from to make their own medical information as complete as possible. (The more useful data the better the device can analyze)

35 : Personal interface for the user to include data without using voice.

36 : Personal Data Encrypted hardware, will never leave the device

37 : Anonymous Data Encrypted hardware, this data will be checked for diagnosis inside a personal protected cloud. No person from outside is able to come in. No one can interact with the data without triple verification. High Banking/ Credit card security protocol in place.

38 : Cloud Server where the analysis takes place. No user data is stored here. No one can interact with the data with out triple verification. High Banking/Credit card security protocol in place.

39 : Voice input from user, Voice output from device

40 : Visual/data input from user, Visual/ data output from device, (with consent of the user)

41 : Data vault, High encryption protocol.

42 : Interactive model manager. The user interaction, call to action, voice backend.

43a,b,c : Sensors included in the Hardware

44 : AIVY divided software and hardware

45 a,b,c,d,e: Microphone array

46 :Night stand

47a,b : Sphere LED surface

48 :The Aivy can be turned 360 degrees.

49 :Component place for processing Figure Clarification

Fig. la: The front of an Aivy. An medical handheld de vice with a diameter of around 10 cm. The front shows the indication lights [ 3, 3a, 3b ], showing the user when it is on and when it speaks. It also shows the Electrocardiogram pads on the left and right side [ 2, 2a ] these are activat ed by a voice command or by a conversation from the device and the user that results in an action. The surface pointed out at [4] is a complete surface of speakers that make sure that it does not matter where the user stands in the room, the sound will be equally as good. On the left side seen above the pad there is a [1] camera application used to make pictures and videos of different body parts. Next to the camera application there is a [8] light and infrared appli cation that among other things can take pupil measurements. On the top side there is a [5] motion sensor, that is able to react when the user comes in the room and can command Aivy to give a reminder to the user, when these are availa ble. Then around [5] there is [10] a light ring, rainbow spectrum, interactive ring that gets a different colors with different actions and can be personalized by the user when in regular conversation. Integrated in this light ring there is a [45] microphone array. The microphone array is needed for the device to understand the user from each side of the room. Together these elements form an Aivy, a health hub that speaks and interacts in a human way, has multiple build-in sensors, and uses additional sensor data and medi cal researches to give the user full control over their health at home.

Fig. lb: The back of an Aivy, with similar features as shown in Fig. la. It shows the Electrocardiogram pads on the left and right side [ 2b, 2c ] these are activated by a voice command or by a conversation from the device and the user that results in an action. The surface pointed out at [4a] is a complete surface of speakers that make sure that it does not matter where the user stands in the room, the sound will be equally as good. On the right side seen above the pad there is a [la] camera application used to make pic tures and videos of different body parts. On the top side there is a [5a] motion sensor, that is able to react when the user comes in the room and can command Aivy to give a reminder to the user, when these are available. Then around [5a] there is [10a] a light ring, rainbow spectrum, interac tive ring that gets a different colors with different ac tions and can be personalized by the user when in regular conversation. Integrated in this light ring there is a [45a] microphone array. The microphone array is needed for the de vice to understand the user from each side of the room. Then additional to the back of an Aivy there is [6] a on/off/reset button. This button is really small and is able to turn the Aivy off additional to the voice command. It can turn the Aivy on when for example needed by setup and it can give the Aivy a soft reset. This in case the hardware is in need of a refresh. Together these elements form an Aivy, a health hub that speaks and interacts in a human way, has multiple build-in sensors, and uses additional sensor data and medical researches to give the user full control over their health at home.

Fig. 2a corresponds to Fig. la.

Fig. 2b corresponds to Fig. lb.

Fig. 3a: The top of an Aivy existing of different layers visible at [5b, 10b, 45b]. Where [5b] shows a motion sensor, that is able to react when the user comes in the room and can command Aivy to give a reminder to the user, when these are available. To continue with [10b] a light ring, rainbow spectrum, interactive ring that gets a different colors with different actions and can be personalized by the user when in regular conversation. Integrated in this light ring there is a [45b] microphone array. The microphone array is needed for the device to understand the user from each side of the room.

Fig. 3b: The bottom of an Aivy exists out of [11] an aluminum surface that can be charged wireless and can be used to transmit data to the backup docking station.

Fig. 4a: The health hub Aivy is visible on a [12] dock ing for the device to wirelessly charge and transfer data to backup.

Fig. 4b: The health hub Aivy is separate from the dock- ing in [12]. The user can take the device off the docking to use it wirelessly and autonomously and when the user is done, the Aivy can be put on the docking to charge and to back up the data gained while used. The docking has multiple key elements that make it private and secure. Among other things it has [13] a backup system in place that makes sure there is always a backup of the users medical file. Then there is an [41] encryption protocol in place, also called a data vault that makes sure all data is encrypted and se cured. Not accessible to others but the user. Existing out of at least two separate compartments. Then there is [14] another encryption protocol and [15] the possibility to wireless charge the Aivy.

Fig. 5: The interaction that the user does with an Aivy to check the pupillary reflexes. The user picks the device up from the docking station [Fig. 4a & 4b], brings the cam era with guidance of the device towards the left or right eye and then the device will put on the camera, calculate, measure and analyses and then brings back the results in spoken word in an accessible way. The location of which eye and distance depends on the instructions given by the Aivy. Instructions will differ depending on what needs to be meas ured. The same interaction as shown in [Fig. 5] can be used for the measurement of the pupillary reflexes as well as for an eye infection surrounding the eye.

Clarification on the numbers shown. In [lb] there is the camera application used to make pictures and videos of dif ferent body parts. The [8a] light and infrared application combined with the [7] (Nightvision) Camera can take among other things pupil measurements. The [9] in the eye repre sents the pupil. The indication lights [ 3, 3a, 3b ], to gether with the speakers and voice Ai will guide the user through the process of taking measurements.

Fig. 6: Close up of an Aivy checking the pupillary re flexes. The explanation of numbers [7a, 8b, 9a] can be found in Fig. 5. One of the two electrocardiogram pads is shown by [2d]. These are activated by a voice command or by a conver sation from the device and the user that results in an ac tion. Fig. 7: Part of the camera application explained in Fig. 5, here you see the [8c] light and infrared application com bined with the [7b] (Nightvision) Camera that among other things can take pupil measurements. The measurements are shown in [16] the eye overlap of contraction and dilation of pupil. Where [17] explains the difference between size of pupil with dilation or contraction. This difference will be tracked with the camera application and then can be used to calculate whether the response is good enough or that some thing is wrong with among other things the brain of the us er. The camera will measure while making a video recording. The arrow at [18] shows the difference and change in iris.

Fig. 8: AIVY schematics, divided software and hardware elements. Showing the layers and how they connect.

Fig. 9: The interaction the user does when taking ECG measurements. Movement towards the device shown by [20]. The user picks the device up with both hands touching the ECG pads [2d]. The device contains a diversity of [19] pressure sensors. As well in the ECG pads. The device will understand if the user holds the device right with the right amount of pressure to take accurate measurements. The numbers [5c&d, lOc&d, 45c&d] correspond to Fig 3a. The left representation of hands holding an Aivy show the number [21a] while holding the device, an additional accelerometer and ultrasonic sen sor will check the distance and direction the user holds the device. Whether you hold the device in the right way.

Fig. 10: Bathroom environment, one environment represen tation not only or restricted by representation where an Aivy is used. In this figure [0] represents an Aivy complete by itself, and interacts in a human way, has multiple build- in sensors, and uses additional sensor data and medical re searches to give the user full control over their health at home. In this figure Aivy is presented as an additional hab it. Right next to when the user [24] brushes its teeth, they can also do a health check. This health check can be done on a daily basis. Gaining data that is able to predict and pre vent diseases. In this figure [23] represents a mirror and [22] represents a bathroom sink. Fig. 11: Bedroom environment, one environment represen tation not only or restricted by representation where an Aivy is used. In this figure [0a] represents an Aivy com plete by itself, and interacts in a human way, has multiple build-in sensors, and uses additional sensor data and medi cal researches to give the user full control over their health at home. In this figure Aivy is presented as an addi tional habit. Right next to the bed on the night stand. When the user [24a] gets ready for bed they can also do a health check. This health check can be done on a daily basis. Gain ing data that is able to predict and prevent diseases. In this figure [26] represents the bed, [25] represents a night lamp and [46] represents a night stand.

Fig. 12: This figure contains [0b] an Aivy complete by itself, and interacts in a human way, has multiple build-in sensors, and uses additional sensor data and medical re searches to give the user full control over their health at home. It shows different additional features that can be add in a later stage and are not included in the beginning. The following features can be included and attached in or on an Aivy. A [27] (smart) phone and [28] (smart) watch, both platforms with data that you can include in the device, add ing third party data. Additional you can add an [29] (Smart) temperature measurement device and [30] (smart) humidifier, both platforms that generate (human) data that you can in clude in the device, adding third party data. Then if the user has additional features in and around their home, they can add all their [31] connected home, useful data with a diversity of smart devices. ( for example a smart heater, smart light, air quality sensor, radiation, but not limited by these examples) Additional data that can be added to an Aivy is data from a [32] (smart) scale. A lot of people al ready have a scale present so it will be easy to add that data. Making the users medical file as complete as possible.

Fig. 13: Diversity of sensors included in the hardware of the first version of Aivy.

Fig. 14: Different connections are made more visible here. [0c] an Aivy complete by itself, and interacts in a human way, has multiple build-in sensors, and uses addition- al sensor data and medical researches to give the user full control over their health at home. It shows [33] additional sensors that the user will be able to add to AIVY and which are completely private and secure. [34] Additional platforms from other parties the user is able to add data from to make their own medical information as complete as possible. (The more useful data the better the device can analyze). [35]

Personal interface for the user to include data without us ing voice. This interface is completely private and secure. Only the user has access to this. [36] Personal data en crypted hardware, will never leave the device. This is one of at least two compartments. Then [37] is the anonymous da ta encrypted hardware, this data will be checked for diagno sis inside a personal protected [38] CloudServer where the analysis takes place. No person from outside is able to come in. No one can interact with the data without triple verifi cation and restrictions. High Banking/ Creditcard security protocol in place. [39] Represents the voice input from the user and voice output from the device. Also call the voice interaction or main interface. [40] represents the visu al/data input from user, visual/ data output from the de vice. (with consent of the user)

Fig. 15: Interactive model manager. [42] The user inter action, call to action, voice backend. A snippet from the voice interaction tree which gets expanded everyday but shows the different steps before the device called for ac tion.

Fig. 16: This is a code snippet part of an Aivy.

Fig. 17: Aivy Sphere The Aivy sphere comprises different hardware layers. In the center there is an aluminum center with the components for processing and analyzing, separate storage compartments and technology to wireless charge and back up. On top of that layer there are speaker arrays that will give the user a 360 degrees sound experience. So it does not matter in which orientation the user holds the de vice, the sound stays equally good. Between the speaker ar rays, there are microphone arrays that are used for Natural Language Processing. The microphone arrays are distributed all over the 360 degrees surface so it does not matter where the user stands, the microphone will always pick up the an swers and commands given by the user. The top layer is the LED light surface that will help with locating different sensors when asked by the user and device. This layer is al so used to communicate additional information.

Then the layer between the aluminum center and the speaker/microphone arrays comprises different inside sen sors, like an accelerometer and other sensors that need more hardware. Then between the speaker/microphone layer and the LED light surface there is another sensor layer with more surface and touch sensors.

When the user is able to turn Aivy 360 degrees, the user is not limited or restricted to the position of what the de vice is in, making it a better modular and autonomous hand held device.

Extracted different hardware layers. In the center there is an aluminum center [lib , 49] with the components to do some processing and analyzing, separate storage compartments and technology to wireless charge and back up. On top of that layer there are [4b] speaker arrays that will give the user a 360 degrees sound experience. So it does not matter how the user holds the device, the sound stays equally as good. Between the speaker arrays, there are [45d] microphone ar rays that are used for Natural Language Processing. The mi crophone arrays are distributed all over the 360 degrees surface so it does not matter where the user stands, the mi crophone will always pick up the answers and commands given by the user. The top layer is the [47a] LED light surface that will help with locating different sensors when asked by the user and device. This layer is also used to communicate extra information. Then the layer between the aluminum cen ter and the speaker/microphone arrays exists out of differ ent inside sensors [lc, 2e, 5e, 7c, 19b, 21b, 43b], like an accelerometer and other sensors that need more hardware.

Then between the speaker/microphone layer and the LED light surface there is another sensor layer with more surface and touch sensors. Fig. 18: An Aivy complete on itself, and interacts in a hu man way, has multiple build-in sensors, and uses additional sensor data and medical researches to give the user full control over their health at home. The different hardware layers and features as shown in Fig. 17 more in an usable setting. The device is rounded and the user is able to hold it. The voice Ai in combination with the LED light surface will guide the user to hold it in the right way for the de vice to measure. Fig. 19: An Aivy complete by itself, and interacts in a human way, has multiple build-in sensors, and uses addition al sensor data and medical researches to give the user full control over their health at home. The Aivy can be turned [48] 360 degrees. The device will instruct the user with voice and light how to keep the device while using it.

Different angles have different sensors included. When the user is able to turn Aivy 360 degrees, the user is not lim ited or restricted to the position of where the device is in, making it a better modular and autonomous hand held de- vice.

It should be appreciated that for commercial appli cation it may be preferable to use one or more variations of the present system, which would similar be to the ones dis closed in the present application and are within the spirit of the invention.

Claims

1. A stand-alone and Combined medical home device comprising a sensor for obtaining an electrocardiogram, a switch for activating or deactivating the device, a transceiver for receiving and sending data, preferably a wireless transceiver, speech recognition software adapted to receive spoken instructions, an interactive model manager for controlling the device, for providing an interface between device and user, and adapted for providing questions, at least one microphone for receiving spoken instruc tion, at least one speaker for providing audio-output, at least two data storages, each data storage being physically and functionally separated from one and another, at least one first data storage for storing received sensor or camera input, and at least one second data storage for storing output data, a processor for processing input data, providing output data, and for transceiving data, optionally a power source, such as a battery, optionally a loader for loading the power source, optionally a display, optionally at least one light source, optionally at least one optical indicator, wherein the device is a handheld device and is user adapt-able.

2. Combined medical home device according to claim 1 com prising an optical camera adapted to measure a pupil reflex upon applying light, and/or wherein the optical camera comprises at least one infrared sensor, and/or comprising an optical camera adapted to measure skin disor ders.

3. Combined medical home device according to any of claims 1-2 comprising a blood sensor for measuring at least one of oxygen content, and blood constituents, and/or comprising a temperature sensor, such as an ear sensor, a pressure sensor, a motion sensor, a clock, and combinations thereof, and/or comprising at least one air-quality sensor, in particular an oxygen concentration sensor, a CO2 concentration sensor, a NO_c concentration sensor, a CO concentration sensor, a hu midity sensor, a particle sensor, in particular a <100 pm particle sensor, and a pollen sensor.

4. Combined medical home device according to any of claims 1-3, wherein output data is anonymized by removing at least use and device characteristics from input data.

5. Combined medical home device according to any of claims 1-4 wherein the device is autonomous and/or stand-alone.

6. Combined medical home device according to any of claims 1-7 wherein the interactive model manager is adapted to pro vide instructions to a user. claims 1-8.

7. Combined medical home device according to claim 6, wherein the interactive model manager comprises at least two computer program layers which interface with each other, preferably at least three layers.

8. Combined medical home device according to claim 7, wherein the interactive model manager comprises a first conversational layer adapted to receive spoken instruc tions from a user and to provide audio and optical output to the user, a second measurement layer adapted to re ceive user measurement data obtained from the user using the combined medical home device, and an analytics layer adapted to analyze said measurement data and to provide measurement data based output to the user, and an option al visual layer.

9. Combined medical home device according to claim 7 or 8, wherein the interactive model manager comprises at least one machine learning routine, preferably a machine learn ing routing per computer program layer.

10. Combined medical home device according to any of claims 1-9, wherein the device has a substantially spherical shape, such as comprising a multitude of hexagonal faces and/or a plurality of pentagonal faces, in particular wherein each face individually comprises at least one of a microphone, a microphone array, a sensor, an optical indicator, a light source, a speaker, at least one electrical contact for providing power and for operating each individual face, and combinations thereof, and/or comprising a centrally sub-surface located computer brain.

11. A computer program for a stand-alone and combined medi cal home device comprising instructions for operating the medical home device according to one of the claims 1-10, the instructions causing the medical home device to carry out the following steps: providing an interface between device and user, the in terface adapted for providing questions and giving instruc tions, providing instructions to a user, receiving spoken instructions from the user, controlling the device using an interactive model manag er for, activating the sensor for obtaining an electrocardiogram and measuring and obtaining said electrocardiogram, storing said electrocardiogram in the first storage, processing input data and providing output data to the second storage, and transmitting said output data to a central processing unit.

12. Computer program according to claim 11, further compris ing instructions for comparing input data with reference da ta.

13. A combined medical home device substantially as de scribed in the description, or in the figures, or in the above claims, or a combination thereof.