WO2021209989A1 - Improvement of disease prevention, diagnosis and treatment by application of multiple information sources - Google Patents

Improvement of disease prevention, diagnosis and treatment by application of multiple information sources Download PDF

Info

Publication number
WO2021209989A1
WO2021209989A1 PCT/IL2021/050418 IL2021050418W WO2021209989A1 WO 2021209989 A1 WO2021209989 A1 WO 2021209989A1 IL 2021050418 W IL2021050418 W IL 2021050418W WO 2021209989 A1 WO2021209989 A1 WO 2021209989A1
Authority
WO
WIPO (PCT)
Prior art keywords
algorithm
datum
condition
lifestyle
user
Prior art date
Application number
PCT/IL2021/050418
Other languages
French (fr)
Inventor
Zakharia FRENKEL
Andrey GORIN
Original Assignee
Healthspace Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Healthspace Ltd filed Critical Healthspace Ltd
Publication of WO2021209989A1 publication Critical patent/WO2021209989A1/en

Links

Classifications

    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H20/00ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
    • G16H20/70ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mental therapies, e.g. psychological therapy or autogenous training
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/10Services
    • G06Q50/22Social work
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H10/00ICT specially adapted for the handling or processing of patient-related medical or healthcare data
    • G16H10/60ICT specially adapted for the handling or processing of patient-related medical or healthcare data for patient-specific data, e.g. for electronic patient records
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00Apparatus for radiation diagnosis, e.g. combined with radiation therapy equipment
    • A61B6/44Constructional features of apparatus for radiation diagnosis
    • A61B6/4417Constructional features of apparatus for radiation diagnosis related to combined acquisition of different diagnostic modalities
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/44Constructional features of the ultrasonic, sonic or infrasonic diagnostic device
    • A61B8/4416Constructional features of the ultrasonic, sonic or infrasonic diagnostic device related to combined acquisition of different diagnostic modalities, e.g. combination of ultrasound and X-ray acquisitions

Abstract

A computer-implemented method of monitoring and analyzing a lifestyle of a user and predicting health implications thereon comprises steps of: (a) inputting at least one datum relating to the lifestyle; the at least one datum selected from the group consisting of: a medical examination datum, a training session datum, a food intake datum, a subjective life-style assessment datum, an environmental impact datum and any combination thereof; (b) storing input data; (c) analyzing the input data; and (d) predicting the health implications of the lifestyle defined by the input data.

Description

IMPROVEMENT OF DISEASE PREVENTION, DIAGNOSIS AND TREATMENT BY APPLICATION OF MULTIPLE INFORMATION SOURCES
FIELD OF THE INVENTION
The present invention generally pertains to a system and method for collection, analysis and utilization of medically-related, behaviorally-related and environmentally-related information which can be useful for predicting, preventing, diagnosing and treating a medical condition or for providing guidance toward lifestyle choices beneficial for short- and long-term health outcomes.
BACKGROUND OF THE INVENTION
In the prior art, medical information typically is amassed either via questioning of a patient by a medical professional, with the questions and the patient's responses, as recorded by the medical professional, added to a medical history, or via one or more tests, also added to a medical history. A test, in this sense, can be, for example, an observation (e.g., greying brown hair, blue eyes, dark- skinned European), anthropometric data (height, weight, etc.), a physiological measurement (blood pressure, heart rate, blood oxygen level, etc.), or a chemical test (hemoglobin level, cholesterol level, genetic test, etc.)
It is well-known that lifestyle choices such as, but not limited to, smoking, alcohol consumption, drag consumption, and food choices can affect a person's health. A medical professional does not normally have the time to question a patient in depth about such lifestyle choices. In addition, a person may well mis-remember actions, consider information unimportant, or suppress information about less-desirable behavior.
It is also well-known that present lifestyle choices can affect future health. For example, smoking combined with low vegetable intake can be a predictor of a higher probability of cancers such as lung cancer and promiscuity can be a predictor of a higher probability of cancers of the reproductive system.
It is well-known that human environment such as, but not limited to, mineral composition of drinking water, presence of toxic, carcinogenic or allergenic matters in the air, water, etc., presence of the harmful radiation, pathogenic microbes, noise and so on can affect a person's present and future health. This may manifest itself, for example, in that people, even non-relatives, and sometimes their children, living at the same time in the same place, can discover, after a certain number of years, that they have contracted the same diseases. Often, medical statistics will show the presence of such health-associated factors. However, medical professionals usually are not able to investigate a patient's environmental conditions and history in depth.
It is also well-known that many diseases have several years' history of development. Often, in the early stages, a disease is barely detectable and has only a very small effect on the person's well- being. The "early symptoms", such as, but not limited to, weak, vague and hard-to-describe pains, tiredness, mood changes, feelings that “something wrong with me”, intimate conditions, sleep quality, etc. are very commonly not taken in to account by people, and are typically not reported to the doctor. Consideration of possible "early symptoms" together with lifestyle choice details could provide a good level of disease prediction and early detection of disease.
It is therefore a long felt need to provide a system which does not rely on a person's ability to recall action taken a considerable time in the past, which does not require frequent visits to a medical professional, and which does not require long or complex data entry.
SUMMARY OF THE INVENTION
It is an object of the present invention to disclose a computer-implemented method of monitoring and analyzing a lifestyle of a user and predicting health implications thereon. The aforesaid method comprises steps of: (a) inputting at least one datum relating to said lifestyle; said at least one datum selected from the group consisting of: a medical examination datum, a training session datum, a food intake datum, a subjective life-style assessment datum, an environmental impact datum and any combination thereof; (b) storing input data; (c) analyzing said input data; and (d) predicting said health implications of said lifestyle defined by said input data.
Another object of the present invention is to disclose the step of storing said input data comprising storing said input data in an adaptive data collection tree in a Boolean form.
A further object of the present invention is to disclose the method comprising a step of selecting user's features of interest.
A further object of the present invention is to disclose the step of storing said input data comprising storing selected user's features of interest in said adaptive data collection tree.
A further object of the present invention is to disclose the step of predicting said health implications which comprises performing Chi squared test.
A further object of the present invention is to disclose the said medical examination datum selected from the group consisting of height, weight, body mass index, eye color, hair color, skin color, blood test result, an imaging test result and any combination thereof.
A further object of the present invention is to disclose the subjective life-style assessment datum selected from the group consisting of a location of a body sensation, intensity said body sensation, a start time of said body sensation, a start location of said body sensation; variability of said body sensation, an end time of said body sensation, an end location of said body sensation, a body location of a condition, intensity of said condition, a start time of said condition, a start location of said condition; variability of said condition, an end time of said condition, an end location of said condition, a medical action performed in relation to said sensation, a medical action performed in relation to said condition, a medical procedure performed in relation to said sensation, a medical procedure performed in relation to said condition, a sleep-and-wake pattern and any combination thereof.
A further object of the present invention is to disclose the environmental impact datum selected from the group consisting of a timeline datum, friend's news, a social media post, a family history datum, a number of pets, a type of said pet and any combination thereof
A further object of the present invention is to disclose the step of analyzing said input data comprising an algorithm selected from the group consisting of a principal component analysis (PCA) algorithm, a factor analysis algorithm, a correlation analysis algorithm, a regression algorithm, a clustering algorithm, a decision tree algorithm, an adaptive decision tree algorithm, a Fisher linear discriminant algorithm, a ^-nearest neighbors algorithm, a neural network, a convolutional neural networks, a support vector machine algorithm, a Naïve Bayes classifiers algorithm, an adaptive boosting algorithm, a supervised learning algorithm, an unsupervised learning algorithm, a semi-supervised learning algorithm, a reinforcement learning algorithm , an association rule algorithm, a deep learning algorithm and any combination thereof.
A further object of the present invention is to disclose the clustering algorithm selected from the group consisting of k-means, partitioning around medoids, empirical modeling, spectral clustering and any combination thereof.
A further object of the present invention is to disclose the step of predicting said health implications comprising determining a probable upcoming health condition.
A further object of the present invention is to disclose the probable upcoming health condition selected from the group consisting of a heart attack, bowel cancer, lung cancer and any combination thereof.
A further object of the present invention is to disclose the step of predicting said health implications comprising a step of identifying a correlation between at least one lifestyle change and a health condition.
A further object of the present invention is to disclose the lifestyle change selected from the group consisting of a change in exercise activity, smoking, food consumption, sleeping pattern and any combination thereof.
A further object of the present invention is to disclose the step of inputting at least one datum relating to said lifestyle comprising imaging a food to be eaten a predetermined distance and estimating a caloric value thereof.
A further object of the present invention is to disclose the computer-implemented system for monitoring and analyzing a lifestyle of a user and predicting health implications thereon; said system comprising: (a) a server configured for processing input data; (b) at least one user's device connected to said server; said at least one user's device configured for inputting data by said user and displaying at least one datum relating to said lifestyle processed by said server; said at least one datum selected from the group consisting of: a medical examination datum, a training session datum, a food intake datum, a subjective life-style assessment datum, an environmental impact datum and any combination thereof; (c) a memory configured for storing input data said memory storing instructions to said server which, when executed by said server, direct said server to: (i) receiving said at least one datum relating to said lifestyle; (ii) storing input data; (iii) analyzing said input data; and (iv) predicting said health implications of said lifestyle defined by said input data. A further object of the present invention is to disclose the input data are stored in an adaptive data collection tree in a Boolean form.
A further object of the present invention is to disclose the instructions comprising selecting user's features of interest.
A further object of the present invention is to disclose the step of storing said input data comprising storing selected user's features of interest in said adaptive data collection tree.
A further object of the present invention is to disclose the instruction of predicting said health implications comprising performing Chi squared test.
A further object of the present invention is to disclose the system comprising an imaging unit configured for imaging a food to be eaten at a predetermined distance and estimating a caloric value thereof.
A further object of the present invention is to disclose the at least one user's device selected from the group consisting of a smartphone, a desktop computer, a laptop computer, a tablet computer, a game console and any combination thereof.
A further object of the present invention is to disclose a method of training an algorithm of predicting health implications. The aforesaid method comprises steps of: (a) imaging each meal before eating; (b) identifying an imaged meal; (c) determining dimensions of identified components and caloric value thereof; (d) recording data relating to a dietary pattern, physical- exercise activities and a sleep pattern; (e) collecting and storing recorded data; (f) building an algorithm of predicting health implications.
A further object of the present invention is to disclose the step of building an algorithm performed on the basis of machine learning or a neural network.
A further object of the present invention is to disclose the health implications comprising change in user's weight depending on a dietary pattern, physical-exercise activities and a sleep pattern.
BRIEF DESCRIPTION OF THE FIGURES In order to better understand the invention and its implementation in practice, a plurality of embodiments will now be described, by way of non-limiting example only, with reference to the accompanying drawings, wherein
Fig. 1 is a flowchart of a method of monitoring and analyzing a lifestyle of a user and predicting health implications thereon;
Fig. 2 is a schematic presentation of data processing in a system for monitoring and analyzing a lifestyle of a user and predicting health implications thereon;
Fig. 3 is a schematic diagram of a system for monitoring and analyzing a lifestyle of a user and predicting health implications thereon;
Fig. 4A-E schematically illustrates an embodiment of a portion of a data collection tree;
Fig. 5 illustrates exemplary information input according to the hierarchical structure of the information;
Figs. 6-8, 9A-B, 10, 11A-B, 12 and 13 illustrate an exemplary embodiment of screens presented to a user;
Figs. 14A-B schematically illustrates changing the order of items so that more-used items are closer to the top;
Fig. 15 is a schematic flowchart of a method of estimating a caloric value of a meal to be eaten;
Fig. 16 is a schematic flowchart of a method imaging a meal to be eaten;
Fig. 17 is a schematic presentation of calibration of an imaging device;
Fig. 18 illustrates analysis of a food to eaten; and
Fig. 19 is a flowchart of two-dimensional wavelet form transformation.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
The following description is provided, alongside all chapters of the present invention, so as to enable any person skilled in the art to make use of said invention and sets forth the best modes contemplated by the inventor of carrying out this invention. Various modifications, however, will remain apparent to those skilled in the art, since the generic principles of the present invention have been defined specifically to provide a means and method for collection, analysis and utilization of medically-related, behaviorally-related and environmentally-related information which can be useful for prediction, preventing, diagnosing and treating a medical condition or for providing guidance toward lifestyle choices beneficial for short and long term health outcomes.
The term ‘user' hereinafter refers to a person who is the concern of the service of the present invention. A user will be referred to as a ‘patient' when receiving or registered to receive medical treatment or medical diagnosis.
The term ‘subject' hereinafter refers to the abovementioned person when functionally connected to or interacting with any external non-medical device, software or medium.
The term ‘medical condition' hereinafter refers to a disorder, an unhealthy state or a condition which reduces the medical fitness of a person. A medical condition can be a disease, a lesion, a disorder, an illness, an injury and any combination thereof. A person may contemporaneously or simultaneously be any combination of a user, a subject and a patient within the definitions set out in this description.
The term ‘decision tree' hereinafter refers to a classifier expressed as a recursive partition of the instance space. The decision tree consists of nodes that form a rooted tree, meaning it is a directed tree with a node called “roof' that has no incoming edges. All other nodes have exactly one incoming edge. A node with outgoing edges is called an internal or test node. All other nodes are called leaves (also known as terminal or decision nodes). In a decision tree, each internal node splits the instance space into two or more sub-spaces according to a certain discrete function of the input attributes values. In the simplest and most frequent case, each test considers a single attribute, such that the instance space is partitioned according to the attribute's value. In the case of numeric attributes, the condition refers to a range. Each leaf is assigned to one class representing the most appropriate target value. Alternatively, the leaf may hold a probability vector indicating the probability of the target attribute having a certain value. Instances are classified by navigating them from the root of the tree down to a leaf, according to the outcome of the tests along the path.
The term ‘adaptive decision tree' hereinafter refers to a decision tree which is adaptive to a changing environment and is capable of machine learning and adaptation to said changing environment. The term ‘data collection tree' or 'tree' hereinafter refers to a rooted tree wherein the terminating nodes (“leaves”) correspond to specific data, whereas the internal nodes do not necessarily correspond to specific data.
The term ‘adaptive data collection tree' hereinafter refers to a data collection tree which is adaptive to a changing environment and is capable of machine learning and adaptation to said changing environment.
In the prior art, medical information typically is amassed either via questioning of a patient by a medical professional, with the questions and the patient's responses, as recorded by the medical professional, added to a medical history, or via one or more tests, also added to a medical history. A test, in this sense, can be, for example, an observation (e.g., greying brown hair, blue eyes, dark- skinned European), anthropometric data (height, weight, etc.), a physiological measurement (blood pressure, heart rate, blood oxygen level, etc.), or a chemical test (hemoglobin level, cholesterol level, genetic test, etc.)
It is well-known that lifestyle choices such as, but not limited to, smoking, alcohol consumption, drug consumption, and food choices can affect a person's health. A medical professional does not normally have the time to question a patient in depth about such lifestyle choices. In addition, a person may well mis-remember actions, consider information unimportant, or suppress information about less-desirable behavior.
It is also well-known that present lifestyle choices can affect future health. For example, smoking combined with low vegetable intake can be a predictor of a higher probability of cancers such as lung cancer and promiscuity can be a predictor of a higher probability of cancers of the reproductive system.
It is well-known that human environment such as, but not limited to, mineral composition of drinking water, presence of toxic, carcinogenic or allergenic matters in the air, water, etc., presence of the harmful radiation, pathogenic microbes, noise and so on can affect a person's present and future health. This may manifest itself, for example, in that (even non-relatives) people living at the same time at the same place, after a certain number of years they discover the same diseases in themselves (or even in their children). Often, the presence of such health-associated factors is reflected in medical statistics obtained from the correspondent place. A medical professional does not usually able to investigate patient environmental conditions and history in depth. It is also well-known that many of diseases have several years' history of development. Often, in the early stages, a disease is barely detectable and has only a very small effect on the person's well-being. The "early symptoms", such as, but not limited to, weak, vague and hard-to-desciibe pains, tiredness, mood changes, feelings that “something wrong with me”, intimate conditions, sleep quality, etc. are very commonly not taken in to account by people, and are typically not reported to the doctor. Consideration of possible "early symptoms" together with lifestyle choice details could provide a good level of disease prediction and early detection of disease.
The present invention, therefore, comprises a system which, for each user, automatically accesses multiple information sources and automatically amasses from these sources information that is medically relevant, or that may possibly be medically relevant. By analysis of at least a subset of this information from one or multiple patients, early detection of at least one disease can be done, at least one prediction can be made of at least one of the following: the development of at least one disease or at least one ameliorable health condition; and, , analysis of the data can be used to determine at least one important factor responsible for the appearance of at least one disease or at least one ameliorable health condition. Such a detection and a prediction can be provided to medical personnel and to the user.
In some embodiments, if analysis shows that a user has an increased likelihood of development of a medical condition, one or more simulations can be run. Such a simulation would change one or more factors of the patient's behavior or lifestyle and can determine therefrom a probability of development of at least one medical condition if the behavior(s) or lifestyle factor(s) are changed. For non-limiting example, if analysis shows that a user is “on course” for obesity, five simulations could be run, one in which the user reduces the size of the hamburger he buys at lunch and purchases a salad with the hamburger, one in which the patient has a brisk walk for half an hour before buying his usual lunch, one in which he has the brisk walk before the modified lunch, and one where he has a half-hour run after work 3 days a week. The user could then be shown a probable weight range and the probability of a heart attack before the age of 50 for each of the 6 scenarios (no change and each of the 5 simulations). In some embodiments, if the patient modifies his lifestyle based on one or more of the suggestions, he could be shown, after a period of time, such as a month, a probable reduction in the risk of a heart attack. The medical and lifestyle choice information can be collected from conventional sources (e.g., a medical questionnaire filled in by a patient, a medical questionnaire filled in by a medical profession, a medical professional's report, a test).
Medical information can also be collected automatically by analysis of subject-connected (non- conventional) sources. Such sources can include, but are not limited to, a subject's Facebook or other social media page, WhatsApp or other SMS message, a telephone conversation, a conversation (or other sound) heard via a microphone (e.g., in a cellphone in a patient's pocket), an electronic diary, a sensor carried by the patient (e.g., a heart rate sensor worn during exercise), geolocation sensors (such as GPS), and any other source wherein a subject's behavior and/or lifestyle choices are exposed.
Collection, of both conventional and subject-related data, can be by means of character recognition of typed text, character recognition of written text, contact (by a user or a medical person) with a touchscreen (e.g., touching an answer on a screen or a picture on a screen), speech recognition, image recognition, identification of characteristic sounds (panting, snoring), and any combination thereof.
The non-conventional sources can be also used to increase convenience and ease of filling out the questionnaire. For example, when, according to geolocation, a location of the user coincided with a restaurant, and a word "hamburger" was detected via speech recognition, the system can ask the question: "Did you eat a hamburger?" Oral or written confirmation is much more convenient than the usual process of filling in a questionnaire, and is expected to provide the user with additional motivation to keep an accurate record of activities.
Collected information is stored in a universal compact format, any subset of which can be readily added to, transferred, analyzed and visualized. In some embodiments, a controlled vocabulary is provided of (1) events (e.g. activities, actions, meals taken, etc.), (2) observations (e.g. red rash), (3) specific body coordinates (lower abdomen), (4) environmental events/description, etc. An information record could then consist of a chain indexes of vocabulary entries and time stamps: eventX - timel - observation Y -placeN- time2, etc. Each entry could be coded by a few bytes (4 bytes will allow a controlled vocabulary of 4 million words), allowing search and pattern comparison on encoded data. Reference is now made to Fig.1 presenting a flowchart of method 100 of monitoring and analyzing a lifestyle of a user and predicting health implications thereon. Method 100 starts with step 110. At the aforesaid step, the user inputs at least one datum relating to his/her life style. Described below, the life-style datum can be a medical examination datum, a training session datum, a food intake datum, a subjective life-style assessment datum and/or an environmental impact datum. The input data are stored within a data base (step 120) available to analytical algorithms. Analyzing the input data is performed at step 130. At final step 140, health implication stemming from the lifestyle defined by the input data are predicted. The predicted health implications are available to the user.
Reference is now made to Fig.2 presenting a schematic diagram of data processing 300 in a system for monitoring and analyzing a lifestyle of a user and predicting health implications. Device 260 functioning as a user's terminal is configured for inputting the abovementioned data relating to his/her life style. The input data are analyzed in iterative dichotomizer 3 (ID3) decision tree 320 and stored in database 230. Algorithm unit 310 is configured for selection of life-style features which are statistically relevant to influence on medical conditions of the user. The selected life- style features are examined in unit 320 by means of the Chi squared test. The affirmed life-style features are used for rubricating the input data when the aforesaid data is stored in an adaptive data collection tree in a Boolean form. The selected statistically significant life-style features can be suggested to the user (when displayed on the user's personal device) as optional activities which can be beneficial user's medical conditions.
Reference is now made to Fig. 3 presenting system 200 for monitoring and analyzing a lifestyle of a user and predicting health implications system. Numeral 260 refers to a plurality of user's devices 260 configured for inputting and outputting information. User's device 260 can be embodied on any digital device such as smartphone, a desktop computer, a laptop computer, a tablet computer or a game console. At server side 210 being in wire or wireless communication with the plurality of user's devices 260, program framework provides connectivity between user's devices 260 and application interface 240. The personal data of the users 220 are stored in database 230.
Reference is now made to Fig. 4A-E schematically illustrating an embodiment of a portion of a data collection tree where, for clarity, the tree has been split over Figs. 4A-E. The numbers in the circles connect the parts of the figure; for example, the circled “1” in Fig. 1A connects to the circled “1” in Fig. 4B, the circled “2” in Fig. 4B connects with the circled “2” in Fig. 4C, and so on. “Time line information” (1005) and “I did... ” (1045) are repeated across Fig.4A-E for clarity.
Data collection has a hierarchical structure. "Global" subjects such as "feeling" (Type 2, 1025), "actions" and "observations" are divided into "sub-global" subjects. For example, the "actions" (“I did..., 1045) are divided into "food/drink" (1050), "Sport/ movement" (Type 6, 1075), "Medical procedure/ measuring" (Type 7, 1085) etc. The sub-global subjects can also be sub- divided; further subdivisions, sub-subdivisions, etc. can be made as and where necessary.
Each leaf of the hierarchical tree - (1005-1120 in Figs. 4A-1E) - corresponds to a specific type of stored timeline information. Each type of storable timeline information can take one of a plurality of discrete values, the values chosen from a list associated with the type of storable timeline information. For example, Type 3, "Location in body" (Fig.4A, 1035) )can have values (Fig.4A, 1040) such as, but not limited to: "Head", "Neck", "Chest", "Right hand", "Left hand", "Right leg" and "Left leg".
Each type can have corresponding descriptive parameters, the nature of the descriptive parameter depending on the type. For example, Type 5, "Drink" (Fig. 4B, 1065), can be associated with the descriptor, the name of the drink, or a type, the type of drink (Fig. 4B, 1070) and a parameter, "volume" (not shown). In the embodiment shown, the type of drink (Fig. 4B, 1065) (e.g., alcohol, water, tea, coffee, soft drink, etc.) is associated with a subtype, the name of the drink. For non- limiting example, the type of drink “alcohol” can be associated with the subtypes “wine”, “whiskey”, “beef”, etc. In other embodiments, the type of drink (Fig. 4B, 1065) can be the descriptor “wine, beer, whiskey, tea, coffee, soda, etc. Type 6, "Sport/movement" (Fig.4C, 1075) is associated with a parameter, "duration" (not shown) of the sport/movement. The number of descriptive parameters will usually be equal for different indexes of the same type, but exceptions (some indexes requiring fewer or more descriptive parameters) are possible.
A type can be “simple”, where all of its descriptive parameters are descriptors, or "complicated", where at least one of its descriptive parameters is a type (subtype). For example, the type "Feeling" (Type 2, Fig. 4A, 1025) has as descriptive parameter the type "Location in body" (Type 3, Fig. 4A, 1035). Such organization of the collected information simplify the input, storage, retrieval, visualization and analysis of the collected information. Medical information, such as test results, diagnosis, and so on, will be saved in a similar way.
Fig. 4A schematically illustrates a part of an illustrative embodiment of a data collection tree (1000). The root of the tree is the “Time line information” (1005) to be collected. Type 1 information is when did the item start (1010), the options being (1015) “now” or entering/selecting a date and time. The next category of information is what the user feels (1020), with Type 2, “feeling” (1025) shown. Feelings include the type of sensation (1030), such as, but not limited to, pain, itch, dizziness, nausea, and others, and (Type 3) the location of the sensation in the body (1035), which can be (1040), but is not limited to, locations such as the head, neck, chest, right hand, left hand, right leg, left leg, and others, which can be added if not supplied automatically.
Fig. 4B-D schematically illustrate a part of an illustrative embodiment of a data collection tree (1000). The root of the tree is the “Time line information” (1005) to be collected. In this part of the data collection tree (1000), the user's actions (1045, “I did... ”) are collected.
In the illustrative example of Fig.4B, information is collected on food and drink (1050) consumed. Selection can be made of the type of food (1055, Type 4), such as, but not limited to (1060) a starch (potato, bread, pasta, etc.), a meat (chicken, turkey, beef, etc.), a vegetable (tomato, lettuce, broccoli, etc.), fish (not shown, desserts (not shown), candy (not shown, and others. Similarly, selection can be made of the type of drink (1065, Type 5), for no-limiting example (1070), milk, coffee, tea, soda (coke, Sprite, root beer, etc.), alcohol (wine, beer, vodka, whiskey, etc.), water (not shown) and others. In addition, a selection of ready menus (not shown) containing several types of food and drink can also be provided.
In the illustrative example of Fig. 4C, information is collected on sports and movements (1075, Type 6) and on medical procedures and measurements (1085, Type 7). Sports and movements (1075) can include (1080), but are not limited to, running, swimming, Pilates and others. Medical procedures and measurements (1085) can include (1090), but are not limited to, getting an enema, taking medication, with a sub-screen of the name of the medication, applying an ointment, with a sub-screen of the name of the ointment, being weighed, having a temperature taken, and others.
In the illustrative example of Fig. 4D, information is collected on other types of activity (1095, Type 8) such as (1100), but not limited to, smoking, sex, and others. Fig. 4E schematically illustrates a part of an illustrative embodiment of a data collection tree (1000). The root of the tree is the “Time line information” (1005) to be collected. Type 9 information is observations of physical data (1105) such as (1110), but not limited to, is diarrhea present, normal/abnormal mucus production, normal/abnormal sputum, normal/abnormal seminal emission, and others, with the name and description, if needed, to be added. Other information (1115) can also be collected; if not otherwise included in the data collection tree, such as the name of the type of data and, where needed, the variety of the item. For example, if the type of data is “good/bad deed”, the variety could include, but not be limited to, gave charity, visited a sick person, volunteered in a soup kitchen, speeding while driving, swearing, and so on.
Fig. 5 illustrates exemplary information input according to the hierarchical structure of the information. Each node of the hierarchical tree in Figs 4-5 is assigned a corresponding index and weight. The index points to the node content, including the node's parent, children, grandchildren, etc. The weight reflects the user's activity associated with input via the corresponding node. The tree is adaptive because the hierarchical structure of the tree can be changed, moving the nodes with the highest weights towards the top. This provides a user with the quickest access to the user's most common activities. This procedure is similar to the tree modification in Adaptive Huffman coding [Donald E. Knuth, "Dynamic Huffman Coding", Journal of Algorithm, 6(2), 1985, pp 163-180.].
The hierarchical organization described above is associated only with the type of the input data (the leaf), whereas specific events are saved as the usual array. For non-limiting example, drinking 100 ml cola at 18:30 on 22 February 2019 will be added to the Timeline Array as (5, 4.b, 100, 18, 30, 22, 2, 19), where "5" points to the type "drink", "4.b" points to "cola", and so on. Obviously, the order in which the data are stored in not limiting, as long as the same order is used throughout an array.
Fig. 5 shows an embodiment of relationships between screens shown to a user (2000), where the screens are related to the leaves in the hierarchical tree of the embodiment of Fig. 1. The left-most screen (2005) corresponds to the top level “Time Line Information” (1005) of Fig. 1. On this top- level screen are the suggestions of data to be entered, including “I feel” (2022) and “I did ...” (2047). If the user chooses “I feel... ” (2022), a screen for types of feeling (2020) will be shown (dark arrow). From here, sub-screens (“Select feeling”, When it started”, etc.) can be selected. If the user chooses “I did...” (2047), a screen for types of actions (2045) will be shown (medium grey arrow). If the user further selects “food related” (2067), a food related screen (2065) will be displayed (light grey arrow), from which a user can select the type of food/drink, reactions to the food taken, and so on. A similar hierarchy can be available for many other activities. As time progresses, common activities, such as entering common foods, will move up the hierarchy, as discussed below. In time, a common food or, in some embodiments, a common meal, might occur on a top-level (leftmost in Fig. 5) or second-level screen; thus avoiding, on entering a food eaten, clicking on “I did”, then “food related”, then “Select food/drink”, and then the food(s).
Figs. 6-13 illustrate an exemplary embodiment of screens presented to a user. The screens are exemplary and non-limiting; in some embodiments, there can be: a different arrangement of data on a screen, a different order of presentation of screens, more screens, fewer screens, a more vertical hierarchy with more sub-screen, a more horizontal hierarchy with fewer sub-screens, a different categorization of data, more information collection, less information collected and any combination thereof. In Figs 6-13, preferably, the screens are leaves in a data collection tree configured to collect at least a portion of both conventional and subject-related data. In some embodiments, the data collection tree is adaptive. In some embodiments of an adaptive data collection tree, the questions or requests for information provided by the screens of the data collection tree can be variable. In some embodiments of an adaptive data collection tree, the types of screen shown can be variable. In some embodiments of an adaptive data collection tree, the screen on which a question or other request for information is placed can also be variable.
In Figs. 6-13, the user can be the subject whose data are being collected, a medical professional, and any combination thereof. In some cases, data will probably be entered by a medical professional; for non-limiting example, tests and examinations performed and the results of the tests and examinations. In some cases, data will probably be entered by the subject, for non- limiting example, foods eaten and exercise taken. Each of the screens in any of Figs. 6-13 can lead to one or more sub-screens, based on stored information, entered information, the structure of the data collection tree and any combination thereof.
Fig. 6 schematically illustrates a top-level screen (3000) of an exemplary embodiment.
In the exemplary embodiment of Fig. 6, the category “Account” (3010) comprises information of little or no medical utility, configured to identify the patient and to provide financial and other details of the patient's interactions with the system. If “Account” (3100) is selected, a screen such as the exemplary screen of Fig. 7 (below) will be displayed.
The category “Medical Card” (3200) comprises type of information normally captured by a physician. If “Medical Card” (3200) is selected, a screen such as the exemplary screen of Fig. 8 (below) will be displayed.
The category “Tell what's happened” (3300) comprises information describing an immediate health condition. If “Tell what's happened” (3300) is selected, a screen such as the exemplary screen of Fig. 10 (below) will be displayed.
The category “My network” (3400) comprises information enabling the system to connect to a social network for exchange of subject-related behavioral and lifestyle information. If “My network” (3400) is selected, a screen such as the exemplary screen of Fig. 12 (below) will be displayed.
The category “More activities” (3500) comprises information on other activities, in order to provide a user with additional motivation to keep an accurate record. If “More activities” (3500) is selected, a screen such as the exemplary screen of Fig. 13 (below) will be displayed.
Additionally, preferably, at least one help screen (3600), to assist the user in understanding the information on the screen, and at least one Contact us screen (3700), to enable the user to contact useful persons/institutions (a doctor, a hospital, etc.) are provided. These are conventional Help and Contact us screens that will not be further described. The Help and contact us screens do not limit the system of the present invention.
As schematically illustrated in Fig. 7, the information of little or no medical utility (3100) comprises “Account info” (3110) comprising, but not limited to, the patient's name, social security number, ID number, etc.; “Settings” (3120), comprising a, preferred setup of the screens; “Bills” (3130) comprising, but not limited to, invoices and payments; and “Activity history” (3140) comprising, but not limited to, information about interactions with the system. Interactions with the system can comprises user interactions, interactions by medical personnel and any combination thereof. Fig. 8 schematically illustrates an exemplary medical card (3200), configured to capture the type of information normally provided by a physician, although a subject can provide the information. Via this screen, new data can be entered (3210), existing data can be displayed, analyzed or both (3220), at least a fraction of the data can be shared (3230), a note can be entered (3240). Other information of this type (3250) can also be collected via sub-screens entered from the medical card screen.
Figs. 9A and 9B schematically illustrate exemplary sub-screens (3210, 3220) of the medical card (3200), with Fig. 9A schematically illustrating the types of data that can be input (3210) and Fig. 9B schematically illustrating a top-level view of a screen to set up showing and/or analysis of data (3220).
In the exemplary embodiment of the Input new data screen (3210) of Fig. 9A, the types of data that can be entered comprise Anthropometric data (3211), Medical test data or medical inspection data (3212), a report on a Doctor visit (3213) (which can be a visit to the doctor by the subject or a visit by the doctor to the subject), data describing the subject's past (3214), data describing the subject's environment (3215), data uploaded from a device (3216), typically a device comprising one or more sensors, a document (3217) to be stored in the system, and an image (3217) to be stored in the system. Other types of data (3218) can also be collected via sub-screens entered from the Input new data screen. The device can be any conventional device providing data about the functioning of the subject's body. Non-limiting examples of a device are a pulse oximeter, a glucose meter, an ECG, a heartrate meter, a thermometer and any combination thereof.
In the exemplary embodiment of the Show/Analyze screen (3220) of Fig. 9B, a selection can be made of the category(s) of data to be displayed (3221), the category(s) of data to be analyzed (3221), a time period over which the data are displayed can be selected, a time period over which the data are analyzed can be selected (3222), whether the results are to be displayed as a table (3223) or a graph (3224), whether a correlation analysis (3225) is to be carried out, and whether a statistical analysis (3226), is to be carried out.
In preferred embodiments, tables, graphs, and the results of analyses can be stored if desired.
Fig. 10 schematically illustrates an exemplary screen for collecting data concerning an immediate health condition. (“Tell what's happened” 3300). The subject is invited to describe how he or she feels (3310), what he or she has done (3320), what he or she can see (3320) (for non-limiting example, results of a visual inspection of the subject's body, and observation of unusual or unexpected changes in the functioning of the eye), what is in the subject's environment (3340), and other data concerning the immediate health condition (3350) as appropriate.
Figs. 11A and 11B schematically illustrate exemplary sub-screens of the health condition card (3300).
Fig. 11A schematically illustrates descriptors of the feeling (3310), such as location on the body (3311), type of sensation(s) (3312), possible reason(s) for sensation(s) (3313), time and location of start (or intensification) of sensation(s) (3314), time and location of end (or amelioration) of sensation(s) (3315), possible reason why the sensation(s) stopped (3316), general feeling of bodily well-being or otherwise (3315), and other information as noted down by the doctor or patient (3318), and any other descriptors as appropriate (3319)..
Fig. 11B schematically illustrates how something might have happened (3320) - was it food- related (3321), sport or movement related (3322), from a medical action (3323) or procedure (3324), an action (good (3325), e.g., “I was doing my physiotherapy”; bad (3326)e.g., “I really shouldn't have bent over so fast”; other ((3327) e.g., “I was just stepping off the curb”), and other information as noted down by the doctor or patient (3328). Each of these screens can lead to one or more sub- and other data concerning how something might have happened (3350) as appropriate.
Other descriptors and other sub-screens (in my environment, etc.) can be provided.
Fig. 12 schematically illustrates an exemplary screen for connecting the system to one or more social networks (My Network, 3400) for exchange of subject-related behavioral and lifestyle information. The system can look on a user's timeline (3410), read Friends News (3420), Make a Post (3430), etc. The exchange of the information can be via a network account (3440), a network activity (3450), a computer (cell, laptop, desktop, cloud, etc.) memory (3460), Facebook or other social media page, WhatsApp or other SMS message, a telephone conversation, a conversation (or other sound) heard via a microphone (e.g., a microphone in a cellphone in a subject's pocket), an electronic diary, a credit report, a schedule, and any combination thereof. Other social network data can be provided (3470) as appropriate. Network data can be entered manually or, preferably, automatically. If the network data are entered automatically, the user enters, via sub-screens of the My Network (3400) screen, identifiers) of the social network(s) from which data are to be collected, the identifiers) allowing the system access to the social network(s).
The connection of the system to a social network provides the users with additional motivation to keep an accurate record of activities, can provide the possibility of direct sharing of the accumulated information with specialists, forums or friends, can be used to find similar cases and to obtain advice from different sources.
Fig. 13 schematically illustrates an exemplary screen of other activities (3500) for providing a user with additional motivation to keep an accurate record of activities, improving diagnosis, improving prognosis, improving the patient's current state of health and any combination thereof. These can include, but are not limited to, finding help (3510) (for non-limiting example, a person or organization to assist in behavior modification, a specialist professional for a medical condition, and an organization or location to enable appropriate exercise); finding a location or organization to volunteer with or donate to (3520); finding a location or organization to provide a volunteer to assist a subject (3530), writing a family medical history (3540); writing an account of family members and/or pets (3550), which can include numbers, relationships, and types and quality of interactions with the family member and/or pet; building a vector of life (3560); providing a location of a relevant health shop (3570); other relevant information (3580), and any combination thereof.
In some embodiments, a subject can be encouraged to join an appropriate social network (such as, but not limited to, a hiking club, Facebook diet page, etc.)
In some embodiments, a device equipped to determine location, such as, but not limited to a smartphone with GPS or equipped to determine Wi-Fi strength, can detect a subject's location and can suggest to the subject to input an event or action related to the place, time, subject history, etc. For non-limiting example, a subject at a restaurant in the evening can be prompted to enter his meal and drink choices, and how much of the meal and/or drink was consumed.
In some embodiments, the automatic location determining can also be applied for the prediction, prevention, diagnosis and treatment of a medical condition by taking into account of the environmental condition or medical statistics associated with a given location. Preferably, the information will be stored in Boolean form. Preferably, the data can be stored as an adaptive data collection tree. Preferably, free text input is aUowed; the free text input can be used to improve the structure of the data collection tree. How often each tree node is actually used will be recorded, and frequently used nodes will be moved up, toward the top (first) screen. EventuaUy, the user will be able to make most routine notes from the top screen and screens immediately below the top one.
Figs. 14A-B schematically illustrates changing the order of items so that more-used items are closer to the top. Fig. 14A schematically illustrates an illustrative set of weights (4000) for foods. Type “Starch” (4100) is eaten most often, with a weight of 49, Type “Vegetable” (4300), next most often, weight 46, and Type “Meat” (4200) least often, weight 35. Within Type “Starch” bread (4110) is eaten most (weight 20), then potato (4210, weight 15); pasta (4320) is eaten least often (weight 14). Within Type “Meat” (4200), turkey ((4220), weight 16) is eaten more often than chicken ((4210), weight 14); both are eaten more often than beef (4230 weight 5). Within Type “Vegetable” broccoli 4330 is eaten most (weight 20), then tomato (4310 weight 16); lettuce 4320 is eaten least often (weight 10). Other types of starch (4140), meat (4240) and vegetable (4340) and other types of foodstuff (4400) are eaten, but are not displayed for simplicity and clarity.
Fig. 14B schematically illustrates rearrangement of the screen. The original screen (4000) is on the left, with the rearranged screen (4000A) on the right Type “Vegetable” (4300) (4200) moves up (dashed line) from third place to second place, with Type “Meat' moving down (solid line) to third place. Chicken (4210) and turkey (4220) swap places (dark arrows), with turkey now first (2b → 3a) and chicken second (2a → 3b). Similarly, broccoli (4330) moves up to first place (long light arrow, 3c → 2a), and tomato (4310) (3a → 2b) and lettuce (4320) (3b → 2c) both move down one place (short light arrows). Similarly, rearrangements of screens can take place, as weU as moving items from one screen to another, based on their weights.
The system can comprise at least one algorithm for data analysis. Non-limiting examples of an algorithm for data analysis include: principal component analysis (PCA), factor analysis, correlation analysis, regression, clustering (such as, but not limited to, k-means, PAM, EM and spectral clustering), a decision tree, an adaptive decision tree, Fisher Unear discriminant, K-NN, a neural network, a convolutional neural networks, SVM, Naive Bayes, Adaboost, supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, association rules and deep learning.
One or more of these algorithms can be applied to data collected from all of the users, or from at least one of them for achieving the multiple goals. All of the applications take into account individual constraints, such as subject specific biological properties (i.e. family or genetic associated factors), lifestyle (including the lifestyle history), environment (including the environmental history), and medical history.
Other lifestyle factors include, but are not limited to, eating habits, work/life balance, sleeping habits, type of entertainment, and any combination thereof.
Other environment factors include, but are not limited to, type of dwelling, type of workplace, location of dwelling, air quality, water quality, food quality and any combination thereof.
The major goals are:
• Early detection of diseases.
• Prediction of disease development in the future, including even minor health deterioration and/or gradual health deterioration, (providing, for example, expected time and probability).
. Detection of the main factors causing a disease or a health deterioration.
. Proposing at least one optimal recommendation for minimizing the probability of the appearance of a disease or a health deterioration.
Early detection of disease can include providing a warning (“flagging up”) of the disease. This can be especially important for dangerous diseases such as, but not limited to, heart attack, bowel cancer, and lung cancer.
The approach of a heart attack often manifests in a sharp arm pain that has no apparent cause. The system of the present invention can raise a warning flag, ask the subject additional questions about his condition, and advise him to take additional care and/or to go to medical provider immediately. If it is a mobile application, in some embodiments, it can include a button to notify friends/relatives/medical providers of a need for action.
Bowel cancer, on other hand, develops over several decades, but early symptoms can be almost inconspicuous or can masquerade as another problem of the digestive system (ulcer, etc.). To raise a warning flag, a most detailed record must be collected of meals eaten, times of eating and reactions to the food. Currently, there is no system capable of collecting this information in sufficient detail, and a doctor could collect such information only by placing the patient in a hospital for a sufficiently long period. In practice, because of the costs of hospitalization, the observation time would almost certainly be too short.
Reference is now made to Fig. 15 presenting a schematic flowchart of method 800 of predicting a change in weight of a user on the basis of life-style records such as diet and sleep patterns and physical exercise activities. Method 800 includes imaging a meal before the user eats it (step 810). As described below in detail, the system of the present invention is configured for identify the imaged meal according its components and line off the identified components (step 820). On the basis of the obtained data, a geometric model of the imaged meal is built and then amounts of the meal components and their caloric values are determined (step 830). In parallel, the user inputs data relating to his/her life-style such as dietary and sleep patterns and physical exercise activities (step 840). It should be noted that each individual is characterized by his/her metabolic rate such that that the identical meal has different influence on medical and life-style records of different individuals. The metabolic rate can depend on a body weight, a body fat content, muscle mass, digestion efficiency and other life circumstances (quality of sleep and other parameters identified by the system) of the individual. Thus, collection of medical and life-style records during a predetermined time period (for example, one month) allows to establish a database statistically sufficient for estimating individual metabolic rate and predicting effect of nutrition on medical and life-style records. All obtained data are collected and stored in a database 230 being a part of the server side of the system 210 (see Fig. 3). The abovementioned data can be used for training a machine-learning algorithm for predicting a change in weight of a user (step 850).
It should be emphasized that the method 800 of the present invention allows to avoid mistakes in determining an effect of dieting or physical exercising.. The machine-learning algorithm after collecting a substantial database predicts an effect of dieting, exercising and other aspects of life style on the basis the individual response of the user to his/her life style.
It should be mentioned that the present invention provides an opportunity of setting any goal relating to the life-style of the user and getting a set of recommendations of changes in the user's life style leading to achieving the set goal. The advantage of the approach based on the present invention is in real-time adjustinent of the recommendations provided to the user because all data reflecting changes in the medical conditions, his/her activities are input into the system in real time either. Dynamic monitoring the medical and other indicators of the user's condition can be also used for early identifying a potential development of any pathological condition and providing the user with a specific recommendation of performing an in-depth diagnostic test. The present invention defines a so-called LifeVector parameter integrally characterizing a health status and reflecting weak signals (medical records or subjective sensations potentially leading to any pathology.
Reference is now made to Fig. 16 presenting a method estimating a caloric value of a meal to be eaten. At step 510, a calibration of the user's device is performed (described in detail further). Step 520 relates to taking a picture of the meal at predetermined conditions defined by calibration at step 510. At next step 520, meal recognition is performed. As the result, a number optional variants are suggested (step 530). The user confirms the relevant variant (step 540) and select the relevant type of a plate (step 550). In parallel, the dimensions of the meal are identified (step 560). Then, texture of the meal is detected (step 560). The obtained data are stored in the database (step 580). On the basis of textural and dimensional data, content of nutrients in the serve is calculated (step 600). Optionally, USDA food composition database is used (step 610). Finally, a caloric value is obtained (step 620).
Reference is now made to Fig. 17 showing a setup for calibrating the user's device 260 which is positioned at a predetermined distance from reference object 270. Reproducible conditions of imaging the meal is provided by a standard dimension of the referent object in the coarse of imaging each meal.
Reference is now made to Fig. 18 showing a schematic flowchart of texture classification.
Texture classification and obtaining color features are performed in order to divide dish 720 in plate 710 to its components (the main component (a steak), dressing and other components and define boundaries between them. At the end of this process, grid 700 of small pixel portions 730 belonging to specific food is obtained as shown in Fig. 17.
Texture Classification is a fundamental issue in computer vision and image processing, playing a significant role in many applications such as medical image analysis, remote sensing, object recognition, document analysis, environment modeling, content-based image retrieval and many more areas. Color features are the most intuitive and most obvious image features, they are commonly used in image retrieval and are very robust, meaning they are not sensitive to rotation, translation and scale, also it is fairly easy to calculate color features.
There are a lot of different features that can be extracted from images such as features pretending to color, texture, and shape, texture classification has been around for some time now and there are a lot of different way to extract features from pictures. A lot of researches have been made in these subjects however at the time of the writing of this paper we could not find any research as to the best way to compare small portions of pixels from a picture to other pictures or food items. Thus, at the second part of this project we will test to find the best way to classify the texture of these pictures in order to find their best match from the database. The methods that will be tested are a single case or a combination of a few of the following methods:
Texture analysis using gray level co-occurrence matrix features:
In the grey level method, the elements are the relative frequencies of occurrence of grey level combinations among pairs of image pixels. The GLCM can consider the relationship of image pixels in different directions such as horizontal, vertical, diagonal and antidiagonal. The co- occurrence matrix includes second order grey level information, which is mostly related to human perception and the discrimination of textures. Four statistical features of the GLCMs are computed. The features are energy, entropy, contrast, and homogeneity. G x G GLCM Pd for a displacement vector d = (dx, dy) is defined as follows. The (i, j) of Pd is the number of occurrences of the pair of gray-level i and j which are a distanced apart. A number of texture features are listed as follows:
Figure imgf000025_0001
Discrete wavelet transform (DWT) is used to modify images from the spatial domain to the domain of frequency. Small wavelet transformation represents the coverage of the home function, known as the main wavelength functions. In the small wavelet transform, by passing the low pass filter and a high pass filter to retrieve information from different signal levels. The wavelet offers a good energy compression and multi-resolution capability. The wavelet is stable offset color intensity, you can get the texture and shape of the information efficiently. Small wavelet transform can be linearly calculated on time, it can be very efficient algorithm. The signal to a set of base functions and wavelet length functions is got by decomposition done by DWT. Two-dimensional wavelet form transformation is a multi-resolution method using recursive filtering and also sub sampling. In discrete wavelet transforms represent the levels and decompose it into different levels, it is decomposed image on four frequencies sub-band LL(low- low), LH(low-high), HL(high-low) and HH(high-high) (where, L is a low frequency, H is a high frequency) as shown in Fig. 19.
Additional uses for the system can include:
1. Providing individualized risk assessments based on lifestyle choices.
Lifestyle choices can have an enormous influence on human health. It is estimated that just 5 lifestyle factors (physical inactivity, smoking, obesity, wrong dietary choices, and uncontrolled blood pressure) are responsible for over 50% of all disease conditions in the human population. At the same time people's awareness of the risk factors is low (“Poor Awareness of Risk Factors for Cancer in Irish Adults: Results of a Large Survey and Review of the Literature”, The Oncologist, (2015), 20, 4:372-378). Furthermore, in general, the subject specific data (for particular genders, ages, geographical locations, etc.) usually do not exist.
2. Discovering novel correlations between lifestyle and disease/deficiency conditions.
With the growth of enrollment in the system and the accumulation of statistical evidence, correlations can be found between lifestyle choices and disease occurrence that were not known before and known correlations can be verified. In addition, it can be found how specific subject characteristics (e.g., ethnicity, age, occupation) affect the correlations.
Correlations can also be found for non-disease conditions that can affect health such as, but not limited to, sleep quality, reduction of physical mobility, alertness, and mood. 3. Providing an interactive approach for lifestyle modifications with the aim of lowering the probability of major diseases.
It is known that lifestyle changes can significantly lower the probability of certain diseases (e.g. “Modifiable lifestyle factors that could reduce the incidence of colorectal cancer in New Zealand”, The New Zealand Medical Journal (2016), 129, 1447:16), but in practice people may find it difficult to make the necessary adjustments. In some embodiments, the system of the present invention can assist in discovering which modification(s) would be most effective for the individual user, for non-limiting example, tailoring the suggested change(s) to the user's habits, life situation, specific place of residence (for non-limiting example, regular outdoor walks are easy to advocate in California, but could be very difficult during the summer in Texas or in the winter in Minnesota), the user's age, physical disabilities, etc.
The advantages of the system of the present invention include: (a) as information is collected from a particular subject, certain specific choices can be identified that seem to working for him; (b) inclusion of data from many people of the same age, gender, occupation and geographical location as the user can be used to improve the quality of the suggestions, the range of suggestions, and any combination thereof.
4. Supplementing disease treatment with lifestyle changes.
It is also strongly suspected that “it is never too late”. It has been observed, for example, that people who have been operated on for lung cancer and who have been able to start physical exercise after the operation had significantly better outcomes that those who were unable or unwilling to start exercise. It is probable that this may also be applicable to other conditions. In some embodiments, the system of the present invention can allow systematic exploration of possibilities for supplementing medical interventions with lifestyle modifications.
This can be done in three steps. First, the relevant statistics are collected in order to determine what is possible. Second, recommendations for lifestyle modification can be formulated. Third, assistance can be provided to the patient to implement the modifications into his life and to find implementation strategies (See point 4, above). Currently, even the first stage, obtaining relevant reliable statistics, is expensive and complex even for most common diseases. And without the knowledge it is not possible to formulate desirable modifications. 5. Treatment of physical condition deficiencies that do not fall under disease definition.
As mentioned in point 4 above, there are many physical condition deficiencies where lifestyle changes are more appropriate than medical intervention. These include various movement restrictions, mild insomnia, headaches (that do not raise to migraine level), etc. In some embodiments, the system of the present invention can allow systematic exploration of possibilities for identifying lifestyle modifications that can reduce or ameliorate such physical condition deficiencies. Determining appropriate lifestyle modifications can be done in three steps. First, the relevant statistics are collected in order to determine what is possible. Second, recommendations for lifestyle modification can be formulated. Third, assistance can be provided to the patient to implement the modifications into his life and to find implementation strategies. In addition, such physical condition deficiencies often are felt only during specific time intervals. This can make it difficult to report for the deficiency to a doctor and can make it difficult for the doctor to detect a potential cause of a deficiency. In the system of the present invention, however, continuous, "real time" collection of data makes it possible to report even minor changes of physical condition, which will make it possible to predict and treat of these changes, as suggested above.
6. Contributing to standard treatments for common medical conditions.
Health insurance programs in several countries have stated a goal to find “treatment gold standards” for many common and widespread disease conditions (such as diabetes, for example). Lifestyle modifications must be included in the treatment standards for the vast majority of these common illnesses.
7. Fine differentiation of medical conditions and finding the best treatment options for them.
In preferred embodiments, the system of the present invention can record a medical history for each user in unprecedented detail and can record medical histories in detail for a large number of subjects (possibly millions or more). It is therefore likely that this will provide a better differentiation between medical conditions and can improve determination of the most suitable treatments. For example, deficiencies in the musculoskeletal system are highly variable, can be very painful, can severely degrade the quality of life and, very often, either no decisive medical treatment is available or it is virtually impossible to determine an effective treatment. Fine differentiation can help in determining the underlying cause(s) of the deficiency and can assist in determining the most effective treatment. The underlying variability of the conditions and of the possible response combinations can come from the known and observable factors (age, gender, occupation, geographical location) or can come from other factors, some unknown and some not commonly measured (e.g. certain combinations of genetic factors and/or environmental factors that are not recognized). In some embodiments, the three-step technique disclosed above can be used to discover and cluster together similar cases even if underlying factors are not known or are not determinable.
8. Assistance with the diagnostics, portability of medical data, etc.
In some embodiments, medical care providers can use the lifestyle information and the records of observations for diagnostic purposes. In some variants of these embodiments, medical test data can be automatically uploaded/added to an individual patient's record, so that it can be taken to another provider, used for remote medical consultations, etc.
9. Assistance with the development of the beneficial life regiments, etc.
In some embodiments, comprehensive lifestyle information collected with our system could be presented in the compact form of the LifeStyleRecord data structures. The LifeStyleRecords is a life regiment that may include any direction on dieting, exercising, taking specific measurements (blood pressure, weight, etc), taking specific medications or medical procedures and/or excluding specific types of food, activities, environmental influences, etc. The LifeStyleRecord is a slice of our collected data that we believe may help to achieve a certain desirable goal, like slim down, improve level of fitness, increase levels of vigor and happiness, etc. Our System will: (a) Do algorithmic search for LifeStyleRecords delivering various specific desirable outcomes; (b). Store them for our customers; (c) Facilitate exchange of them between different users; (d) Validate/verify them with the doctors/specialists; (e) Check suitability of the specific exchanges (some LifeStyleRecords can be in contradiction with the customer age, physical conditions, dietary restrictions and other possible medical or cultural restrictions); (f) Modify LifeStyleRecords for circumstances of the specific customer. For example, individual allergies, diseases, age of the person; individual food preferences; food availability in the shops in his/her geographic location; his/her budget; matching his/her time allocated for cooking and cooking skill level; his/her other life goals, immediate and strategic (e.g., if the person in the active muscle build up program right now) Many variations of the present invention are possible once the present invention is known to those skilled in the arts and are within the spirit and scope of the present invention. Those skilled in the arts will be able to make many variations on the present invention once this invention is known to the arts.

Claims

Claims:
1. A computer-implemented method of monitoring and analyzing a lifestyle of a user and predicting health implications thereon; said method comprising steps of: a. inputting at least one datum relating to said lifestyle; said at least one datum selected from the group consisting of: a medical examination datum, a training session datum, a food intake datum, a subjective life-style assessment datum, an environmental impact datum and any combination thereof; b. storing input data; c. analyzing said input data; and d. predicting said health implications of said lifestyle defined by said input data.
2. The method ofclaim 1, wherein said step of storing said input data comprises storing said input data in an adaptive data collection tree in a Boolean form.
3. The method of claim 1 comprising a step of selecting user's features of interest.
4.The method of claim 1 , wherein said step of storing said input data comprises storing selected user's features of interest in said adaptive data collection tree.
5. The method of claim 1, wherein said step of predicting said health implications comprises performing Chi squared test.
6. The method of claim 1, wherein said medical examination datum is selected from the group consisting of height, weight, body mass index, eye color, hair color, skin color, blood test result, an imaging test result and any combination thereof.
7.The method of claim 1, wherein said subjective life-style assessment datum is selected from the group consisting of a location of a body sensation, intensity said body sensation, a start time of said body sensation, a start location of said body sensation; variability of said body sensation, an end time of said body sensation, an end location of said body sensation, a body location of a condition, intensity of said condition, a start time of said condition, a start location of said condition; variability of said condition, an end time of said condition, an end location of said condition, a medical action performed in relation to said sensation, a medical action performed in relation to said condition, a medical procedure performed in relation to said sensation, a medical procedure performed in relation to said condition, a sleep- and-wake pattern and any combination thereof.
8. The method of claim 1, wherein said environmental impact datum is selected from the group consisting of a timeline datum, friend's news, a social media post, a family history datum, a number of pets, a type of said pet and any combination thereof
9.The method of claim 1, wherein said step of analyzing said input data comprises an algorithm selected from the group consisting of a principal component analysis (PCA) algorithm, a factor analysis algorithm, a correlation analysis algorithm, a regression algorithm, a clustering algorithm, a decision tree algorithm, an adaptive decision tree algorithm, a Fisher linear discriminant algorithm, a k-nearest neighbors algorithm, a neural network, a convolutional neural networks, a support vector machine algorithm, a Naive Bayes classifiers algorithm, an adaptive boosting algorithm, a supervised learning algorithm, an unsupervised learning algorithm, a semi-supervised learning algorithm, a reinforcement learning algorithm , an association rule algorithm, a deep learning algorithm and any combination thereof.
10. The method of claim 9, wherein said clustering algorithm is selected from the group consisting of k-means, partitioning around medoids, empirical modeling, spectral clustering and any combination thereof.
11. The method of claim 1, wherein said step of predicting said health implications comprises determining a probable upcoming health condition.
12. The method of claim 11, wherein said probable upcoming health condition is selected from the group consisting of a heart attack, bowel cancer, lung cancer and any combination thereof.
13. The method of claim 11, wherein said step of predicting said health implications comprises a step of identifying a correlation between at least one lifestyle change and a health condition.
14. The method of claim 13, wherein said lifestyle change is selected from the group consisting of a change in exercise activity, smoking, food consumption, sleeping pattern and any combination thereof.
15. The method ofclaim 1, wherein said step of inputting at least one datum relating to said lifestyle comprising imaging a food to be eaten a predetermined distance and estimating a caloric value thereof.
16. A computer-implemented system for monitoring and analyzing a lifestyle of a user and predicting health implications thereon; said system comprising: a. a server configured for processing input data; b. at least one user's device connected to said server; said at least one user's device configured for inputting data by said user and displaying at least one datum relating to said lifestyle processed by said server; said at least one datum selected from the group consisting of: a medical examination datum, a training session datum, a food intake datum, a subjective life-style assessment datum, an environmental impact datum and any combination thereof c. a memory configured for storing input data said memory storing instructions to said server which, when executed by said server, direct said server to i. receiving said at least one datum relating to said lifestyle; ii. storing input data; iii. analyzing said input data; and iv. predicting said health implications of said lifestyle defined by said input data.
17. The system of claim 16, wherein said input data are stored in an adaptive data collection tree in a Boolean form.
18. The system of claim 16, wherein said instructions comprise selecting user's features of interest.
19. The system of claim 16, wherein said step of storing said input data comprises storing selected user's features of interest in said adaptive data collection tree.
20. The system of claim 18, wherein said instruction of predicting said health implications comprises performing Chi squared test.
21. The system of claim 16, wherein said medical examination datum is selected from the group consisting of height, weight, body mass index, eye color, hair color, skin color, blood test result, an imaging test result and any combination thereof.
22. The system of claim 16, wherein said subjective life-style assessment datum is selected from the group consisting of a location of a body sensation, intensity said body sensation, a start time of said body sensation, a start location of said body sensation; variability of said body sensation, an end time of said body sensation, an end location of said body sensation, a body location of a condition, intensity of said condition, a start time of said condition, a start location of said condition; variability of said condition, an end time of said condition, an end location of said condition, a medical action performed in relation to said sensation, a medical action performed in relation to said condition, a medical procedure performed in relation to said sensation, a medical procedure performed in relation to said condition, a sleep-and-wake pattern and any combination thereof.
23. The system of claim 16, wherein said environmental impact datum is selected from the group consisting of a timeline datum, friend's news, a social media post, a family history datum, a number of pets, a type of said pet and any combination thereof
24. The system of claim 16, wherein said instruction of analyzing said input data comprises an algorithm selected from the group consisting of a principal component analysis (PCA) algorithm, a factor analysis algorithm, a correlation analysis algorithm, a regression algorithm, a clustering algorithm, a decision tree algorithm, an adaptive decision tree algorithm, a Fisher linear discriminant algorithm, a k-nearest neighbors algorithm, a neural network, a convolutional neural networks, a support vector machine algorithm, a Naive Bayes classifiers algorithm, an adaptive boosting algorithm, a supervised learning algorithm, an unsupervised learning algorithm, a semi-supervised learning algorithm, a reinforcement learning algorithm , an association rule algorithm, a deep learning algorithm and any combination thereof.
25. The system of claim 24, wherein said clustering algorithm is selected from the group consisting of k-means, partitioning around medoids, empirical modeling, spectral clustering and any combination thereof.
26. The system of claim 16, wherein said step of predicting said health implications comprises determining a probable upcoming health condition.
27. The system of claim 26, wherein said probable upcoming health condition is selected from the group consisting of a heart attack, bowel cancer, lung cancer and any combination thereof.
28. The system of claim 26, wherein said step of predicting said health implications comprises a step of identifying a correlation between at least one lifestyle change and a health condition.
29. The system of claim 28, wherein said lifestyle change is selected from the group consisting of a change in exercise activity, smoking, food consumption, sleeping pattern and any combination thereof.
30. The system of claim 16 comprising an imaging unit configured for imaging a food to be eaten at a predetermined distance and estimating a caloric value thereof.
31. The system of claim 16, wherein said at least one user's device is selected from the group consisting of a smartphone, a desktop computer, a laptop computer, a tablet computer, a game console and any combination thereof.
32. A method of training an algorithm of predicting health implications; said method comprising steps of: a. imaging each meal before eating; b. identifying an imaged meal; c. determining dimensions of identified components and caloric value thereof d. recording data relating to a dietary pattern, physical-exercise activities and a sleep pattern; e. collecting and storing recorded data; f. building an algorithm of predicting health implications.
33. The method according to claim 32, wherein said step of building an algorithm is performed on the basis of machine learning or a neural network.
34. The method according to claim 32, wherein said health implications comprises change in user's weight depending on a dietary pattern, physical-exercise activities and a sleep pattern.
PCT/IL2021/050418 2020-04-13 2021-04-13 Improvement of disease prevention, diagnosis and treatment by application of multiple information sources WO2021209989A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202063008872P 2020-04-13 2020-04-13
US63/008,872 2020-04-13

Publications (1)

Publication Number Publication Date
WO2021209989A1 true WO2021209989A1 (en) 2021-10-21

Family

ID=78084406

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IL2021/050418 WO2021209989A1 (en) 2020-04-13 2021-04-13 Improvement of disease prevention, diagnosis and treatment by application of multiple information sources

Country Status (1)

Country Link
WO (1) WO2021209989A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116795896A (en) * 2023-08-29 2023-09-22 中南大学湘雅医院 Big data-based rehabilitation exercise strategy generation method and device

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20180092905A (en) * 2016-03-21 2018-08-20 하나텍플러스 주식회사 A System for Managing a Personal Nutrition Based on a Health Analysis Using a Application and a Method for the Same
US10517479B2 (en) * 2006-06-30 2019-12-31 Koninklijke Philips N.V. Mesh network personal emergency response appliance

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10517479B2 (en) * 2006-06-30 2019-12-31 Koninklijke Philips N.V. Mesh network personal emergency response appliance
KR20180092905A (en) * 2016-03-21 2018-08-20 하나텍플러스 주식회사 A System for Managing a Personal Nutrition Based on a Health Analysis Using a Application and a Method for the Same

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116795896A (en) * 2023-08-29 2023-09-22 中南大学湘雅医院 Big data-based rehabilitation exercise strategy generation method and device
CN116795896B (en) * 2023-08-29 2023-10-27 中南大学湘雅医院 Big data-based rehabilitation exercise strategy generation method and device

Similar Documents

Publication Publication Date Title
US10846622B2 (en) Methods and systems for an artificial intelligence support network for behavior modification
Frank et al. Health-related behaviors of women physicians vs other women in the United States
Egede et al. Modifiable cardiovascular risk factors in adults with diabetes: prevalence and missed opportunities for physician counseling
Weinstock et al. Reliability of the histopathologic diagnosis of melanocytic dysplasia
Waki et al. DialBetics with a multimedia food recording tool, FoodLog: smartphone-based self-management for type 2 diabetes
US9754075B1 (en) Medical symptoms tracking apparatus, methods and systems
WO2013065246A2 (en) Diathesis determination apparatus, diathesis determination method, health assistance apparatus, health assistance method, program, terminal apparatus, and health assistance system
CN110838367A (en) Comprehensive health risk assessment method
KR102297367B1 (en) Server for providing health care service using biometric information gathering and online checkup
JP2018124702A (en) Etiological analysis device and disease prediction device
US11461700B2 (en) Methods and systems for using artificial intelligence to analyze user activity data
Arcury et al. Older adults’ self-management of daily symptoms: complementary therapies, self-care, and medical care
Taskin Yilmaz et al. The effect of body image on sexual quality of life in obese married women
US20200321121A1 (en) Methods and systems for an artificial intelligence support network for vibrant constitutional guidance
Meyer et al. Sociodemographic differences in fast food price sensitivity
JP2017174168A (en) Health management server and health management system
JP2023535849A (en) Problem recommendation method and its device and its system, electronic equipment, readable storage medium
KR20210052122A (en) System and method for providing user-customized food information service
Jung Digital health care and the fourth industrial revolution
US20220358409A1 (en) Methods and systems for generating a supplement instruction set using artificial intelligence
WO2021209989A1 (en) Improvement of disease prevention, diagnosis and treatment by application of multiple information sources
Grady et al. Patients with diabetes using a new glucose meter with blood sugar mentor and dynamic color range indicator features show improved interpretation and willingness to act on blood glucose results (ASCEND study)
US9116734B1 (en) Dispersive storage area networks
Greaney et al. Factors associated with choice of web or print intervention materials in the healthy directions 2 study
Elran-Barak The associations of healthful weight-control behaviors with psychological distress and changes in body mass index among young adults

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21788946

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21788946

Country of ref document: EP

Kind code of ref document: A1