AU2020102010A4 - PAM-Physical Fitness: PREDICTION OF FITNESS AND PHYSICAL ACTIVITY LEVEL USING MACHINE LEARNING PROGRAMMING - Google Patents

Abstract

Patent Title: PAM-Physical Fitness: PREDICTION OF FITNESS AND PHYSICAL ACTIVITY LEVEL USING MACHINE LEARNING PROGRAMMING. ABSTRACT Our invention "PAM-Physical Fitness" is a process and computer programs are presented for creating a unified data stream from multiple data streams acquired from multiple devices. The Invented single method includes an operation for receiving activity data streams from the devices, each activity data stream being associated with physical activity data of a any user. The invented the method also includes an operation for assembling the unified activity data stream for a period of time. The unified activity data stream includes data segments from the data streams of at least two devices, and the data segments are organized time-wise over the period of time. A server includes a communications module, a memory, and a processor. The communications module is operable to receive a plurality of activity data streams from a plurality of devices and each activity data stream being associated with physical activity data of a user. The invented method the memory is operable to store the plurality of activity data streams and a unified activity data stream that includes data segments from the data streams of at least two devices of the plurality of devices. In addition to the processor is operable to assemble the unified activity data stream for the user over a period of time. 23 Dr. Vineet Tirth (Associate Professor) Dr. Ram Karan Singh (Professor) Dr. Manisha Bhatkulkar (Assistant Professor) Dr. Neeraj Kumar Shukla (Associate Professor) Dr. M. Ramkumar Raja (Associate Professor) Dr. Shilpi Birla (Associate Professor) Prof.(Dr.) S. B. Chordiya (Director-SIMMC-Campus) TOTAL NO OF SHEET: 07 NO OF FIG: 09 Home Ar Quality DQAnl-arO r tiCon 5A4 Po~en Count - medium UV indexbigh idak 0.3 ppm Carbon Monoxide -aceptable 6. rdsenw Hours exposed=3.5, low risk 43 minutes s hors 1253 steps (-SlEp Elency 0 0 emm Tys-.0e meway UM a... Ie~w aVws E- I~aMmy Low mosem FIG.1: IS A DIAGRAM DATA THAT COULD BE COLLECTED DURING A PERSON'S DAILY ROUTINE.

Description

Dr. Vineet Tirth (Associate Professor) Dr. Ram Karan Singh (Professor) Dr. Manisha Bhatkulkar (Assistant Professor) Dr. Neeraj Kumar Shukla (Associate Professor) Dr. M. Ramkumar Raja (Associate Professor) Dr. Shilpi Birla (Associate Professor) Prof.(Dr.) S. B. Chordiya (Director-SIMMC-Campus) TOTAL NO OF SHEET: 07 NO OF FIG: 09

Home Ar Quality DQAnl-arO r tiCon 5A4 Po~en Count - medium UV indexbigh idak 0.3 ppm Carbon Monoxide -aceptable 6. rdsenw Hours exposed=3.5, low risk

43 minutes s hors 1253 steps (-SlEp Elency 0 0

emm

Tys-.0e meway UM a... Ie~w aVws E- I~aMmy Low mosem

FIG.1: IS A DIAGRAM DATA THAT COULD BE COLLECTED DURING A PERSON'S DAILY ROUTINE.

Australian Government IP Australia Innovation Patent Australia

Patent Title: PAM-Physical Fitness: PREDICTION OF FITNESS AND PHYSICAL ACTIVITY LEVEL USING MACHINE LEARNING PROGRAMMING.

Name and address of patentees(s):

Dr. Vineet Tirth (Associate Professor) (Mechanical Engineering Department) Address: College of Engineering, King Khalid University & Research Center for Advanced Materials Science (RCAMS), King Khalid University, P.O. Box 9004, Abha-61413, Asir, Kingdom of Saudi Arabia.

Dr. Ram Karan Singh (Professor) (Dean RDIL and PhD Studies) Address: Department of Civil and Environmental Engineering, The NorthCap University, Gurugram, Haryana-122017, India.

Dr. Manisha Bhatkulkar (Assistant Professor) (Head, Department of Zoology) Address: Jawaharlal Nehru Arts, Commerce and Science College, Wadi, Amrawati Road, Nagpur-440023, Maharashtra, India.

Dr. Neeraj Kumar Shukla (Associate Professor) (Electrical Engineering Department) Address: College of Engineering, King Khalid University, Abha-61413, Asir, Kingdom of Saudi Arabia.

Dr. M. Ramkumar Raja (Associate Professor) (Electrical Engineering Department) Address: College of Engineering, King Khalid University, Abha-61413, Asir, Kingdom of Saudi Arabia.

Dr. Shilpi Birla (Associate Professor) (Department of Electronics & Communication Engineering) Address: Manipal University Jaipur, Jaipur-303007, Rajasthan, India.

Prof.(Dr.) S. B. Chordiya (Director-SIMMC-Campus) Address: Suryadatta Institute of Management & Mass Communication (SIMMC) Sr. No: 342, Bavdhan, Pune-411021, MH, India.

Complete Specification: Australian Government

FIELD OF THE INVENTION

Our Invention "PAM-Physical Fitness" is related to prediction of fitness and physical activity level using machine learning programming and computer programs for consolidating overlapping data provided by multiple devices.

BACKGROUND OF THE INVENTION

The use of portable devices has grown exponentially over the last few decades, and in particularly, the use of biometric monitoring devices that users wear on their bodies to measure activity levels, as well measuring environmental or user parameters (e.g., temperature, heart rate, altitude, etc.). Sometimes, data related to user activities may be acquired from multiple devices, such as a pedometer, a smart phone, a GPS (Global Positioning System) device, a thermometer, a weight scale, etc.

Occasionally, two or more devices can provide information about the same parameter related to activities of the user. For example, a user may get a step count from a pedometer worn on the wrist and from a step counter embedded on a running shoe. However, the accuracy of the different devices may change according to the inherent accuracy of the device or according to environmental circumstances (e.g., poor GPS satellite data reception). When the user gets conflicting information from multiple devices, the user may not know which is the accurate data. A system is desired to identify the best source of information in order to provide the best possible data to the user.

PRIOR ART SEARCH

US2284849A1941-08-291942-06-02 Edward P Schreyer Book end. US2717736A1951-07-171955-09-13 Robert A Schlesinger Exercise counting apparatus. US2883255A1954-04-281959-04-21 Panellit Inc. Automatic process logging system. US3163856A1961-11-141964-12-29 Frederick G Kirby Alarm device for indicating lack of motion. US3250270A1962-09-191966-05-10 Bloom Walter Lyon Device and method for measuring the calories an individual expends. US3918658A1973-06-151975-11-11 Klippan NV. Seat belt retractor having inertial device activated by two stimuli. US4192000A1977-07-141980-03-04 Calorie Counter Limited Partnership Electronic calorie counter. US4244020A1979-01-151981-01-06 Ratcliff Lloyd P Caloric and/or carbohydrate calculator.

This application is a continuation-in-part of Ser. No. 13/959,714, filed Aug. 5, 2013, titled "Methods and Systems for Identification of Event Data Having Combined Activity and Location Information of Portable Monitoring Devices", which claims priority to U.S. Provisional Patent Application 61/680,230 filed Aug. 6, 2012, and is a continuation-in part of U.S. patent application Ser. No. 13/759,485 (now issued as U.S. Pat. No. 8,543,351, issued on Sep. 24, 2013), filed on Feb. 5, 2013, titled "Portable Monitoring Devices and

Methods for Operating Same", which is a divisional of U.S. patent application Ser. No. 13/667,229, filed on Nov. 2, 2012, titled "Portable Monitoring Devices and Methods for Operating Same" (now issued as U.S. Pat. No. 8,543,351, issued on May 7, 2013), which is a divisional of U.S. patent application Ser. No. 13/469,027, now U.S. Pat. No. 8,311,769, filed on May 10, 2012, titled "Portable Monitoring Devices and Methods for Operating Same", which is a divisional of U.S. patent application Ser. No. 13/246,843, now U.S. Pat. No. 8,180,591, filed on Sep. 27, 2011, which is a divisional of U.S. patent application Ser. No. 13/156,304, filed on Jun. 8, 2011, titled "Portable Monitoring Devices and Methods for Operating Same", which claims the benefit of and priority to, under 35 U.S.C. 119§(e), to U.S. Provisional Patent Application No. 61/388,595, filed on Sep. 30, 2010, and titled "Portable Monitoring Devices and Methods for Operating Same" and to U.S. Provisional Patent Application No. 61/390,811, filed on Oct. 7, 2010, and titled "Portable Monitoring Devices and Methods for Operating Same", all of which are hereby incorporated by reference in their entirety.

OBJECTIVES OF THE INVENTION

1. The objective of the invention is to a process and computer programs are presented for creating a unified data stream from multiple data streams acquired from multiple devices. 2. The other objective of the invention is to the Invented single method includes an operation for receiving activity data streams from the devices, each activity data stream being associated with physical activity data of a any user. 3. The other objective of the invention is to the invented the method also includes an operation for assembling the unified activity data stream for a period of time. 4. The other objective of the invention is to the unified activity data stream includes data segments from the data streams of at least two devices, and the data segments are organized time-wise over the period of time. 5. The other objective of the invention is to a server includes a communications module, a memory, and a processor. The communications module is operable to receive a plurality of activity data streams from a plurality of devices and each activity data stream being associated with physical activity data of a user. 6. The other objective of the invention is to the invented method the memory is operable to store the plurality of activity data streams and a unified activity data stream that includes data segments from the data streams of at least two devices of the plurality of devices. The processor is operable to assemble the unified activity data stream for the user over a period of time.

SUMMARY OF THE INVENTION

Methods, devices, systems, and computer programs are presented for consolidating overlapping data provided by multiple sources. It should be appreciated that the present embodiments can be implemented in numerous ways, such as a method, an apparatus, a system, a device, or a computer program on a computer readable medium. Several embodiments are described below.

The invented methods, systems, and computer programs for combining or synthesizing multiple data streams into a unified data stream while maintaining accuracy and integrity of the data. In an embodiment, the data includes environmental and biometric data. In another embodiment, the multiple data streams originate from, or are collected by, multiple devices that monitor the same or similar physical phenomena. For example, one embodiment includes a method used by a cloud-based activity-monitoring server to combine data streams from multiple portable biometric monitoring devices tracking a single user or group of users. In another embodiment, the method pertains to a device (e.g., personal computer, tablet computer, smartphone) that combines data streams from multiple portable biometric monitoring devices tracking a single user or a group of users.

A method includes operations for receiving a plurality of activity data streams from a plurality of devices, each activity data stream being associated with physical activity data of a user. In addition, the method includes an operation for assembling a unified activity data stream for the user over a period of time, the unified activity data stream including data segments from the data streams of at least two devices of the plurality of devices, and the data segments being organized time-wise over the period of time. In one embodiment, the operations of the method are executed by a processor.

A server includes a communications module, a memory, and a processor. The communications module is operable to receive a plurality of activity data streams from a plurality of devices each activity data stream being associated with physical activity data of a user. Further, the memory is operable to store the plurality of activity data streams and a unified activity data stream that includes data segments from the data streams of at least two devices of the plurality of devices. In addition, the processor is operable to assemble the unified activity data stream for the user over a period of time. The data segments are organized time-wise over the period of time.

A non-transitory computer-readable storage medium storing a computer program is provided. The computer-readable storage medium includes program instructions for receiving a plurality of activity data streams from a plurality of devices, each activity data stream being associated with physical activity data of a user. Further, the storage medium includes program instructions for assembling a unified activity data stream for the user over a period of time, the unified activity data stream including data segments from the data streams of at least two devices of the plurality of devices, the data segments being organized time-wise over the period of time.

One method includes operations for receiving data streams regarding activity of a user, each data stream being associated with a respective device of a plurality of devices, and for evaluating one or more rules for consolidating the data streams into a unified activity data stream. Evaluating the one or more rules further includes identifying first time segments where data is available from a single device from the plurality of devices; adding the available data to the unified activity data stream for the first time segments; identifying second time segments where data exists from two or more devices from the plurality of devices; and adding data to the second time segments based on the existing data from the two or more devices and based on the one or more rules. Further, the method includes an operation for storing the unified activity data stream for presentation to the user. In one embodiment, the operations of the method are executed by a processor.

BRIEF DESCRIPTION OF THE DIAGRAM

FIG. 1: is a diagram illustrating data that could be collected during a person's daily routine.

FIG. 2: is a block diagram of data stream synthesis.

FIG. 3A: is a block diagram of data stream synthesis.

FIG. 3B: is an exemplary architecture of the data synthesizer.

FIG. 3C: is a flowchart of a method for combining multiple data streams.

FIG. 4: is a diagram of data stream synthesis from multiple devices.

FIG. 5: is a simplified schematic diagram of a device for implementing.

FIG. 6: is a flowchart an algorithm for performing data synthesis for data provided by a plurality of devices.

FIG. 7: is an example where various types of activities of users can be captured or collected by activity tracking devices.

DESCRIPTION OF THE INVENTION

These metrics include, but are not limited to, energy expenditure (e.g., calorie burn), floors climbed or descended, heart rate, heart rate variability, heart rate recovery, location and/or heading (e.g., through GPS), elevation, ambulatory speed and/or distance traveled, swimming lap count, bicycle distance and/or speed, blood pressure, blood glucose, skin conduction, skin and/or body temperature, electromyography, electroencephalography, weight, body fat, caloric intake, nutritional intake from food, medication intake, sleep periods (i.e., clock time), sleep phases, sleep quality, and/or sleep duration, and respiration rate. The device may also measure or calculate metrics related to the environment around the user such as barometric pressure, weather conditions (e.g., temperature, humidity, pollen count, air quality, rain/snow conditions, wind speed), light exposure (e.g., ambient light, UV light exposure, time and/or duration spent in darkness), noise exposure, radiation exposure, and magnetic field.

As used herein, the term "sync" refers to the action of exchanging data between a first device and a second device to update the second device with new information available to the first device that is not yet available to the second device. Additionally, "sync" may also referred to the exchange of information between two devices to provide updates to one of the devices with information available to the other device, or to coordinate information that is available, overlapping, or redundant in both devices. "Sync" may also be used in reference to sending and/or receiving data to and/or from another computing device or electronic storage devices including, but not limited to, a personal computer, a cloud based server, and a database. In some embodiments, a sync from one electronic device to another may occur through the use of one or more intermediary electronic devices. For example, data from a personal biometric device may be transmitted to a smart phone that forwards the data to a server.

Furthermore, a device or system combining multiple data streams may calculate metrics derived from this data. For example, the device or system can calculate the user's stress levels and the risk level of elevated stress, through one or more of heart rate variability, skin conduction, noise pollution, and sleep quality. Conversely, the device or system may calculate the user's stress reduction (e.g., calmness), or the stress reduction risk through one or more of the preceding parameters. In another embodiment, the device or system may determine the efficacy of a medical intervention (e.g., medication) through the combination of medication intake, sleep data, and/or activity data. In yet another example, the device or system may determine the efficacy of an allergy medication through the combination of pollen data, medication intake, sleep data and/or activity data. These examples are provided for illustration and not to be interpreted to be exclusive or limiting.

The user may employ more than one portable monitoring device. For a case in which the user tracks the same class of data during the same period of time with more than one device, the two resulting data streams are combined or synthesized as described more fully below according to several embodiments.

"Class of data" or "data class" refers to data that represents a quantifiable characteristic or a set of quantifiable characteristics in one or more ways. For example, one data class may be ambulatory steps count. "Data type" refers to a specific representation of a quantifiable characteristic. Examples of data types may include walking steps per minute and running steps per minute. It is noted that both walking steps per minute and running steps per minute may be considered a part of the data class ambulatory step count.

FIG. 1: is a diagram illustrating data that could be collected during a person's daily routine. The one or more tracking devices may capture different types of data and identify different states of the user. For example, the system may identify when the user is sleeping, commuting, working, outdoors, running, at the gym, etc.

For each of the states of the user, different biometric and environmental parameters may be captured, such as the pollen in the air, the amount of carbon monoxide in the air, how many hours the user slept, how much time the user spent commuting, how much activity took place during the workday, how much solar radiation exposure, how many steps climbed, etc. As the user goes about their day, the user may utilize different trackers.

Sometimes, the user may have multiple devices tracking user activities, such as a pedometer, a mobile phone, the step tracker, etc.

Embodiments presented herein consolidate the information obtained from multiple devices to provide the best available picture to the user about the user levels of activity, activities associated there with, environmental factors, etc. However, when the same information is provided by two different trackers, there can be two different data points and the system provides heuristics to determine what is the best data point, or if a combination of the data points is the best approach. For example, if two different devices are tracking the heart rate of the user, in one embodiment, the consolidated data includes the highest peaks and valleys of the heart rate chart. Sometimes a statistical combinations or measurements are used to consolidated data. For example, the consolidated data may include one or more of average, medium, maximum, or minimum of the multiple data points.

FIG.2: is a block diagram of data stream synthesis according to one embodiment. In one embodiment, a user prefers to wear a specific device during the day, for example one which is clipped to the belt, and a different device at night which is more comfortable for sleeping, such as a wrist band. The data synthesizer can use the belt clip device data during the day time, when the wrist band shows no activity, and the wrist band device data during the night, when the belt clip shows no activity. In such a case, the two devices may not be given a "priority" over each other. Indeed, the data synthesizer may not have any prioritization structure at all. Instead, in one embodiment, data streams may be selected solely on the activity level.

In some cases, device data stream priority is used in addition to activity level to determine the selected data stream. In one embodiment, a user wears a belt clip device and an anklet device during the day, and a belt clip device and a wrist band device at night. Assuming the anklet data stream is given priority 1, the wrist band priority 2, and the belt clip priority 3, the data synthesizer shows the data from the anklet device during the day because the anklet device data has the highest priority of the active devices for that time period, and the wrist band device at night because the wrist band device has the highest priority of the active devices for that time period. The anklet has a higher priority than the wrist band, but the anklet is not active at night.

A similar scheme is applicable to gaps in data due to syncing delays. For example, a user uses a pedometer and another device that measures the distance a user walks. The pedometer may have synced more recently than the other device. The device that has not synced recently may have a higher priority for the "distance walked" class of data (for example, a GPS device vs. a pedometer). In this case, the pedometer data is used for the time period between the time of the last GPS device sync and the time of the more recent pedometer sync. When the GPS data syncs, the data synthesizer can replace the distance data from the pedometer with that of the GPS.

The system stores all the sensor data (either locally in the device or remotely in the server), so it is possible to reconcile the data stream for different use cases when new data is available, even going back in time and reconcile historical data with the newly available data. In one embodiment, a storage database keeps all the data collected, as well as the reconciled data, also referred to herein as consolidated data, which is the result of combining two or more data streams. Further, it is possible to do further reconciliation of historical data by going back to a specific time period and request the system to perform a reconciliation of the multiple data streams.

FIG. 3A: is a block diagram of data stream synthesis according to one embodiment. In one embodiment, multiple data streams of the same type are combined using one or more methods. One method of combining data streams of the same type is to average them. Other methods include a weighted average where the weight of one data stream over another is proportional to its accuracy. For example, if the two or more data streams have a probabilistic measure of accuracy, then the combined estimate is taken as the expectation. It may also be taken as the most likely (i.e., the value with the highest probability of being correct).

The algorithm that is used to combine data streams changes depending on various factors including the number of data streams, the quality of data streams (precision and accuracy), and the source of the data streams (which model device, first party or third party, etc.). In some embodiments, the algorithm uses rules that define priorities for the data streams and methods of combining the data from the multiple streams to create a consolidated data stream having information from one or more multiple sources.

When one device does not acquire data for a certain period of time, the data synthesizer is able to maintain a continuous data stream by filling in the gap with data from other devices. The data synthesizer can fill in the gap of data even if one device's data has a lower priority than another device that was not acquiring data. In some cases, more than one device may have the same priority. In such cases, the device that is currently measuring data can be used. This is useful in cases when one device is not able to measure data, for example because of low battery.

The data stream which has data that shows characteristics of being used is selected by the data synthesizer. For example, for the step counts data stream, the data stream which has step counts greater than zero is selected because having more than zero steps is characteristic of the device being used. In another example, an accelerometer data stream shows characteristics of being worn when a certain pattern of accelerations is detected.

FIG.3B: is a exemplary architecture of the data synthesizer, according to one embodiment. In one embodiment, the data synthesizer 202 includes rule processor 1102, rules database 1104, the GPS module 1106, a timekeeper module 1108, a data selector/combiner module 1110, and a data stream database 1112. Data synthesizer 202 receives one or more data streams A, B, C, environmental data, and any other type of external data that may have information regarding user activities (e.g., user calendar). The rule processor identifies the incoming data from the data streams, environmental data, and other data, and analyzes the pertaining rules to apply them to the incoming data. In one embodiment, the rules database 1104 keeps track of the rules, but the rules may be stored elsewhere and might be change over time to fine-tune the performance of the data synthesizer 1102. In one embodiment, the user is able to create rules for prioritizing data, such as identifying the user's favorite (i.e., highest priority) device.

After evaluating the different rules, the data selector/combiner consolidates, also referred to as reconciles, the multiple data streams to create a consolidated data. As used herein, consolidated data refers to the analysis of data received from multiple sources to generate a data source with information available for presentation to the user. For example, consolidated data may include filling gaps in the data stream from one device with data from another device, selecting which data to choose when multiple sources of data provide information about the same event (e.g., step counts during a certain time interval), or any combination thereof.

For example, a user may have three different data streams from three different trackers. One of them may indicate that the user is at work, while the other two indicate that the user is at home. However, the rule processor 1102 checks the user's calendar, and the calendar indicates that the user is at work, so the data selector 1110 may use the data from the tracker that is in accordance with the calendar, even though that tracker may be in the "minority." Therefore, external data may be used to determine the priority of different data sources. Also, external data may be used to determine priority with data that is not in the data stream.

The data synthesizer has access to calendar and contact information for the user. The contacts may have addresses identifying a location, and based on the physical location of the contact, the data synthesizer may derive that the user is at the country club, so walking activity may be associated with the activity of playing golf. In this case, the data from a tracker may be given higher priority when considering that the user is playing golf.

Any method for capturing data may be used for the embodiments. For example, data may be acquired via and API to access a web service (e.g., a web service that captures data for one of the trackers of the user). If the user provides login information for the external website, data synthesizer 202 may access the web service using an API or some other data exchange mechanism. The data is reconciled as the data comes in, that is the data is reconciled "on-the-fly," and the combined, or consolidated, data stream is provided as an output stored in data stream database 1112. The data is reconciled during predetermined time periods (e.g., at night once a day). Sometimes there are delays when capturing the data, so the actual reconciliation with all the available data is done when all the data, or at least data from a plurality of devices, is available. In some embodiments, the same data may be reconciled multiple times; as more data becomes available, new reconciliations may overwrite previous reconciliations.

A data stream database 1112 is a time series database. A time series database server (TSDS) is a software system that is optimized for handling time series data, such as arrays of numbers indexed by time (e.g., a date-time or a date-time range). Sometimes the time series are called profiles, curves, or traces. TSDS are often natively implemented using specialized database algorithms. However, it is possible to store time series as binary large objects (BLOBs) in a relational database or by using a VLDB approach coupled with a pure star schema. Efficiency is often improved if time is treated as a discrete quantity rather than as a continuous mathematical dimension.

Database joins across multiple time series data sets is only practical when the time tag associated with each data entry spans the same set of discrete times for all data sets across which the join is performed. The TSDS allows users to create, enumerate, update and destroy various time series and organize them in some fashion. These series may be organized hierarchically and optionally have companion metadata available with them. The server often supports a number of basic calculations that work on a series as a whole, such as multiplying, adding, or otherwise combining various time series into a new time series. The server can also filter on arbitrary patterns defined by the day of the week, low value filters, high value filters, or even have the values of one series filter another. Some TSDSs also provide statistical functions for synthesizing multiple sources of data.

The data streams are saved in the time series database as the data comes in. In addition, summary series are recalculated in some embodiments; for example, summaries can be calculated for reconciled data for one hour, two hours, daily, weekly, etc. The use of a time series database facilitates the easy extraction of data from the database to present many types of graphs or many types of data made available to the user. For example, a user wants a count of all the steps taken in a year. In one embodiment, the step count is stored as minute data, meaning that the number of steps taken in one minute is stored in the database together with the timestamp. In this case, the user has data for three different trackers that have been used throughout the year. If the system calculates the summary for each of the data streams separately, the final step count may be different for each of the devices, among other reasons, because some devices may not be in use while other devices are in use. In order to get the reconciled total, a reconciliation is made for these three data streams to obtain a resulting data serious that accounts for the step counts throughout each interval (e.g., a minute). In other words, the reconciliation calculates the step count for each minute of the year by looking at the data provided for that minute by all three devices. Once the reconciliation is made by minute, the total for the year is calculated and presented to the user.

As the data comes in, the data synthesizer 202 makes the determination of whether to store all the incoming data, or reconcile the incoming data, or both. It is noted that the embodiments illustrated in FIG. 3B are exemplary. Other embodiments may utilize different modules or group the modules into other configurations, as long as the described functionality is provided by data synthesizer 202. The embodiments illustrated in FIG. 3B should therefore not be interpreted to be exclusive or limiting, but rather exemplary or illustrative.

FIG. 3C is a flowchart of a method for combining multiple data streams, according to one embodiment. In operation 802, device data is obtained from one or more tracking devices. From operation 802, the method flows to operation 804 where additional data is obtained, such as environmental data (time, temperature, whether, barometric pressure, etc.). However, in one embodiment, operation 804 is optional and the data reconciliation only applies to data streams from tracking devices. From operation 804, the method flows to operation 806 where the rules for combining or selecting data are obtained. In one embodiment, the rules are kept in rules database 1104 of FIG. 3B, but in other embodiments, the rules database may be stored in other places, such as in the server memory.

From operation 806, the method flows to operation 808 where the user activity is identified for one or more time periods, based on the obtained data and the obtained rules. From operation 808, the method flows to operation 810 where the system sets a directive for combining or selecting data based on the rules and user activity. This directive is used to identify the process by which data is to be combined from the multiple data streams.

From operation 810, the method flows to operation 812 where the data from the multiple data sources is combined or reconciled based on the directive identified in operation 810. The result is an output data stream of consolidated data, also referred to as reconciled data. The output stream is stored in operation 814. In one embodiment, the output stream is stored in data stream database 1112 of FIG. 3B. In operation 816 the output stream is provided to the user or to another computing device (e.g., a smart phone). While the various operations in this flowchart are presented and described sequentially, one of ordinary skill will appreciate that some or all of the operations may be executed in a different order, be combined or omitted, or be executed in parallel.

FIG. 4: is a diagram of data stream synthesis from multiple devices according to one embodiment. However, it should be noted that the data synthesizer can also use data streams from multiple users to improve the data quality of a single user. The data synthesizer uses the data streams of more than one user to create statistics which are used to set defaults to a single user's account. For example, if a user does not enter their stride length data on their device or on the account associated with the data synthesizer, a default stride length is set for them based on typical stride lengths for users of similar age, height, and gender. Users can opt in or opt out of allowing the data synthesizer to use their data on other users' accounts. Users can also opt in or opt out of allowing other users' data to modify their data.

The data synthesizer uses a subset of all of the users determined to be more relevant to the individual user. For example, the data synthesizer uses only data from users who have linked accounts through online social connections. Such "shared" data may be used in the method described above to determine a user's stride length if it is not entered by the user.

Other uses of shared data may involve an algorithm which determines when two users have performed the same activity together. For example, if two users have location aware devices that also contain activity monitors, the data synthesizer determines that the two users have completed a workout together, if the users were in a pre-defined proximity at the same time they each had a similar activity level.

Once the data synthesizer determines that two users have completed a workout together, the data synthesizer uses the data of the respective users to improve or add to the biometric data collected by each user. For example, if two users go for a walk together, and user 1 had a device with an altimeter and user 2 did not, the data synthesizer can add the altimeter data stream from the account of user 1 for the duration of the walk to the account of user 2. The data synthesizer can also prompt both users to connect socially online by "friending" each other, if the users have not already. Further, the detection of a jointly performed workout or social gathering may be awarded by the data synthesizer as in a virtual badge or activity points.

The accuracy or trustworthiness of individual sensors (or sensor models or types) is calculated by one or more of a user rating and prioritization settings in the system. The data synthesizer may automatically rank the accuracy of connected sensors (e.g., a particular model of pedometer) based on the information from the data synthesis system. This ranked data may be shared with the user to recommend a more accurate or trustworthy sensor instead of the current sensor. This data can also be used to train the default prioritization settings of new users.

The data synthesizer may export data in various ways. Data streams can be viewed on a web site using any computing device with a web browser or appropriate mobile application (app). A user can choose to export their data streams to other devices or services. For example, a user may want to have the synthesized data shown on one or more of their biometric monitoring devices. The user may choose to have their data shared with various health monitoring services such as Microsoft Health Vault1M or MyFitnessPal T. MThe use of a standard data identification and structure aids in the compatibility of third party services, and so enhances the user experience.

The data synthesizer is able to determine which data streams are the most accurate. With this knowledge, devices that have incorrect data, or have data whose accuracy is lower than the most accurate data, have their data replaced with "better," more accurate data. For example, suppose a user has a pedometer device that uses inertial sensors to determine the distance a user has walked, and also has a GPS enabled device. The data synthesizer can determine that the distance data as measured by the GPS enabled device is more accurate than that of the inertial sensor pedometer. The data synthesizer can update the inertial pedometer with the more accurate GPS pedometer data.

The determination of which data stream is most accurate is based on predetermined accuracies for specific sensors. In another embodiment, algorithms determine the accuracy of a data stream solely based on the data stream's data and/or the data stream compared to a second data stream which may indicate the accuracy of the first data stream. This can be extended to the comparison of more than two data streams. For example, data from a GPS, inertial sensor pedometer, and location as determined by cellular tower triangulation may be compared to determine which is most accurate.

The data stream synthesizer can also create a function that relates a device's raw data to a more accurate data stream or combination of data streams. The data stream synthesizer compares a single device's data stream to a data stream that is more accurate. The data stream with higher accuracy may be from a device that is known to have higher accuracy, a device that is determined to have higher accuracy through a comparison with other data streams, or it may be from a combination of multiple data streams that together have a higher accuracy than any single data stream. The data stream synthesizer determines a functional relationship between a single device's data stream and the higher accuracy data stream. This functional relationship is shared with the device so that the device can correct its algorithms and improve its accuracy. In one embodiment, this functional relationship is characterized as a calibration. In a case in which the device displays data, this allows the user to see higher accuracy biometric or environmental signals before these signals are uploaded and combined in the data synthesizer.

The data synthesizer automatically calibrates a user's stride length, or the function mapping of the user's stride length (in the case that the stride length is a function of step frequency, foot contact time, surface slope, etc.). In an embodiment, this is accomplished by using the data stream of a GPS or other location aware device to create an accurate measurement of the distance that the user walks, runs, jogs, etc., in addition to a data stream from a pedometer. The distance as measured from the GPS may also be used in combination with the number of steps measured by a pedometer to create an accurate measurement of the user's stride length.

The system performs a discrete calibration event. In an embodiment, this involves the user uploading, sharing, or entering high accuracy data with the data synthesizer. This data is then used to calibrate any devices connected to the user's account. For example, a user uploads, enters, or authorizes the sharing of data that was collected from a doctor's appointment using high accuracy equipment including but not limited to data regarding blood pressure, body fat, resting heart rate, height and body weight. Such data may also be used to retroactively correct a user's historical data.

The data synthesizer provides the synthesized data to a user interface so that the user can review data. In an embodiment, the user interface allows the user to perform such tasks as correcting incorrect data, entering biometric data, modifying data stream prioritization settings, entering quality metric data for data streams, setting goals, entering data that is not tracked by the user's devices and interacting socially and competitively with other users. The user interface can also allow the user to quickly review and comprehend data from many different devices with varying sampling frequencies.

The data streams or algorithmic combinations of data streams may be used by the data synthesizer or a computing device connected to the data synthesizer (such as a server) to provide a rich interaction between the user and their data, and between the user and the data of others. In one embodiment the user is presented with a simplified view of all of their data streams. For example, a user's account website shows the user the total number of steps the user has as determined by a combination of all of their step counting data streams in a specific period of time including but not limited to hour, day, week, month, year, and the time period that the user has had an account. Other totals of data are selectively provided to the user in a similar manner including, but not limited to calories burned, distance of walks and/or runs, floors climbed, hours asleep, hours in bed, hours of a specific activity or combination of activities such as walking, biking, running, and/or swimming.

The user is able to compare user's data with the data of other users, and compete with other users or with themselves. For example, the user can choose to be ranked according to one or more data streams in a leaderboard with other users that the users have "friended." The user is able to compare their current data statistics with their own statistics from a different time period. For example, a user is shown how many more or less steps the user took the previous week compared to the current week. A user's data can also be used in virtual contests with set goals, goals based on the user's own statistics, or goals based on other user's statistics. Real or virtual prizes are awarded for winning such a contest. For example, a group of users competing in a corporate wellness challenge win prizes based on data acquired and combined by the data synthesizer.

A user is awarded a virtual badge for reaching a certain data metric using the data acquired and/or combined by the data synthesizer. For example, a user is awarded a badge for climbing 50 floors in one day. In another example, a user is awarded a badge for taking a total of 10,000 steps since the user started acquiring step data.

FIG. 5: is a simplified schematic diagram of a device for implementing embodiments described herein. The monitoring device 152 is an example of any of the monitoring devices described herein, and including a step tracker, a fitness tracker without buttons, or a fitness tracker defined to be clipped onto the belt of a user, etc. The monitoring device 152 includes processor 154, memory 156, one or more environmental sensors 158, one or more position and motion sensors 160, watch 162, vibrotactile feedback module 164, display driver 168, touchscreen 206, user interface/buttons 170, device locator 172, external event analyzer 174, motion/activity analyzer 176, power controller 178, and battery 180, all of which may be coupled to all or some of the other elements within monitoring device 152.

Examples of environmental sensors 158 include a barometric pressure sensor, a weather condition sensor, a light exposure sensor, a noise exposure sensor, a radiation exposure sensor, and a magnetic field sensor. Examples of a weather condition sensor include sensors for measuring temperature, humidity, pollen count, air quality, rain conditions, snow conditions, wind speed, or any combination thereof, etc. Examples of light exposure sensors include sensors for ambient light exposure, ultraviolet (UV) light exposure, or a combination thereof, etc. Examples of air quality sensors include sensors for measuring particulate counts for particles of different sizes, level of carbon dioxide in the air, level of carbon monoxide in the air, level of methane in the air, level of other volatile organic compounds in the air, or any combination thereof.

Examples of the position/motion sensor 160 include an accelerometer, a gyroscope, a rotary encoder, a calorie measurement sensor, a heat measurement sensor, a moisture measurement sensor, a displacement sensor, an ultrasonic sensor, a pedometer, an altimeter, a linear position sensor, an angular position sensor, a multi-axis position sensor, or any combination thereof, etc. In some embodiments, the position/motion sensor 160 measures a displacement (e.g., angular displacement, linear displacement, or a combination thereof, etc.) of the monitoring device 152 over a period of time with reference to a three-dimensional coordinate system to determine an amount of activity performed by the user during a period of time. In some embodiments, a position sensor includes a biological sensor, which is further described below.

The vibrotactile module 164 provides sensory output to the user by vibrating portable device 152. Further, the communications module 166 is operable to establish wired or wireless connections with other electronic devices to exchange data (e.g., activity data, geo-location data, location data, a combination thereof, etc.). Examples of wireless communication devices include, but are not limited to, a Wi-Fi adapter, a Bluetooth device, an Ethernet adapter, and infrared adapter, an ultrasonic adapter, etc.

The touchscreen 206 may be any type of display with touch sensitive functions. In another embodiment, a display is included but the display does not have touch-sensing capabilities. The touchscreen may be able to detect a single touch, multiple simultaneous touches, gestures defined on the display, etc. The display driver 168 interfaces with the touchscreen 206 for performing input/output operations. In one embodiment, display driver 168 includes a buffer memory for storing the image displayed on touchscreen 206.

The buttons/user interface may include buttons, switches, cameras, USB ports, keyboards, or any other device that can provide input or output functions. Device locator 172 provides capabilities for acquiring data related to the location (absolute or relative) of monitoring device 152. Examples device locators 172 include a GPS transceiver, a mobile transceiver, a dead-reckoning module, a camera, etc. As used herein, a device locator may be referred to as a device or circuit or logic that can generate geo location data. The geo-location data provides the absolute coordinates for the location of the monitoring device 152. The coordinates may be used to place the monitoring device 152 on a map, in a room, in a building, etc. in some embodiments, a GPS device provides the geo-location data. In other embodiments, the geo-location data can be obtained or calculated from data acquired from other devices (e.g., cell towers, Wi-Fi device signals, other radio signals, etc.), which can provide data points usable to locate or triangulate a location.

External event analyzer 174 receives data regarding the environment of the user and determines external events that might affect the power consumption of the user. For example, the external event analyzer 174 may determine low light conditions in a room, and assume that there is a high probability that the user is sleeping. In addition, the external event analyzer 174 may also receive external data, such as GPS location from a smart phone, and determine that the user is on a vehicle and in motion.

The processor 154 receives one or more geo-locations measured by the device locator 172 over a period of time and determines a location of the monitoring device 152 based on the geo-locations and/or based on one or more selections made by the user, or based on information available within a geo-location-location database of the network. For example, the processor 154 may compare the current location of the monitoring device against known locations in a location database, to identify presence in well-known points of interest to the user or to the community. In one embodiment, upon receiving the geo-locations from the device locator 172, the processor 154 determines the location based on the correspondence between the geo-locations and the location in the geo-location-location database.

The one or more environmental sensors 158 may sense and determine one or more environmental parameters (e.g., barometric pressure, weather condition, amount of light exposure, noise levels, radiation levels, magnetic field levels, or a combination thereof, etc.) of an environment in which the monitoring device is placed. The watch 162 is operable to determine the amount of time elapsed between two or more events. In one embodiment, the events are associated with one or more positions sensed by the position sensor 160, associated with one or more environmental parameters determined by the environmental sensor 158, associated with one or more geo-locations determined by the device locator 172, and/or associated with one or more locations determined by the processor 154.

Power controller 178 manages and adjusts one or more power operational parameters defined for the monitoring device 152. In one embodiment, the power operational parameters include options for managing the touchscreen 206, such as by determining when to turn ON or OFF the touchscreen, scan rate, brightness, etc. In addition, the power controller 178 is operable to determine other power operational parameters, besides the parameters associated with the touchscreen, such as determining when to turn ON or OFF other modules (e.g., GPS, environmental sensors, etc.) or limiting the frequency of use for one or more of the modules within monitoring device 152.

FIG. 6: is a flowchart illustrating an algorithm for performing data synthesis for data provided by a plurality of devices, according to one embodiment. In operation 852, a plurality of devices is associated with a user, each device being operable to capture data associated with the user. From operation 852 the method flows to operation 854, where captured data is received from the plurality of devices, the captured data being about a first activity associated with a first period of time. From operation 854 the method flows to operation 856, where a detection takes place regarding the received captured data from two or more devices provides overlapping information about the first activity.

From operation 856 the method flows to operation 858, where one or more rules are evaluated, the one or more rules being for consolidating the overlapping information to produce consolidated data. The consolidated data provides a unified view of the first activity during the first period of time. From operation 858 the method flows to operation 860 for storing the consolidated data for presentation to the user. In one embodiment, the operations of the method are executed by a processor.

FIG. 7: is an example where various types of activities of users 900A-9001 can be captured or collected by activity tracking devices, in accordance with various embodiments of the present embodiments. As shown, the various types of activities can generate different types of data that can be captured by the activity tracking device 100. The data, which can be represented as motion data (or processed motion data) can be transferred 920 to a network176 for processing and saving by a server, as described above. In one embodiment, the activity tracking device 100 can communicate to a device using a wireless connection, and the device is capable of communicating and synchronizing the captured data with an application running on the server. In one embodiment, an application running on a local device, such as a smart phone or tablet or smart watch can capture or receive data from the activity tracking device 100 and represent the tract motion data in a number of metrics.

The device collects one or more types of physiological and/or environmental data from embedded sensors and/or external devices and communicates or relays such metric information to other devices, including devices capable of serving as Internet-accessible data sources, thus permitting the collected data to be viewed, for example, using a web browser or network-based application. For example, while the user is wearing an activity tracking device, the device may calculate and store the user's step count using one or more sensors. The device then transmits data representative of the user's step count to an account on a web service, computer, mobile phone, or health station where the data may be stored, processed, and visualized by the user. Indeed, the device may measure or calculate a plurality of other physiological metrics in addition to, or in place of, the user's step count.

Some physiological metrics include, but are not limited to, energy expenditure (for example, calorie burn), floors climbed and/or descended, heart rate, heart rate variability, heart rate recovery, location and/or heading (for example, through GPS), elevation, ambulatory speed and/or distance traveled, swimming lap count, bicycle distance and/or speed, blood pressure, blood glucose, skin conduction, skin and/or body temperature, electromyography, electroencephalography, weight, body fat, caloric intake, nutritional intake from food, medication intake, sleep periods (i.e., clock time), sleep phases, sleep quality and/or duration, pH levels, hydration levels, and respiration rate. The device may also measure or calculate metrics related to the environment around the user such as barometric pressure, weather conditions (for example, temperature, humidity, pollen count, air quality, rain/snow conditions, wind speed), light exposure (for example, ambient light, UV light exposure, time and/or duration spent in darkness), noise exposure, radiation exposure, and magnetic field.

Still further, other metrics can include, without limitation, calories burned by a user, weight gained by a user, weight lost by a user, stairs ascended, e.g., climbed, etc., by a user, stairs descended by a user, steps taken by a user during walking or running, a number of rotations of a bicycle pedal rotated by a user, sedentary activity data, driving a vehicle, a number of golf swings taken by a user, a number of forehands of a sport played by a user, a number of backhands of a sport played by a user, or a combination thereof. In some embodiments, sedentary activity data is referred to herein as inactive activity data or as passive activity data. In some embodiments, when a user is not sedentary and is not sleeping, the user is active. In some embodiments, a user may stand on a monitoring device that determines a physiological parameter of the user. For example, a user stands on a scale that measures a weight, a body fat percentage, a biomass index, or a combination thereof, of the user.

Furthermore, the device or the system collating the data streams may calculate metrics derived from this data. For example, the device or system may calculate the user's stress and/or relaxation levels through a combination of heart rate variability, skin conduction, noise pollution, and sleep quality. In another example, the device or system may determine the efficacy of a medical intervention (for example, medication) through the combination of medication intake, sleep and/or activity data. In yet another example, the device or system may determine the efficacy of an allergy medication through the combination of pollen data, medication intake, sleep and/or activity data. These examples are provided for illustration only and are not intended to be limiting or exhaustive.

This information can be associated to the users account, which can be managed by an activity management application on the server. The activity management application can provide access to the users account and data saved thereon. The activity manager application running on the server can be in the form of a web application. The web application can provide access to a number of websites screens and pages that illustrate information regarding the metrics in various formats. This information can be viewed by the user, and synchronized with a computing device of the user, such as a smart phone.

The data captured by the activity tracking device 100 is received by the computing device, and the data is synchronized with the activity measured application on the server. In this example, data viewable on the computing device (e.g., smart phone) using an activity tracking application (app) can be synchronized with the data present on the server, and associated with the user's account. In this way, information entered into the activity tracking application on the computing device can be synchronized with application illustrated in the various screens of the activity management application provided by the server on the website.

The user can therefore access the data associated with the user account using any device having access to the Internet. Data received by the network 176 can then be synchronized with the user's various devices, and analytics on the server can provide data analysis to provide recommendations for additional activity, and or improvements in physical health. The process therefore continues where data is captured, analyzed, synchronized, and recommendations are produced. In some embodiments, the captured data can be itemized and partitioned based on the type of activity being performed, and such information can be provided to the user on the website via graphical user interfaces, or by way of the application executed on the user's smart phone (by way of graphical user interfaces).

The sensor or sensors of a device 100 can determine or capture data to determine an amount of movement of the monitoring device over a period of time. The sensors can include, for example, an accelerometer, a magnetometer, a gyroscope, or combinations thereof. Broadly speaking, these sensors are inertial sensors, which capture some movement data, in response to the device 100 being moved. The amount of movement (e.g., motion sensed) may occur when the user is performing an activity of climbing stairs over the time period, walking, running, etc. The monitoring device may be worn on a wrist, carried by a user, worn on clothing (using a clip, or placed in a pocket), attached to a leg or foot, attached to the user's chest, waist, or integrated in an article of clothing such as a shirt, hat, pants, blouse, glasses, and the like. These examples are not limiting to all the possible ways the sensors of the device can be associated with a user or thing being monitored.

A biological sensor can determine any number of physiological characteristics of a user. As another example, the biological sensor may determine heart rate, a hydration level, body fat, bone density, fingerprint data, sweat rate, and/or a bio impedance of the user. Examples of the biological sensors include, without limitation, a biometric sensor, a physiological parameter sensor, a pedometer, or a combination thereof. A data associated with the user's activity can be monitored by the applications on the server and the user's device, and activity associated with the user's friends, acquaintances, or social network peers can also be shared, based on the user's authorization. This provides for the ability for friends to compete regarding their fitness, achieve goals, receive badges for achieving goals, get reminders for achieving such goals, rewards or discounts for achieving certain goals, etc.

As noted, an activity tracking device 100 can communicate with a computing device (e.g., a smartphone, a tablet computer, a desktop computer, or computer device having wireless communication access and/or access to the Internet). The computing device, in turn, can communicate over a network, such as the Internet or an Intranet to provide data synchronization. The network may be a wide area network, a local area network, or a combination thereof. The network may be coupled to one or more servers, one or more virtual machines, or a combination thereof. A server, a virtual machine, a controller of a monitoring device, or a controller of a computing device is sometimes referred to herein as a computing resource. Examples of a controller include a processor and a memory device.

The processor may be a general purpose processor. In another embodiment, the processor can be a customized processor configured to run specific algorithms or operations. Such processors can include digital signal processors (DSPs), which are designed to execute or interact with specific chips, signals, wires, and perform certain algorithms, processes, state diagrams, feedback, detection, execution, or the like. In some embodiments, a processor can include or be interfaced with an application specific integrated circuit (ASIC), a programmable logic device (PLD), a central processing unit (CPU), or a combination thereof, etc.

One or more chips, modules, devices, or logic can be defined to execute instructions or logic, which collectively can be viewed or characterized to be a processor. Therefore, it should be understood that a processor does not necessarily have to be one single chip or module, but can be defined from a collection of electronic or connecting components, logic, firmware, code, and combinations thereof.

Examples of a memory device include a random access memory (RAM) and a read-only memory (ROM). A memory device may be a Flash memory, a redundant array of disks (RAID), a hard disk, or a combination thereof. Embodiments described in the present disclosure may be practiced with various computer system configurations including hand-held devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers and the like. Several embodiments described in the present disclosure can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a wire-based or wireless network.

With the above embodiments in mind, it should be understood that a number of embodiments described in the present disclosure can employ various computer implemented operations involving data stored in computer systems. These operations are those requiring physical manipulation of physical quantities. Any of the operations described herein that form part of various embodiments described in the present disclosure are useful machine operations. Several embodiments described in the present disclosure also relate to a device or an apparatus for performing these operations. The apparatus can be specially constructed for a purpose, or the apparatus can be a computer selectively activated or configured by a computer program stored in the computer. In particular, various machines can be used with computer programs written in accordance with the teachings herein, or it may be more convenient to construct a more specialized apparatus to perform the required operations.

Various embodiments described in the present disclosure can also be embodied as computer-readable code on a non-transitory computer-readable medium. The computer readable medium is any data storage device that can store data, which can thereafter be read by a computer system. Examples of the computer-readable medium include hard drives, network attached storage (NAS), ROM, RAM, compact disc-ROMs (CD-ROMs), CD recordables (CD-Rs), CD-rewritables (RWs), magnetic tapes and other optical and non optical data storage devices. The computer-readable medium can include computer readable tangible medium distributed over a network-coupled computer system so that the computer-readable code is stored and executed in a distributed fashion.

Although the method operations were described in a specific order, it should be understood that other housekeeping operations may be performed in between operations, or operations may be performed in an order other than that shown, or operations may be adjusted so that they occur at slightly different times, or may be distributed in a system which allows the occurrence of the processing operations at various intervals associated with the processing.

WE CLAIMS 1) Our invention "PAM-Physical Fitness" is a process and computer programs are presented for creating a unified data stream from multiple data streams acquired from multiple devices. The Invented single method includes an operation for receiving activity data

streams from the devices, each activity data stream being associated with physical activity data of an any user. The invented the method also includes an operation for

assembling the unified activity data stream for a period of time. The unified activity

data stream includes data segments from the data streams of at least two devices, and the data segments are organized time-wise over the period of time. A server includes

a communications module, a memory, and a processor. The communications module is operable to receive a plurality of activity data streams from a plurality of devices and

each activity data stream being associated with physical activity data of a user. The invented method the memory is operable to store the plurality of activity data streams

and a unified activity data stream that includes data segments from the data streams of at least two devices of the plurality of devices. In addition to the processor is

operable to assemble the unified activity data stream for the user over a period of time.

2) According to claim# the invention is to a process and computer programs are presented for creating a unified data stream from multiple data streams acquired from

multiple devices.

3) According to claim,2# the invention is to the Invented single method includes an operation for receiving activity data streams from the devices, each activity data stream being associated with physical activity data of a any user.

4) According to claim,2,3# the invention is to the invented the method also includes an operation for assembling the unified activity data stream for a period of time.

) According to claim,2,4# the invention is to the unified activity data stream includes data segments from the data streams of at least two devices, and the data segments are organized time-wise over the period of time.

6) According to claim,2,4,5# the invention is to a server includes a communications module, a memory, and a processor. The communications module is operable to

receive a plurality of activity data streams from a plurality of devices and each activity data stream being associated with physical activity data of a user.

7) According to claim,2,6# the invention is to the invented method the memory is operable to store the plurality of activity data streams and a unified activity data stream that includes data segments from the data streams of at least two devices of the

plurality of devices. The processor is operable to assemble the unified activity data stream for the user over a period of time.

26/8/2020 Dr. Vineet Tirth (Associate Professor) Dr. Ram Karan Singh (Professor) Dr. Manisha Bhatkulkar (Assistant Professor) Dr. Neeraj Kumar Shukla (Associate Professor) Dr. M. Ramkumar Raja (Associate Professor) Dr. Shilpi Birla (Associate Professor) Prof.(Dr.) S. B. Chordiya (Director-SIMMC-Campus)

FOR Dr. Vineet Tirth (Associate Professor) Dr. Ram Karan Singh (Professor) Dr. Manisha Bhatkulkar (Assistant Professor) Dr. Neeraj Kumar Shukla (Associate Professor) Dr. M. Ramkumar Raja (Associate Professor) Dr. Shilpi Birla (Associate Professor) Prof.(Dr.) S. B. Chordiya (Director-SIMMC-Campus) 27 Aug 2020

TOTAL NO OF SHEET: 07 NO OF FIG: 09 2020102010

FIG. 1: IS A DIAGRAM DATA THAT COULD BE COLLECTED DURING A PERSON'S DAILY ROUTINE.

FOR Dr. Vineet Tirth (Associate Professor) Dr. Ram Karan Singh (Professor) Dr. Manisha Bhatkulkar (Assistant Professor) Dr. Neeraj Kumar Shukla (Associate Professor) Dr. M. Ramkumar Raja (Associate Professor) Dr. Shilpi Birla (Associate Professor) Prof.(Dr.) S. B. Chordiya (Director-SIMMC-Campus) 27 Aug 2020

TOTAL NO OF SHEET: 07 NO OF FIG: 09 2020102010

FIG. 2: IS A BLOCK DIAGRAM OF DATA STREAM SYNTHESIS.

FOR Dr. Vineet Tirth (Associate Professor) Dr. Ram Karan Singh (Professor) Dr. Manisha Bhatkulkar (Assistant Professor) Dr. Neeraj Kumar Shukla (Associate Professor) Dr. M. Ramkumar Raja (Associate Professor) Dr. Shilpi Birla (Associate Professor) Prof.(Dr.) S. B. Chordiya (Director-SIMMC-Campus) 27 Aug 2020

TOTAL NO OF SHEET: 07 NO OF FIG: 09 2020102010

FIG. 3A: IS A BLOCK DIAGRAM OF DATA STREAM SYNTHESIS.

FIG. 3B: IS AN EXEMPLARY ARCHITECTURE OF THE DATA SYNTHESIZER.

FOR Dr. Vineet Tirth (Associate Professor) Dr. Ram Karan Singh (Professor) Dr. Manisha Bhatkulkar (Assistant Professor) Dr. Neeraj Kumar Shukla (Associate Professor) Dr. M. Ramkumar Raja (Associate Professor) Dr. Shilpi Birla (Associate Professor) Prof.(Dr.) S. B. Chordiya (Director-SIMMC-Campus) 27 Aug 2020

TOTAL NO OF SHEET: 07 NO OF FIG: 09 2020102010

FIG. 3C: IS A FLOWCHART OF A METHOD FOR COMBINING MULTIPLE DATA STREAMS.

FIG. 4: IS A DIAGRAM OF DATA STREAM SYNTHESIS FROM MULTIPLE DEVICES.

FOR Dr. Vineet Tirth (Associate Professor) Dr. Ram Karan Singh (Professor) Dr. Manisha Bhatkulkar (Assistant Professor) Dr. Neeraj Kumar Shukla (Associate Professor) Dr. M. Ramkumar Raja (Associate Professor) Dr. Shilpi Birla (Associate Professor) Prof.(Dr.) S. B. Chordiya (Director-SIMMC-Campus) 27 Aug 2020

TOTAL NO OF SHEET: 07 NO OF FIG: 09 2020102010

FIG. 5: IS A SIMPLIFIED SCHEMATIC DIAGRAM OF A DEVICE FOR IMPLEMENTING.

FOR Dr. Vineet Tirth (Associate Professor) Dr. Ram Karan Singh (Professor) Dr. Manisha Bhatkulkar (Assistant Professor) Dr. Neeraj Kumar Shukla (Associate Professor) Dr. M. Ramkumar Raja (Associate Professor) Dr. Shilpi Birla (Associate Professor) Prof.(Dr.) S. B. Chordiya (Director-SIMMC-Campus) 27 Aug 2020

TOTAL NO OF SHEET: 07 NO OF FIG: 09 2020102010

FIG. 6: IS A FLOWCHART AN ALGORITHM FOR PERFORMING DATA SYNTHESIS FOR DATA PROVIDED BY A PLURALITY OF DEVICES.

FOR Dr. Vineet Tirth (Associate Professor) Dr. Ram Karan Singh (Professor) Dr. Manisha Bhatkulkar (Assistant Professor) Dr. Neeraj Kumar Shukla (Associate Professor) Dr. M. Ramkumar Raja (Associate Professor) Dr. Shilpi Birla (Associate Professor) Prof.(Dr.) S. B. Chordiya (Director-SIMMC-Campus) 27 Aug 2020

TOTAL NO OF SHEET: 07 NO OF FIG: 09 2020102010

FIG. 7: IS AN EXAMPLE WHERE VARIOUS TYPES OF ACTIVITIES OF USERS CAN BE CAPTURED OR COLLECTED BY ACTIVITY TRACKING DEVICES.