US20200000379A1 - Device and method for classifying the activity and/or counting steps of a user - Google Patents

Device and method for classifying the activity and/or counting steps of a user Download PDF

Info

Publication number
US20200000379A1
US20200000379A1 US16/558,739 US201916558739A US2020000379A1 US 20200000379 A1 US20200000379 A1 US 20200000379A1 US 201916558739 A US201916558739 A US 201916558739A US 2020000379 A1 US2020000379 A1 US 2020000379A1
Authority
US
United States
Prior art keywords
window
peak
processing window
current
accelerometer data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US16/558,739
Inventor
Khine Cho Cho THEIN
Kittipong Kasamsook
Aye AUNG
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nitto Denko Corp
Original Assignee
Nitto Denko Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nitto Denko Corp filed Critical Nitto Denko Corp
Priority to US16/558,739 priority Critical patent/US20200000379A1/en
Assigned to NITTO DENKO CORPORATION reassignment NITTO DENKO CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: AUNG, Aye, KASAMSOOK, Kittipong, THEIN, Khine Cho Cho
Publication of US20200000379A1 publication Critical patent/US20200000379A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/103Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1123Discriminating type of movement, e.g. walking or running
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/103Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1118Determining activity level
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7203Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
    • A61B5/7207Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal of noise induced by motion artifacts
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7221Determining signal validity, reliability or quality
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01CMEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C22/00Measuring distance traversed on the ground by vehicles, persons, animals or other moving solid bodies, e.g. using odometers, using pedometers
    • G01C22/006Pedometers
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01PMEASURING LINEAR OR ANGULAR SPEED, ACCELERATION, DECELERATION, OR SHOCK; INDICATING PRESENCE, ABSENCE, OR DIRECTION, OF MOVEMENT
    • G01P15/00Measuring acceleration; Measuring deceleration; Measuring shock, i.e. sudden change of acceleration
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2562/00Details of sensors; Constructional details of sensor housings or probes; Accessories for sensors
    • A61B2562/02Details of sensors specially adapted for in-vivo measurements
    • A61B2562/0219Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/103Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B24/00Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
    • A63B24/0003Analysing the course of a movement or motion sequences during an exercise or trainings sequence, e.g. swing for golf or tennis

Definitions

  • the present invention relates broadly to device and method for classifying the activity and/or counting steps of a user.
  • step counting devices that use accelerometer(s) typically require the user to first position the device in a limited set of orientations. For example, the user has to secure the device on their body such that a dedicated axis (i.e. a direction of the device) is substantially and continuously aligned with the direction of the gravitational force throughout the measurement period.
  • a dedicated axis i.e. a direction of the device
  • Embodiments of the present invention provide a system and method for activity monitoring that seek to address at least one of the above problems.
  • a device for classifying the activity of a user wearing the device comprising an accelerometer for measuring accelerometer data for a plurality of axes; and a processor configured to:
  • a device for counting the number of steps taken by a user wearing the device comprising an accelerometer for measuring accelerometer data for at least one axis; and a processor configured to:
  • a method for classifying the activity of a user comprising measuring accelerometer data for a plurality of axes; identifying a most active one of the plurality of axes based on the accelerometer data; and classifying the activity of the user based on a signal amplitude of the accelerometer data for the most active axis and one or more threshold values.
  • a method for counting the number of steps taken by a user comprising measuring accelerometer data for at least one axis; applying a derivative operator to the accelerometer data in successive processing windows; counting peaks in the derivative of the accelerometer data in each processing window; and eliminating over counted peaks based on time difference between a first peak in a current window and a last peak in a preceding processing window.
  • FIG. 1 Shows the process of activity classification according to an example embodiment.
  • FIG. 2 Shows a spike in the raw accelerometer data.
  • FIG. 3 Shows the accelerometer data of FIG. 2 with the attenuated spike after applying a smoothing process according to an example embodiment.
  • FIG. 4 Shows example signal amplitudes and determined characteristics of an accelerometer signal according to an example embodiment.
  • FIG. 5 Shows the process of classifying activity using threshold levels according to an example embodiment.
  • FIG. 6 Shows the signal amplitude measured during different activities according to an example embodiment.
  • FIG. 7 Shows the process of peak detection and step count according to an example embodiment.
  • FIG. 8 Shows the signal waveform of a wrist-worn device before and after applying the Pan Tompkins derivation operator according to an example embodiment.
  • FIG. 9 Shows the adaptive threshold value (A) used for processing of subsequent window(s) according to an example embodiment.
  • FIG. 10 Shows peak to peak distance between the peaks for two consecutive windows according to an example embodiment.
  • FIG. 11 Shows the process flow of activity classification and step count are being performed according to an example embodiment.
  • FIG. 12 Shows schematic drawing of an assembly comprising a wearable device and a communication device, according to an example embodiment.
  • FIG. 13 Shows a block diagram of an assembly comprising a wearable device and a communication device, according to an example embodiment.
  • FIG. 14 Shows a block diagram of a device for classifying the activity of a user according to an example embodiment.
  • FIG. 15 Shows a block diagram of a device for counting steps of a user according to an example embodiment.
  • FIG. 16 Shows a flow chart illustrating a method for classifying the activity of a user according to an example embodiment.
  • FIG. 17 Shows a flow chart illustrating a method for counting steps of a user according to an example embodiment.
  • Embodiments of the present invention relate to a wearable device and method for classifying the activity of the user and a wearable device and method of counting the number of steps taken by the user.
  • the device is worn on the wrist of the user and is configured to obtain the user's acceleration information for processing to classify the activity of the user and/or to count the number of steps taken by the user.
  • the present specification also discloses an apparatus, which may be internal and/or external to the wearable device in example embodiments, for performing the operations of the methods.
  • Such apparatus may be specially constructed for the required purposes, or may comprise a general purpose computer or other device selectively activated or reconfigured by a computer program stored in the computer.
  • the algorithms and displays presented herein are not inherently related to any particular computer or other apparatus.
  • Various general purpose machines may be used with programs in accordance with the teachings herein. Alternatively, the construction of more specialized apparatus to perform the required method steps may be appropriate.
  • the structure of a conventional general purpose computer will appear from the description below.
  • the present specification also implicitly discloses a computer program, in that it would be apparent to the person skilled in the art that the individual steps of the method described herein may be put into effect by computer code.
  • the computer program is not intended to be limited to any particular programming language and implementation thereof. It will be appreciated that a variety of programming languages and coding thereof may be used to implement the teachings of the disclosure contained herein.
  • the computer program is not intended to be limited to any particular control flow. There are many other variants of the computer program, which can use different control flows without departing from the spirit or scope of the invention.
  • Such a computer program may be stored on any computer readable medium.
  • the computer readable medium may include storage devices such as magnetic or optical disks, memory chips, or other storage devices suitable for interfacing with a general purpose computer.
  • the computer readable medium may also include a hard-wired medium such as exemplified in the Internet system, or wireless medium such as exemplified in the GSM mobile telephone system.
  • the computer program when loaded and executed on such a general-purpose computer effectively results in an apparatus that implements the steps of the preferred method.
  • the invention may also be implemented as hardware modules. More particular, in the hardware sense, a module is a functional hardware unit designed for use with other components or modules. For example, a module may be implemented using discrete electronic components, or it can form a portion of an entire electronic circuit such as an Application Specific Integrated Circuit (ASIC). Numerous other possibilities exist. Those skilled in the art will appreciate that the system can also be implemented as a combination of hardware and software modules.
  • ASIC Application Specific Integrated Circuit
  • FIG. 1 shows a flowchart 100 illustrating a method for classifying the activity of a user according to an example embodiment, the method comprising:
  • step 102 obtaining the acceleration signals of a user through the wearable device
  • step 104 splitting the acceleration signals data into a plurality of processing windows
  • step 106 smoothing the data with a moving average function
  • the accelerometer data is converted into g value data
  • step 110 identifying the most active axis of the acceleration signals
  • step 112 classifying the activity of the user based on the acceleration signals from the most active axis
  • the size of a window is herein described as the time duration of the window, by way of example only. It was found that accuracy suitable for classifying activities and/or counting of steps can for example, but without limitation, be achieved by utilizing window sizes between about >0 to 5 seconds depending on specific accuracy requirement. Generally, a longer window size can lower the chance of an over-counted peak in a window transition area and may thus increase the accuracy. A shorter window may better capture the transition between different activities, which can happen within a very short period of time.
  • Raw accelerometer data are typically noisy.
  • a stray spike 200 in example accelerometer data 202 is shown in FIG. 2 .
  • the stray spike 200 may result in the wrong axis being identified as the most active axis for the subsequent processing.
  • smoothing is advantageously applied to the raw accelerometer data 202 .
  • smoothing is achieved by applying a moving average operation to the raw accelerometer data 200 .
  • FIG. 3 shows the accelerometer data 202 * after smoothing according to one example embodiment, with an attenuated spike 200 * that preferably does not affect the identification of the most active axis for the subsequent processing.
  • the most active axis in example embodiments is identified as the accelerometer axis with the greatest amplitude signal, representative of the contributory effect of the gravitational force and the impact force of the wearer's foot on the ground.
  • the most active axis in example embodiments the accelerometer axis that is closest to the vertical direction, i.e. substantially perpendicular to the earth.
  • the most active axis can be easily identified over the other axes regardless of the position or orientation in which the device is being used or worn by the user.
  • the most active axis in a preferred embodiment is selected by comparing the smoothed x, y and z-axis (i.e. the three accelerometer axes irrespective of actual orientation relative to the ground) data in each window, referred to herein as the moving average.
  • the absolute value of each axis from each window is determined, referred to herein as the absolute moving average.
  • the mean value of each axis based on the absolute value is then determined, referred to herein as the mean absolute moving average. From a comparison of the three mean values, the axis with the largest mean value is identified as the most active axis for each window, in a preferred embodiment.
  • the user's activity is classified into light activity, moderate activity or heavy activity state using the moving average data from the most active axis in example embodiments.
  • the signal amplitude of the most active axis is calculated by subtracting the mean 400 or minimum (min) g-value 402 from the maximum (max) g-value 404 of most active axis data 406 in, for example, a window 408 with a size between about >0 to 5 s, as shown in FIG. 4 .
  • FIG. 5 shows a flowchart 500 illustrating the process of classifying activity using threshold levels, according to an example embodiment.
  • the device is wrist-worn by the user
  • experimental results has shown that, for example, the threshold levels of 0.1 g and 1 g can be used for activity classification.
  • the signal amplitude (SA) of the most active axis is determined.
  • SA signal amplitude
  • step 504 if SA is smaller than a threshold 1, for example SA ⁇ 0.1 g, classification into the light activity state is performed, see step 506 .
  • step 508 if SA is smaller than a threshold 2, for example SA ⁇ 1 g, classification into the moderate activity state is performed, see step 510 .
  • the classification is into the moderate activity state.
  • SA ⁇ the threshold 2 for example SA ⁇ 1 g
  • classification into the heavy activity state is performed, see step 512 .
  • Light activity Stationary motions such as standing still, waiting for traffic lights to turn green or typing (e.g. on desk),
  • plots 600 and 602 show the SA for the most active axis calculated based on the maximum value minus the minimum value in the processing window, and calculated based on the maximum value minus the mean value in the processing window, respectively.
  • basing the SA on the maximum value minus the mean value in the processing window can be preferred for classification based on thresholds, as it results in a smoother plot with reduced absolute values and narrower bands of values in the respective activity regions,
  • FIG. 7 shows a flow chart 700 illustrating a process for counting steps according to an example embodiment.
  • the accelerometer data is captured.
  • the data is split into process windows.
  • low pass filtering is applied to the data from the most active axis based on classified activity.
  • a derivative of the filtered data using Pan Tompkins derivative operator is computed.
  • the output of the derivative signal is normalized.
  • peak detection is performed for step counting.
  • an adaptive threshold value is determined based on the current process window and updated to be used for the next process window.
  • over counted peaks are eliminated.
  • the cut-off frequency of the low pass filter is preferably adjusted based on the activity type to remove the correct frequency range of motion noise. Choosing a correct cut-off frequency can advantageously improve the signal geometry for avoiding false peaks or undetected peaks during the subsequent peak detection processing.
  • the activity type may be input by the user, or may be identified using the activity classification method and device described above, or by any other method or device.
  • y ( n ) (1 ⁇ 8)[2 x ( n )+ x ( n ⁇ 1) ⁇ x ( n ⁇ 3) ⁇ 2 x ( n ⁇ 4)] Equation (1).
  • n denotes the data point, and n>4
  • Pan Tompkins Derivative operator in equation (1) is used in a preferred embodiment to further improve the accuracy of peak detection by advantageously enlarging the slope information and optimizing the signal condition for peak detection.
  • Acceleration changes can also depend on the change in hand swing and impact force of feet.
  • the impact force is typically not strong enough and most of the acceleration changes in e.g. a wrist-worn device are due to the hand swing.
  • the inconsistent peaks and troughs of the signal waveform can make it difficult to implement an accurate peak detection algorithm, and in a preferred embodiment, the Pan Tompkins derivation operator results in a more consistent peaks and troughs pattern.
  • FIGS. 8 a ) and b ) show the signal waveform of a wrist-worn device before (curve 800 ) and after (curve 802 ) applying Pan Tompkins derivation operator according to an example embodiment, respectively.
  • the normalization process in example embodiments is preferred such that the derivative signals are scaled correctly for the peak detection processing.
  • the signal is normalized by first subtracting the minimum value of the signal from the signal and then dividing by the maximum value of the subtracted signal.
  • a peak will be detected (i.e. a step counted), when the normalized derivative signal exceeds a threshold level, according to example embodiments.
  • a threshold value ⁇ l for a first processing window can e.g. be set to any pre-determined value between >0 and 0.5.
  • Step 714 Determining and Updating Adaptive Threshold (Step 714 , FIG. 7 )
  • an adaptive threshold value ( ⁇ w ) will be determined and used for subsequent windows, which can be set at any pre-determined value between about >0% and 50% of the difference between the last peak and valley values of the current window.
  • ⁇ w an adaptive threshold value
  • a w adaptive threshold value of a next window w
  • a last peak value in a current window (w ⁇ 1)
  • b last valley value in the said current window (w ⁇ 1), and w ⁇ 2.
  • Pan Tompkins derivative operation can have a tendency to generate a false first peak in the subsequent windows which can result in over counting of peaks, i.e. steps.
  • FIG. 10 illustrates the method of advantageously identifying such false peaks to be removed from the count in a preferred embodiment.
  • the false peaks are identified by checking the distance d between a first peak 1000 of the current measured window 1002 and the last peak 1004 of the previous window 1006 .
  • a threshold may be set at 4 steps per second in one embodiment. For example, in the data point domain of a sampling rate of 80 Hz, the distance threshold may be set at 20 data points based on 4 steps per second. If the peak to peak distance d is smaller than 20 data points, the current peak 1200 is identified as false and is removed from the step count.
  • FIG. 11 shows a flow chart illustrating the process flow 1100 of one embodiment, in which both the activity classification and the activity based step counting techniques are integrated into a single wrist wearable device.
  • acceleration data of the user is captured by the device through a 3-axis accelerometer.
  • the accelerometer signal is split into processing windows between about >0 to 5 seconds.
  • the signal in each window is smoothed by performing a moving average function.
  • the most active axis is identified by comparing the mean absolute moving average value of each axis. The axis with the largest value will be the most active axis.
  • the activity in each window is classified by calculating the signal amplitude (max-mean) of the moving average signal of the most active axis. Classification will be based on 0.1 g and 1 g thresholds in one non-limiting example. Once a low activity state is identified (step 1110 ), the process will move on to the next window (step 1112 ), as no steps are required to be counted.
  • step 1114 For moderate and heavy activity states (step 1114 ), the step counting is performed. At step 1116 and 1118 , the cut-off frequency of the low pass filter for each window will be updated based on the activity identified, and the low pass filtering applied accordingly.
  • the Pan Tompkins derivative operator is then applied.
  • the derivative signal is then normalized to adjust the signal amplitude within, for example, 0 to 1 in each window (first subtracting the minimum value of the signal from the signal and then dividing by the maximum value of the subtracted signal in one non-limiting example).
  • step 1124 peak detection is then performed where each peak detected contributes to a step count.
  • the threshold In the first window after transiting to a moderate or heavy activity state from a light activity state, the threshold will be set between >0 to 0.5 in one non-limiting example. In other words, a first window transiting from a light activity state does not typically require step elimination. For subsequent windows (be it being a moderate or heavy activity state), over counted peak detection and elimination is advantageously performed until a light activity state is detected for the window. In the subsequent windows, an adaptive threshold is used and will be set between about >0 to 50% of the height difference between the last peak and valley of the previous window, in one non-limiting example, step 1126 .
  • false peaks originating from the Pan Tompkins operation are then removed using e.g. a peak to peak distance (current window first peak vs. previous window last peak) threshold.
  • the threshold in one non-limiting example is based on 4 steps per second. For example, in the data point domain of a sampling rate of 80 Hz, the distance threshold is set at 20 data point. If the peak to peak distance is smaller than 20 data points, the current peak is identified as false and is removed from the step count.
  • FIG. 12 shows an assembly 1200 comprising a wearable device in the form of a wrist watch 1201 according to an example embodiment. It will be appreciated that in different embodiments the device may also be in any other form suitable to be worn on any part of the user's body such as his/her arms, waist, hip or foot.
  • the wrist watch 1201 classifies the user's activity and/or computes the number of steps taken by the user and communicates the result(s) wirelessly to a telecommunication device of the assembly 1200 such as a mobile phone 1202 or other portable electronic devices, or computing devices such as desk top computers, laptop computer, tab computers etc.
  • FIG. 13 shows a schematic block diagram of an assembly 1300 comprising a wearable device 1301 according to an example embodiment, for classifying activity and/or counting steps of the user.
  • the device 1301 includes a signal sensing module 1302 , such as an accelerometer or gyroscope, for obtaining the acceleration information of the user.
  • a signal sensing module 1302 such as an accelerometer or gyroscope
  • a preferred accelerometer that can be adapted for use in the device is a triple-axis accelerometer MMA8652FC available from Freescale Semiconductor, Inc. This accelerometer can provide the advantage of measuring acceleration in all three directions with a single package. Alternatively, several single-axis accelerometers oriented to provide three-axis sensing can be used in different embodiments.
  • the device 1301 also includes a data processing and computational module 1304 , such as a processor, which is arranged to receive and process the acceleration information from the signal sensing module 1302 .
  • the device 1301 also includes a display unit 1306 for displaying a result to a user of the device 1301 .
  • the device 1301 in this embodiment further includes a wireless transmission module 1308 arranged to communicate wirelessly with a telecommunications device 1310 of the assembly 1300 .
  • the telecommunication device 1310 includes a wireless receiver module 1312 for receiving signals from the wearable device 1301 and a display unit 1314 for displaying a result to a user of the telecommunication device 1310 .
  • FIG. 14 shows a block diagram of a device 1400 for classifying the activity of a user wearing the device, according to one embodiment.
  • the device 1400 comprises an accelerometer 1402 for measuring accelerometer data for a plurality of axes; and a processor 1404 configured to identify a most active one of the plurality of axes based on the accelerometer data and to classify the activity of the user based on a signal amplitude of the accelerometer data for the most active axis and one or more threshold values.
  • the processor 1404 may be configured to apply a smoothing to the accelerometer data prior to identifying the most active axis.
  • the processor 1404 may be configured to identify the most active axis by determining the axis having the largest mean absolute amplitude of the smoothed signal, which may also be referred to as the largest mean absolute moving average signal value.
  • the processor 1404 may be configured to classify the activity based on a maximum signal amplitude of the accelerometer data for the most active axis.
  • the processor 1404 may be configured to classify the activity based on the maximum signal amplitude of the accelerometer data minus the mean signal amplitude for the most active axis.
  • the processor 1404 may be configured to identify the most active axis and to classify the activity of the user based on accelerometer data measured over a predetermined processing window.
  • the processing window may be in a range from about >0 to 5 seconds.
  • FIG. 15 shows a block diagram of a device 1500 for counting the number of steps taken by a user wearing the device, according to one embodiment.
  • the device 1500 comprises an accelerometer 1502 for measuring accelerometer data for at least one axis; and a processor 1504 configured to apply a derivative operator to the accelerometer data in successive processing windows, to count peaks in the derivative of the accelerometer data in each processing window, and to eliminate over counted peaks based on time difference between a first peak in a current window and a last peak in a preceding processing window.
  • the processor 1504 may be configured to count the peaks based on a first threshold value if the current processing window is a first processing window after transiting from a light activity state to a moderate or a heavy activity state.
  • the processor may further be configured to update an adaptive threshold for the next window.
  • the adaptive threshold for the next window may be based on the difference in signal amplitude for a last successive peak and valley pair in the current processing window.
  • the adaptive threshold for the next window may be between about >0 to 50% of the difference in signal amplitude for the last successive peak and valley pair in the current processing window.
  • the processor 1504 may be configured to eliminate peaks from the peak count in the current window if the time difference between the first peak in the current window and the last peak in the preceding processing window is smaller than a second threshold value.
  • the second threshold value may be about 1 ⁇ 4 seconds.
  • the derivative operator may comprise a Pan Tomkins derivative operator.
  • the devices 1400 , 1500 may be implemented in a wearable device.
  • the devices 1400 , 1500 may be implemented in an assembly comprising a wearable device and a communication device.
  • the devices 1400 , 1500 may be implemented in an assembly comprising a wearable device and a wireless communication device.
  • FIG. 16 shows a flow chart 1600 illustrating a method for classifying the activity of a user, according to an example embodiment.
  • accelerometer data is measured for a plurality of axes.
  • a most active one of the plurality of axes is identified based on the accelerometer data.
  • the activity of the user is classified based on a signal amplitude of the accelerometer data for the most active axis and one or more threshold values.
  • the method may comprise applying a smoothing to the accelerometer data prior to identifying the most active axis.
  • the method may comprise identifying the most active axis by determining the axis having the largest mean absolute value of the smoothed signal, which may also be referred to as the largest mean absolute moving average signal value.
  • the method may comprise classifying the activity based on a maximum signal amplitude of the accelerometer data for the most active axis.
  • the method may comprise classifying the activity based on the maximum signal amplitude of the accelerometer data minus the mean signal amplitude for the most active axis.
  • the method may comprise identifying the most active axis and classifying the activity of the user based on accelerometer data measured over a predetermined processing window.
  • the processing window may be in a range from about >0 to 5 seconds.
  • FIG. 17 shows a flow chart 1700 illustrating a method for counting the number of steps taken by a user.
  • accelerometer data is measured for at least one axis.
  • a derivative operator is applied to the accelerometer data in successive processing windows.
  • peaks in the derivative of the accelerometer data in each processing window are counted.
  • over counted peaks are eliminated based on time difference between a first peak in a current window and a last peak in a preceding processing window.
  • the method may comprise counting the peaks based on a first threshold if the current processing window is a first processing window after transiting from a light activity state to a moderate or a heavy activity state.
  • the method may further comprise updating an adaptive threshold for the next window.
  • the adaptive threshold for the next window may be based on the difference in signal amplitude for a last successive peak and valley pair in the current processing window.
  • the adaptive threshold for the next window may be between about >0 to 50% of the difference in signal amplitude for the last successive peak and valley pair in the current processing window.
  • the method may comprise eliminating peaks from the peak count in the current processing window if the time difference between the first peak in the current processing window and the last peak in the preceding processing window is smaller than a second threshold value.
  • the second threshold value may be about 1 ⁇ 4 seconds.
  • the derivative operator may comprise a Pan Tomkins derivative operator.
  • Example embodiments of the present invention advantageously do not require the device to be attached at a fixed orientation throughout the monitoring period. This is because the device in example embodiments constantly detects within processing windows for the most active axis (which will be the one closest to the direction of the impact force), to be used for subsequent processes.
  • the most active axis can be identified in example embodiments by looking for the axis having the largest mean absolute moving average acceleration signal within each window.
  • Embodiments of the present invention can advantageously be able to detect changes and differentiate the signals corresponding to light, moderate or heavy states by splitting the accelerometer data into process windows and observing the signal amplitude (e.g. max-mean or max-min) against threshold levels (e.g. 0.1 g and 1 g) within each window.
  • the signal amplitude e.g. max-mean or max-min
  • threshold levels e.g. 0.1 g and 1 g
  • motion noise can advantageously be correctly removed using low pass filters with suitable cut-off frequencies depending on the activity.
  • a Pan Tompkins derivative operation together with a peak elimination algorithm based on peak distance and adaptive thresholds based on max-min value of a peak advantageously helps the device and method to accurately identify peaks as step counts.
  • the device may be worn on the arms, hip, waist or foot of the user.
  • the device and method may perform only one of the activity classification and the step counting as described above, or both at the same time.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Veterinary Medicine (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Physiology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Public Health (AREA)
  • Signal Processing (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Dentistry (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Psychiatry (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

Device and method for classifying the activity and/or counting steps of a user. A method for classifying the activity of a user can comprise measuring accelerometer data for a plurality of axes; identifying a most active one of the plurality of axes based on the accelerometer data; and classifying the activity of the user based on a signal amplitude of the accelerometer data for the most active axis and one or more threshold values.

Description

    CLAIM FOR PRIORITY
  • This application is a divisional of U.S. patent application Ser. No. 15/315,090, filed Nov. 30, 2016, which claims priority to PCT/S G2014/000248, filed 30 May 2014.
  • FIELD OF INVENTION
  • The present invention relates broadly to device and method for classifying the activity and/or counting steps of a user.
  • BACKGROUND
  • In recent years, people are becoming more health conscious and have taken up running and walking as a convenient and effective way to exercise. There is also an increased interest in self-tracking of daily activities through wearable sensors. Hence there is need for a step counting device that can accurately record a wearer's gait status as well as the number of steps taken.
  • Conventional step counting devices that use accelerometer(s) typically require the user to first position the device in a limited set of orientations. For example, the user has to secure the device on their body such that a dedicated axis (i.e. a direction of the device) is substantially and continuously aligned with the direction of the gravitational force throughout the measurement period.
  • Existing devices are typically unable to detect changes or differentiate signals that correspond to different gaits of the user, resulting in an incorrect count of the steps taken. Motion noises experienced by the devices will also cause false steps to be measured and actual steps to be missed in existing step counting devices.
  • Also, as the forward and backward swing of a body part to which the device is attached is not same for every step, existing devices are typically unable to correctly identify peaks as a step count due to the inconsistencies in the signal peaks and troughs.
  • Embodiments of the present invention provide a system and method for activity monitoring that seek to address at least one of the above problems.
  • SUMMARY
  • In accordance with a first aspect of the present invention there is provided a device for classifying the activity of a user wearing the device, the device comprising an accelerometer for measuring accelerometer data for a plurality of axes; and a processor configured to:
  • identify a most active one of the plurality of axes based on the accelerometer data; and
  • classify the activity of the user based on a signal amplitude of the accelerometer data for the most active axis and one or more threshold values.
  • In accordance with a second aspect of the present invention there is provided a device for counting the number of steps taken by a user wearing the device, the device comprising an accelerometer for measuring accelerometer data for at least one axis; and a processor configured to:
  • apply a derivative operator to the accelerometer data in successive processing windows;
  • count peaks in the derivative of the accelerometer data in each processing window; and
  • eliminate over counted peaks based on time difference between a first peak in a current window and a last peak in a preceding processing window.
  • In accordance with a third aspect of the present invention there is provided a method for classifying the activity of a user, the method comprising measuring accelerometer data for a plurality of axes; identifying a most active one of the plurality of axes based on the accelerometer data; and classifying the activity of the user based on a signal amplitude of the accelerometer data for the most active axis and one or more threshold values.
  • In accordance with a fourth aspect of the present invention there is provided a method for counting the number of steps taken by a user, the method comprising measuring accelerometer data for at least one axis; applying a derivative operator to the accelerometer data in successive processing windows; counting peaks in the derivative of the accelerometer data in each processing window; and eliminating over counted peaks based on time difference between a first peak in a current window and a last peak in a preceding processing window.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • Embodiments of the invention will be better understood and readily apparent to one of ordinary skill in the art from the following written description, by way of example only, and in conjunction with the drawings, in which:
  • FIG. 1—Shows the process of activity classification according to an example embodiment.
  • FIG. 2—Shows a spike in the raw accelerometer data.
  • FIG. 3—Shows the accelerometer data of FIG. 2 with the attenuated spike after applying a smoothing process according to an example embodiment.
  • FIG. 4—Shows example signal amplitudes and determined characteristics of an accelerometer signal according to an example embodiment.
  • FIG. 5—Shows the process of classifying activity using threshold levels according to an example embodiment.
  • FIG. 6—Shows the signal amplitude measured during different activities according to an example embodiment.
  • FIG. 7—Shows the process of peak detection and step count according to an example embodiment.
  • FIG. 8—Shows the signal waveform of a wrist-worn device before and after applying the Pan Tompkins derivation operator according to an example embodiment.
  • FIG. 9—Shows the adaptive threshold value (A) used for processing of subsequent window(s) according to an example embodiment.
  • FIG. 10—Shows peak to peak distance between the peaks for two consecutive windows according to an example embodiment.
  • FIG. 11—Shows the process flow of activity classification and step count are being performed according to an example embodiment.
  • FIG. 12—Shows schematic drawing of an assembly comprising a wearable device and a communication device, according to an example embodiment.
  • FIG. 13—Shows a block diagram of an assembly comprising a wearable device and a communication device, according to an example embodiment.
  • FIG. 14—Shows a block diagram of a device for classifying the activity of a user according to an example embodiment.
  • FIG. 15—Shows a block diagram of a device for counting steps of a user according to an example embodiment.
  • FIG. 16—Shows a flow chart illustrating a method for classifying the activity of a user according to an example embodiment.
  • FIG. 17—Shows a flow chart illustrating a method for counting steps of a user according to an example embodiment.
  • DETAILED DESCRIPTION
  • Embodiments of the present invention relate to a wearable device and method for classifying the activity of the user and a wearable device and method of counting the number of steps taken by the user. In one non-limiting example embodiment, the device is worn on the wrist of the user and is configured to obtain the user's acceleration information for processing to classify the activity of the user and/or to count the number of steps taken by the user.
  • The present specification also discloses an apparatus, which may be internal and/or external to the wearable device in example embodiments, for performing the operations of the methods. Such apparatus may be specially constructed for the required purposes, or may comprise a general purpose computer or other device selectively activated or reconfigured by a computer program stored in the computer. The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Various general purpose machines may be used with programs in accordance with the teachings herein. Alternatively, the construction of more specialized apparatus to perform the required method steps may be appropriate. The structure of a conventional general purpose computer will appear from the description below. In addition, the present specification also implicitly discloses a computer program, in that it would be apparent to the person skilled in the art that the individual steps of the method described herein may be put into effect by computer code. The computer program is not intended to be limited to any particular programming language and implementation thereof. It will be appreciated that a variety of programming languages and coding thereof may be used to implement the teachings of the disclosure contained herein. Moreover, the computer program is not intended to be limited to any particular control flow. There are many other variants of the computer program, which can use different control flows without departing from the spirit or scope of the invention.
  • Furthermore, one or more of the steps of the computer program may be performed in parallel rather than sequentially. Such a computer program may be stored on any computer readable medium. The computer readable medium may include storage devices such as magnetic or optical disks, memory chips, or other storage devices suitable for interfacing with a general purpose computer. The computer readable medium may also include a hard-wired medium such as exemplified in the Internet system, or wireless medium such as exemplified in the GSM mobile telephone system. The computer program when loaded and executed on such a general-purpose computer effectively results in an apparatus that implements the steps of the preferred method.
  • The invention may also be implemented as hardware modules. More particular, in the hardware sense, a module is a functional hardware unit designed for use with other components or modules. For example, a module may be implemented using discrete electronic components, or it can form a portion of an entire electronic circuit such as an Application Specific Integrated Circuit (ASIC). Numerous other possibilities exist. Those skilled in the art will appreciate that the system can also be implemented as a combination of hardware and software modules.
  • FIG. 1 shows a flowchart 100 illustrating a method for classifying the activity of a user according to an example embodiment, the method comprising:
  • At step 102, obtaining the acceleration signals of a user through the wearable device,
  • at step 104, splitting the acceleration signals data into a plurality of processing windows;
  • and in each processing window,
  • at step 106, smoothing the data with a moving average function,
  • at step 108, the accelerometer data is converted into g value data,
  • at step 110, identifying the most active axis of the acceleration signals,
  • at step 112, classifying the activity of the user based on the acceleration signals from the most active axis,
  • Details of example features that may be used separately and/or in combination with one or more of the other example features will be described next.
  • Processing Windows
  • The size of a window is herein described as the time duration of the window, by way of example only. It was found that accuracy suitable for classifying activities and/or counting of steps can for example, but without limitation, be achieved by utilizing window sizes between about >0 to 5 seconds depending on specific accuracy requirement. Generally, a longer window size can lower the chance of an over-counted peak in a window transition area and may thus increase the accuracy. A shorter window may better capture the transition between different activities, which can happen within a very short period of time.
  • Smoothing of Accelerometer Data
  • Raw accelerometer data are typically noisy. A stray spike 200 in example accelerometer data 202 is shown in FIG. 2. The stray spike 200 may result in the wrong axis being identified as the most active axis for the subsequent processing. In example embodiments, smoothing is advantageously applied to the raw accelerometer data 202. Preferably, smoothing is achieved by applying a moving average operation to the raw accelerometer data 200.
  • FIG. 3 shows the accelerometer data 202* after smoothing according to one example embodiment, with an attenuated spike 200* that preferably does not affect the identification of the most active axis for the subsequent processing.
  • Most Active Axis
  • The most active axis in example embodiments is identified as the accelerometer axis with the greatest amplitude signal, representative of the contributory effect of the gravitational force and the impact force of the wearer's foot on the ground. In other word, the most active axis in example embodiments the accelerometer axis that is closest to the vertical direction, i.e. substantially perpendicular to the earth.
  • Advantageously, the most active axis can be easily identified over the other axes regardless of the position or orientation in which the device is being used or worn by the user.
  • The most active axis in a preferred embodiment is selected by comparing the smoothed x, y and z-axis (i.e. the three accelerometer axes irrespective of actual orientation relative to the ground) data in each window, referred to herein as the moving average. The absolute value of each axis from each window is determined, referred to herein as the absolute moving average. The mean value of each axis based on the absolute value is then determined, referred to herein as the mean absolute moving average. From a comparison of the three mean values, the axis with the largest mean value is identified as the most active axis for each window, in a preferred embodiment.
  • Activity Classification
  • The user's activity is classified into light activity, moderate activity or heavy activity state using the moving average data from the most active axis in example embodiments.
  • In one embodiment, the signal amplitude of the most active axis is calculated by subtracting the mean 400 or minimum (min) g-value 402 from the maximum (max) g-value 404 of most active axis data 406 in, for example, a window 408 with a size between about >0 to 5 s, as shown in FIG. 4.
  • FIG. 5 shows a flowchart 500 illustrating the process of classifying activity using threshold levels, according to an example embodiment. In the example case that the device is wrist-worn by the user, experimental results (see FIG. 6) has shown that, for example, the threshold levels of 0.1 g and 1 g can be used for activity classification.
  • At step 502, the signal amplitude (SA) of the most active axis is determined. In step 504, if SA is smaller than a threshold 1, for example SA<0.1 g, classification into the light activity state is performed, see step 506. In step 508, if SA is smaller than a threshold 2, for example SA<1 g, classification into the moderate activity state is performed, see step 510. In other words, in one example for 0.1 g≤SA<1 g, the classification is into the moderate activity state. If SA≥the threshold 2, for example SA≥1 g, classification into the heavy activity state is performed, see step 512.
  • Examples for the Classifications Include:
  • Light activity—Stationary motions such as standing still, waiting for traffic lights to turn green or typing (e.g. on desk),
  • Moderate activity—Slow walking, brisk walk
  • Heavy activity—Jogging, running
  • Most other activities e.g. climbing stairs, cycling, swimming, playing football, which involve the movement of the body can also be classifiable in example embodiment, e.g. based on:
  • Further splitting into multiple SA thresholds (on top of the thresholds 1 & 2 in the example above) that corresponds to each of the different activities and/or
  • Further processing of the above 3 generic classifications can be performed in example embodiments by:
  • Filtering and identification using frequency classifications and/or signal matching techniques.
  • With reference to FIG. 6, plots 600 and 602 show the SA for the most active axis calculated based on the maximum value minus the minimum value in the processing window, and calculated based on the maximum value minus the mean value in the processing window, respectively. As can be seen from FIG. 6, basing the SA on the maximum value minus the mean value in the processing window can be preferred for classification based on thresholds, as it results in a smoother plot with reduced absolute values and narrower bands of values in the respective activity regions,
  • FIG. 7 shows a flow chart 700 illustrating a process for counting steps according to an example embodiment. At step 702, the accelerometer data is captured. At step 704, the data is split into process windows. At step 706, low pass filtering is applied to the data from the most active axis based on classified activity. At step 708, a derivative of the filtered data using Pan Tompkins derivative operator is computed. At step 710, the output of the derivative signal is normalized. At step 712, peak detection is performed for step counting. At step 714, an adaptive threshold value is determined based on the current process window and updated to be used for the next process window. At step 716, over counted peaks are eliminated.
  • Details of example features that may be used separately and/or in combination with one or more of the other example features will be described next.
  • Low Pass Filter Based on Activity Type (Step 706, FIG. 7)
  • The cut-off frequency of the low pass filter is preferably adjusted based on the activity type to remove the correct frequency range of motion noise. Choosing a correct cut-off frequency can advantageously improve the signal geometry for avoiding false peaks or undetected peaks during the subsequent peak detection processing.
  • In example embodiments, the activity type may be input by the user, or may be identified using the activity classification method and device described above, or by any other method or device.
  • Pan Tompkins Derivative Operator (Step 708, FIG. 7)

  • y(n)=(⅛)[2x(n)+x(n−1)−x(n−3)−2x(n−4)]  Equation (1).
  • where n denotes the data point, and n>4
  • Pan Tompkins Derivative operator in equation (1) is used in a preferred embodiment to further improve the accuracy of peak detection by advantageously enlarging the slope information and optimizing the signal condition for peak detection.
  • During normal walking and running, for example arm swings are usually in sync with the opposite leg. For example, the right arm swings forward when the left feet steps forward and vice versa. Based on this reason, one acceleration peak is detected in e.g. a wrist-worn device for every step taken by the user.
  • Acceleration changes can also depend on the change in hand swing and impact force of feet. During normal walking condition, the impact force is typically not strong enough and most of the acceleration changes in e.g. a wrist-worn device are due to the hand swing. As the forward and backward swing of the hand is typically not the same for each step, the inconsistent peaks and troughs of the signal waveform can make it difficult to implement an accurate peak detection algorithm, and in a preferred embodiment, the Pan Tompkins derivation operator results in a more consistent peaks and troughs pattern. FIGS. 8a ) and b) show the signal waveform of a wrist-worn device before (curve 800) and after (curve 802) applying Pan Tompkins derivation operator according to an example embodiment, respectively.
  • Normalization (Step 710, FIG. 7)
  • The normalization process in example embodiments is preferred such that the derivative signals are scaled correctly for the peak detection processing. In one embodiment, the signal is normalized by first subtracting the minimum value of the signal from the signal and then dividing by the maximum value of the subtracted signal.
  • Peak Detection (Step 712, FIG. 7)
  • A peak will be detected (i.e. a step counted), when the normalized derivative signal exceeds a threshold level, according to example embodiments. As the data is now normalized (for example min value=0, max value=1), a threshold value Δl for a first processing window can e.g. be set to any pre-determined value between >0 and 0.5.
  • Determining and Updating Adaptive Threshold (Step 714, FIG. 7)
  • As shown in FIG. 9, an adaptive threshold value (Δw) will be determined and used for subsequent windows, which can be set at any pre-determined value between about >0% and 50% of the difference between the last peak and valley values of the current window. In one example:

  • Δw=(a−b)*0.5  Equation (2).
  • where Aw=adaptive threshold value of a next window w, a=last peak value in a current window (w−1), and b=last valley value in the said current window (w−1), and w≥2.
  • Elimination of Over Counted Peaks (Step 716, FIG. 7)
  • It was found by the inventors that the Pan Tompkins derivative operation can have a tendency to generate a false first peak in the subsequent windows which can result in over counting of peaks, i.e. steps.
  • FIG. 10 illustrates the method of advantageously identifying such false peaks to be removed from the count in a preferred embodiment. The false peaks are identified by checking the distance d between a first peak 1000 of the current measured window 1002 and the last peak 1004 of the previous window 1006.
  • A threshold may be set at 4 steps per second in one embodiment. For example, in the data point domain of a sampling rate of 80 Hz, the distance threshold may be set at 20 data points based on 4 steps per second. If the peak to peak distance d is smaller than 20 data points, the current peak 1200 is identified as false and is removed from the step count.
  • FIG. 11 shows a flow chart illustrating the process flow 1100 of one embodiment, in which both the activity classification and the activity based step counting techniques are integrated into a single wrist wearable device.
  • At step 1102, acceleration data of the user is captured by the device through a 3-axis accelerometer. At step 1104, the accelerometer signal is split into processing windows between about >0 to 5 seconds. At step 1106, the signal in each window is smoothed by performing a moving average function. At step 1108, the most active axis is identified by comparing the mean absolute moving average value of each axis. The axis with the largest value will be the most active axis.
  • At steps 1110 and 1114, the activity in each window is classified by calculating the signal amplitude (max-mean) of the moving average signal of the most active axis. Classification will be based on 0.1 g and 1 g thresholds in one non-limiting example. Once a low activity state is identified (step 1110), the process will move on to the next window (step 1112), as no steps are required to be counted.
  • For moderate and heavy activity states (step 1114), the step counting is performed. At step 1116 and 1118, the cut-off frequency of the low pass filter for each window will be updated based on the activity identified, and the low pass filtering applied accordingly.
  • At step 1120, the Pan Tompkins derivative operator is then applied. At step 1122, the derivative signal is then normalized to adjust the signal amplitude within, for example, 0 to 1 in each window (first subtracting the minimum value of the signal from the signal and then dividing by the maximum value of the subtracted signal in one non-limiting example).
  • At step 1124, peak detection is then performed where each peak detected contributes to a step count. In the first window after transiting to a moderate or heavy activity state from a light activity state, the threshold will be set between >0 to 0.5 in one non-limiting example. In other words, a first window transiting from a light activity state does not typically require step elimination. For subsequent windows (be it being a moderate or heavy activity state), over counted peak detection and elimination is advantageously performed until a light activity state is detected for the window. In the subsequent windows, an adaptive threshold is used and will be set between about >0 to 50% of the height difference between the last peak and valley of the previous window, in one non-limiting example, step 1126.
  • At step 1128, false peaks originating from the Pan Tompkins operation are then removed using e.g. a peak to peak distance (current window first peak vs. previous window last peak) threshold. The threshold in one non-limiting example is based on 4 steps per second. For example, in the data point domain of a sampling rate of 80 Hz, the distance threshold is set at 20 data point. If the peak to peak distance is smaller than 20 data points, the current peak is identified as false and is removed from the step count.
  • The process for the next window will loop back and follow through steps 1106 to 1128 described above.
  • FIG. 12 shows an assembly 1200 comprising a wearable device in the form of a wrist watch 1201 according to an example embodiment. It will be appreciated that in different embodiments the device may also be in any other form suitable to be worn on any part of the user's body such as his/her arms, waist, hip or foot. The wrist watch 1201 classifies the user's activity and/or computes the number of steps taken by the user and communicates the result(s) wirelessly to a telecommunication device of the assembly 1200 such as a mobile phone 1202 or other portable electronic devices, or computing devices such as desk top computers, laptop computer, tab computers etc.
  • FIG. 13 shows a schematic block diagram of an assembly 1300 comprising a wearable device 1301 according to an example embodiment, for classifying activity and/or counting steps of the user. The device 1301 includes a signal sensing module 1302, such as an accelerometer or gyroscope, for obtaining the acceleration information of the user.
  • One non-limiting example of a preferred accelerometer that can be adapted for use in the device is a triple-axis accelerometer MMA8652FC available from Freescale Semiconductor, Inc. This accelerometer can provide the advantage of measuring acceleration in all three directions with a single package. Alternatively, several single-axis accelerometers oriented to provide three-axis sensing can be used in different embodiments.
  • The device 1301 also includes a data processing and computational module 1304, such as a processor, which is arranged to receive and process the acceleration information from the signal sensing module 1302. The device 1301 also includes a display unit 1306 for displaying a result to a user of the device 1301. The device 1301 in this embodiment further includes a wireless transmission module 1308 arranged to communicate wirelessly with a telecommunications device 1310 of the assembly 1300. The telecommunication device 1310 includes a wireless receiver module 1312 for receiving signals from the wearable device 1301 and a display unit 1314 for displaying a result to a user of the telecommunication device 1310.
  • FIG. 14 shows a block diagram of a device 1400 for classifying the activity of a user wearing the device, according to one embodiment. The device 1400 comprises an accelerometer 1402 for measuring accelerometer data for a plurality of axes; and a processor 1404 configured to identify a most active one of the plurality of axes based on the accelerometer data and to classify the activity of the user based on a signal amplitude of the accelerometer data for the most active axis and one or more threshold values.
  • The processor 1404 may be configured to apply a smoothing to the accelerometer data prior to identifying the most active axis.
  • The processor 1404 may be configured to identify the most active axis by determining the axis having the largest mean absolute amplitude of the smoothed signal, which may also be referred to as the largest mean absolute moving average signal value.
  • The processor 1404 may be configured to classify the activity based on a maximum signal amplitude of the accelerometer data for the most active axis.
  • The processor 1404 may be configured to classify the activity based on the maximum signal amplitude of the accelerometer data minus the mean signal amplitude for the most active axis.
  • The processor 1404 may be configured to identify the most active axis and to classify the activity of the user based on accelerometer data measured over a predetermined processing window.
  • The processing window may be in a range from about >0 to 5 seconds.
  • FIG. 15 shows a block diagram of a device 1500 for counting the number of steps taken by a user wearing the device, according to one embodiment. The device 1500 comprises an accelerometer 1502 for measuring accelerometer data for at least one axis; and a processor 1504 configured to apply a derivative operator to the accelerometer data in successive processing windows, to count peaks in the derivative of the accelerometer data in each processing window, and to eliminate over counted peaks based on time difference between a first peak in a current window and a last peak in a preceding processing window.
  • The processor 1504 may be configured to count the peaks based on a first threshold value if the current processing window is a first processing window after transiting from a light activity state to a moderate or a heavy activity state.
  • The processor may further be configured to update an adaptive threshold for the next window. The adaptive threshold for the next window may be based on the difference in signal amplitude for a last successive peak and valley pair in the current processing window. The adaptive threshold for the next window may be between about >0 to 50% of the difference in signal amplitude for the last successive peak and valley pair in the current processing window.
  • The processor 1504 may be configured to eliminate peaks from the peak count in the current window if the time difference between the first peak in the current window and the last peak in the preceding processing window is smaller than a second threshold value. The second threshold value may be about ¼ seconds. The derivative operator may comprise a Pan Tomkins derivative operator.
  • The devices 1400, 1500 may be implemented in a wearable device.
  • The devices 1400, 1500 may be implemented in an assembly comprising a wearable device and a communication device.
  • The devices 1400, 1500 may be implemented in an assembly comprising a wearable device and a wireless communication device.
  • FIG. 16 shows a flow chart 1600 illustrating a method for classifying the activity of a user, according to an example embodiment. At step 1602, accelerometer data is measured for a plurality of axes. At step 1604, a most active one of the plurality of axes is identified based on the accelerometer data. At step 1606, the activity of the user is classified based on a signal amplitude of the accelerometer data for the most active axis and one or more threshold values.
  • The method may comprise applying a smoothing to the accelerometer data prior to identifying the most active axis.
  • The method may comprise identifying the most active axis by determining the axis having the largest mean absolute value of the smoothed signal, which may also be referred to as the largest mean absolute moving average signal value. The method may comprise classifying the activity based on a maximum signal amplitude of the accelerometer data for the most active axis.
  • The method may comprise classifying the activity based on the maximum signal amplitude of the accelerometer data minus the mean signal amplitude for the most active axis.
  • The method may comprise identifying the most active axis and classifying the activity of the user based on accelerometer data measured over a predetermined processing window.
  • The processing window may be in a range from about >0 to 5 seconds.
  • FIG. 17 shows a flow chart 1700 illustrating a method for counting the number of steps taken by a user. At step 1702, accelerometer data is measured for at least one axis. At step 1704, a derivative operator is applied to the accelerometer data in successive processing windows. At step 1706, peaks in the derivative of the accelerometer data in each processing window are counted. At step 1708, over counted peaks are eliminated based on time difference between a first peak in a current window and a last peak in a preceding processing window.
  • The method may comprise counting the peaks based on a first threshold if the current processing window is a first processing window after transiting from a light activity state to a moderate or a heavy activity state.
  • The method may further comprise updating an adaptive threshold for the next window. The adaptive threshold for the next window may be based on the difference in signal amplitude for a last successive peak and valley pair in the current processing window. The adaptive threshold for the next window may be between about >0 to 50% of the difference in signal amplitude for the last successive peak and valley pair in the current processing window.
  • The method may comprise eliminating peaks from the peak count in the current processing window if the time difference between the first peak in the current processing window and the last peak in the preceding processing window is smaller than a second threshold value. The second threshold value may be about ¼ seconds.
  • The derivative operator may comprise a Pan Tomkins derivative operator.
  • Example embodiments of the present invention advantageously do not require the device to be attached at a fixed orientation throughout the monitoring period. This is because the device in example embodiments constantly detects within processing windows for the most active axis (which will be the one closest to the direction of the impact force), to be used for subsequent processes. The most active axis can be identified in example embodiments by looking for the axis having the largest mean absolute moving average acceleration signal within each window.
  • Embodiments of the present invention can advantageously be able to detect changes and differentiate the signals corresponding to light, moderate or heavy states by splitting the accelerometer data into process windows and observing the signal amplitude (e.g. max-mean or max-min) against threshold levels (e.g. 0.1 g and 1 g) within each window.
  • In embodiments of the present invention, motion noise can advantageously be correctly removed using low pass filters with suitable cut-off frequencies depending on the activity.
  • In embodiments of the present invention, a Pan Tompkins derivative operation together with a peak elimination algorithm based on peak distance and adaptive thresholds based on max-min value of a peak advantageously helps the device and method to accurately identify peaks as step counts.
  • It will be appreciated by a person skilled in the art that numerous variations and/or modifications may be made to the present invention as shown in the specific embodiments without departing from the spirit or scope of the invention as broadly described. The present embodiments are, therefore, to be considered in all respects to be illustrative and not restrictive. Also, the invention includes any combination of features, in particular any combination of features in the patent claims, even if the feature or combination of features is not explicitly specified in the patent claims or the present embodiments.
  • For example, while a wrist-worn device is described in some embodiments, the device may be worn on the arms, hip, waist or foot of the user. Also, the device and method may perform only one of the activity classification and the step counting as described above, or both at the same time.

Claims (19)

1. A device for counting the number of steps taken by a user wearing the device, the device comprising:
an accelerometer for measuring accelerometer data for at least one axis; and
a processor configured to:
apply a derivative operator to the accelerometer data in successive processing windows;
count peaks in the derivative of the accelerometer data in each processing window; and
eliminate over counted peaks based on time difference between a first peak in a current window and a last peak in a preceding processing window.
2. The device as claimed in claim 1, wherein the processor is configured to count the peaks based on a first threshold value if the current processing window is a first processing window after transiting from a light activity state to a moderate or a heavy activity state.
3. The device as claimed in claim 1, wherein the processor is further configured to update an adaptive threshold for the next window.
4. The device as claimed in claim 3, wherein the adaptive threshold for the next window is based on the difference in signal amplitude for a last successive peak and valley pair in the current processing window.
5. The device as claimed in claim 4, wherein the adaptive threshold for the next window is between about >0 to 50% of the difference in signal amplitude for the last successive peak and valley pair in the current processing window.
6. The device as claimed in claim 1, wherein the processor is configured to eliminate peaks from the peak count in the current window if the time difference between the first peak in the current window and the last peak in the preceding processing window is smaller than a second threshold value.
7. The device as claimed in claim 6, wherein the second threshold value is about ¼ seconds.
8. The device as claimed in claim 1, wherein the derivative operator comprises a Pan Tomkins derivative operator.
9. The device as claimed in claim 1, wherein the device is implemented in a wearable device.
10. The device as claimed in claim 1, wherein the device is implemented in an assembly comprising a wearable device and a communication device.
11. The device as claimed in claim 1, wherein the device is implemented in an assembly comprising a wearable device and a wireless communication device.
12. A method for counting the number of steps taken by a user, the method comprising:
measuring accelerometer data for at least one axis;
applying a derivative operator to the accelerometer data in successive processing windows;
counting peaks in the derivative of the accelerometer data in each processing window; and
eliminating over counted peaks based on time difference between a first peak in a current window and a last peak in a preceding processing window.
13. The method as claimed in claim 12, comprising counting the peaks based on a first threshold if the current processing window is a first processing window after transiting from a light activity state to a moderate or a heavy activity state.
14. The method as claimed in claim 12, further comprising updating an adaptive threshold for the next window.
15. The method as claimed in claim 14, wherein the adaptive threshold for the next window is based on the difference in signal amplitude for a last successive peak and valley pair in the current processing window.
16. The method as claimed in claim 15, wherein the adaptive threshold for the next window is between about >0 to 50% of the difference in signal amplitude for the last successive peak and valley pair in the current processing window.
17. The method as claimed in claim 12, comprising eliminating peaks from the peak count in the current window if the time difference between the first peak in the current processing window and the last peak in the preceding processing window is smaller than a second threshold value.
18. The method as claimed in claim 17, wherein the second threshold value is about ¼ seconds.
19. The method as claimed in claim 1, wherein the derivative operator comprises a Pan Tomkins derivative operator.
US16/558,739 2014-05-30 2019-09-03 Device and method for classifying the activity and/or counting steps of a user Abandoned US20200000379A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/558,739 US20200000379A1 (en) 2014-05-30 2019-09-03 Device and method for classifying the activity and/or counting steps of a user

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
PCT/SG2014/000248 WO2015183193A1 (en) 2014-05-30 2014-05-30 Device and method for classifying the activity and/or counting steps of a user
US201615315090A 2016-11-30 2016-11-30
US16/558,739 US20200000379A1 (en) 2014-05-30 2019-09-03 Device and method for classifying the activity and/or counting steps of a user

Related Parent Applications (2)

Application Number Title Priority Date Filing Date
US15/315,090 Division US10441197B2 (en) 2014-05-30 2014-05-30 Device and method for classifying the activity and/or counting steps of a user
PCT/SG2014/000248 Division WO2015183193A1 (en) 2014-05-30 2014-05-30 Device and method for classifying the activity and/or counting steps of a user

Publications (1)

Publication Number Publication Date
US20200000379A1 true US20200000379A1 (en) 2020-01-02

Family

ID=54699385

Family Applications (2)

Application Number Title Priority Date Filing Date
US15/315,090 Active 2035-02-23 US10441197B2 (en) 2014-05-30 2014-05-30 Device and method for classifying the activity and/or counting steps of a user
US16/558,739 Abandoned US20200000379A1 (en) 2014-05-30 2019-09-03 Device and method for classifying the activity and/or counting steps of a user

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US15/315,090 Active 2035-02-23 US10441197B2 (en) 2014-05-30 2014-05-30 Device and method for classifying the activity and/or counting steps of a user

Country Status (6)

Country Link
US (2) US10441197B2 (en)
JP (1) JP6567658B2 (en)
KR (2) KR20170019347A (en)
CN (2) CN113303790A (en)
SG (1) SG11201609607QA (en)
WO (1) WO2015183193A1 (en)

Families Citing this family (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016019894A1 (en) * 2014-08-07 2016-02-11 Beijing Zhigu Tech Co., Ltd. Dominant limb identification method and device
US11609242B1 (en) * 2014-10-26 2023-03-21 David Martin Efficient gait data management in mobile
US11045116B1 (en) * 2017-09-15 2021-06-29 David Martin Enhanced determination of cadence for control in mobile
WO2016085403A1 (en) * 2014-11-27 2016-06-02 Razer (Asia-Pacific) Pte. Ltd. Step counter devices and step counting methods
WO2017148701A1 (en) 2016-02-29 2017-09-08 Koninklijke Philips N.V. Measurement apparatus and correction method
CN106108913B (en) * 2016-07-25 2019-05-07 北京顺源开华科技有限公司 Error count walks removing method, device and wearable device
US10952686B2 (en) 2016-08-02 2021-03-23 Medtronic, Inc. Mobile application to prompt physical action to measure physiologic response in implantable device
US10610132B2 (en) * 2016-08-02 2020-04-07 Medtronic, Inc. Step detection using accelerometer axis
CN107063291A (en) * 2017-03-06 2017-08-18 建荣半导体(深圳)有限公司 Step-recording method, step count set, meter step chip and mobile terminal
CN109452728B (en) * 2017-04-12 2021-03-09 佛山市丈量科技有限公司 Intelligent insole based on step length calculation and step length calculation method thereof
WO2019036927A1 (en) * 2017-08-23 2019-02-28 华为技术有限公司 Stride frequency based foot step counting method, apparatus and device
CN110068322B (en) * 2019-04-18 2024-03-01 中国电子科技集团公司电子科学研究院 Pedestrian positioning method and pedestrian positioning device based on terminal
US11717186B2 (en) 2019-08-27 2023-08-08 Medtronic, Inc. Body stability measurement
CN111765899B (en) * 2020-06-30 2022-04-22 歌尔科技有限公司 Step counting judgment method and device and computer readable storage medium
US11602313B2 (en) 2020-07-28 2023-03-14 Medtronic, Inc. Determining a fall risk responsive to detecting body position movements
CN112378416B (en) * 2020-12-07 2023-01-24 歌尔科技有限公司 Step counting method, step counting device and readable storage medium
CN113303789B (en) * 2021-04-30 2023-01-10 武汉齐物科技有限公司 Gait event detection method and device based on acceleration
EP4111959A1 (en) * 2021-06-30 2023-01-04 Koninklijke Philips N.V. Body position detection

Family Cites Families (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6539336B1 (en) * 1996-12-12 2003-03-25 Phatrat Technologies, Inc. Sport monitoring system for determining airtime, speed, power absorbed and other factors such as drop distance
JP4632746B2 (en) 2004-10-29 2011-02-16 三洋電機株式会社 Momentum measuring device and momentum measuring program
US7467060B2 (en) 2006-03-03 2008-12-16 Garmin Ltd. Method and apparatus for estimating a motion parameter
US7457719B1 (en) 2006-11-21 2008-11-25 Fullpower Technologies, Inc. Rotational insensitivity using gravity-based adjustment
US7653508B1 (en) * 2006-12-22 2010-01-26 Dp Technologies, Inc. Human activity monitoring device
WO2008133921A1 (en) 2007-04-23 2008-11-06 Dp Technologies, Inc. Eyewear having human activity monitoring device
EP2260763A1 (en) * 2008-03-31 2010-12-15 Sharp Kabushiki Kaisha Body motion measuring device, mobile telephone, method for controlling the body motion measuring device, body motion measuring device control program, and computer-readable recording medium having the program recorded therein
US8996332B2 (en) * 2008-06-24 2015-03-31 Dp Technologies, Inc. Program setting adjustments based on activity identification
US8187182B2 (en) * 2008-08-29 2012-05-29 Dp Technologies, Inc. Sensor fusion for activity identification
JP5417779B2 (en) 2008-09-18 2014-02-19 オムロンヘルスケア株式会社 Activity meter
GB2467514A (en) 2008-12-23 2010-08-04 Univ Oxford Brookes Gait monitor for sensing vertical displacement
JP5982714B2 (en) * 2009-01-05 2016-08-31 コミシリア ア レネルジ アトミック エ オ エナジーズ オルタネティヴズ Apparatus and method for characterizing operation
FR2942388B1 (en) * 2009-02-26 2012-10-12 Movea SYSTEM AND METHOD FOR DETECTING THE MARKET OF A PERSON
US8956294B2 (en) * 2009-05-20 2015-02-17 Sotera Wireless, Inc. Body-worn system for continuously monitoring a patients BP, HR, SpO2, RR, temperature, and motion; also describes specific monitors for apnea, ASY, VTAC, VFIB, and ‘bed sore’ index
CN102667672B (en) * 2009-07-07 2014-04-02 闫文闻 Acceleration motion identify method and system thereof
FI20096232A0 (en) 2009-11-23 2009-11-23 Valtion Teknillinen Physical activity-based control for a device
JP5617299B2 (en) 2010-03-25 2014-11-05 オムロンヘルスケア株式会社 Activity meter, control program, and activity type identification method
US9167991B2 (en) * 2010-09-30 2015-10-27 Fitbit, Inc. Portable monitoring devices and methods of operating same
US8694251B2 (en) * 2010-11-25 2014-04-08 Texas Instruments Incorporated Attitude estimation for pedestrian navigation using low cost mems accelerometer in mobile applications, and processing methods, apparatus and systems
JP5364744B2 (en) 2011-03-01 2013-12-11 日本電信電話株式会社 Acceleration detection apparatus, method and program
CN102184549B (en) * 2011-04-29 2012-10-10 闫文闻 Motion parameter determination method and device and motion auxiliary equipment
JP5915285B2 (en) * 2012-03-15 2016-05-11 セイコーエプソン株式会社 Status detection device, electronic device, measurement system, and program
JP2014006090A (en) 2012-06-22 2014-01-16 Seiko Epson Corp Portable device
US9999376B2 (en) * 2012-11-02 2018-06-19 Vital Connect, Inc. Determining body postures and activities
US9161172B2 (en) * 2012-11-06 2015-10-13 Qualcomm Incorporated Map-based adaptive sampling of orientation sensors for positioning
US9398867B2 (en) * 2013-01-07 2016-07-26 Archinoetics Llc Efficient activity classification from motion inputs
JP2014212915A (en) * 2013-04-25 2014-11-17 セイコーエプソン株式会社 Action discrimination device, and action discrimination method
EP3058440B1 (en) * 2013-10-14 2021-04-14 NIKE Innovate C.V. Calculating pace and energy expenditure from athletic movement attributes

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Y. Cho, H. Cho and C. -M. Kyung, "Design and Implementation of Practical Step Detection Algorithm for Wrist-Worn Devices," in IEEE Sensors Journal, vol. 16, no. 21, pp. 7720-7730, Nov.1, 2016, doi: 10.1109/JSEN.2016.2603163. (Year: 2016) *

Also Published As

Publication number Publication date
KR102141367B1 (en) 2020-08-05
JP2017522149A (en) 2017-08-10
US20170188897A1 (en) 2017-07-06
SG11201609607QA (en) 2016-12-29
CN106572814A (en) 2017-04-19
WO2015183193A1 (en) 2015-12-03
US10441197B2 (en) 2019-10-15
KR20190112842A (en) 2019-10-07
JP6567658B2 (en) 2019-08-28
CN113303790A (en) 2021-08-27
KR20170019347A (en) 2017-02-21
CN106572814B (en) 2021-06-08

Similar Documents

Publication Publication Date Title
US20200000379A1 (en) Device and method for classifying the activity and/or counting steps of a user
US10506990B2 (en) Devices and methods for fall detection based on phase segmentation
US9797743B2 (en) Method and system for determining whether to reset a height in a height determining device based on the occurrence of steps
US10653339B2 (en) Time and frequency domain based activity tracking system
Pratama et al. Smartphone-based pedestrian dead reckoning as an indoor positioning system
EP3099229B1 (en) Improvements in the detection of walking in measurements of the movement of a user
RU2584459C2 (en) Gesture control for monitoring vital body signs
US10973464B2 (en) Device and method for removing artifacts in physiological measurements
Raknim et al. Gait monitoring for early neurological disorder detection using sensors in a smartphone: Validation and a case study of parkinsonism
US20170311899A1 (en) Apparatus and method for identifying movement in a patient
KR20080029209A (en) Method of step detection at pedometer
JP6867448B2 (en) Devices and methods for classifying user activity and / or counting user steps
TW201918225A (en) Method and system for analyzing gait
US11280634B2 (en) System, method and article for counting steps using an accelerometer
Weng et al. Fall detection based on tilt angle and acceleration variations
Lan et al. Early detection of neurological disease using a smartphone: a case study
Gan Automatic labeling for personalized IoT wearable monitoring
US20230389821A1 (en) Estimating vertical oscillation at wrist
Pham et al. Classification different types of fall for reducing false alarm using single accelerometer
US20240041354A1 (en) Tracking caloric expenditure using a camera
van Oeveren et al. VU Research Portal
US20190282128A1 (en) Processing apparatus and method for determining an ambulation motion of a subject
CN117918827A (en) Step counting and gesture recognition method and device based on multi-parameter fusion

Legal Events

Date Code Title Description
AS Assignment

Owner name: NITTO DENKO CORPORATION, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:THEIN, KHINE CHO CHO;KASAMSOOK, KITTIPONG;AUNG, AYE;REEL/FRAME:050248/0828

Effective date: 20161205

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE AFTER FINAL ACTION FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: ADVISORY ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION