WO2021255740A1

WO2021255740A1 - Posture detection device and system

Info

Publication number: WO2021255740A1
Application number: PCT/IL2021/050739
Authority: WO
Inventors: Gaddi BLUMROSEN; Alexander LOEBEL; Shaked GUTMAN OZ; Raphael Ofer HESS
Original assignee: Labstyle Innovation Ltd.
Priority date: 2020-06-19
Filing date: 2021-06-17
Publication date: 2021-12-23

Abstract

A posture detection device and system including same are provided. The device includes a device body attachable to an individual and including a sensor for sensing a body self-motion. The device further includes a processing module for processing sensor data related to the body self- motion over time to derive spatiotemporal patterns of movement and classifying each of the patterns to a posture to thereby detect a posture of said individual.

Description

POSTURE DETECTION DEVICE AND SYSTEM

RELATED APPLICATION/S

This application claims the benefit of priority of U.S. Provisional Patent Application No. 63/041,127 filed on June 19, 2020, the contents of which are incorporated herein by reference in their entirety.

FIELD AND BACKGROUND OF THE INVENTION

The present invention relates to a posture detection device, system and method. Embodiments of the present invention relate to a posture detection device that tracks body self- motion to extract spatial-temporal data that can be used to classify various posture positions of an individual in real time.

Posture is the arrangement in which an individual holds their body and limbs. Good postures exert the least strain on muscles and ligaments during movement or when stationary (e.g., sitting, standing or lying down) while poor postures can stress muscles, joints and ligaments and lead to pain and injury.

Self-identifying and correcting poor posture can be challenging and requires awareness and attention from the individual.

The prevalence of bad posture especially among individuals of western countries and the negative impact it has on the quality of life and productivity of individuals, has led to the development of devices for monitoring and correcting poor posture.

Such devices typically take the form of a wearable device that tracks body angles or motion to identify poor posture and alert the user. Some devices can also provide the user with instructions or training focused at improving posture.

While such devices can be effective in retraining individuals to maintain good posture, they can be inaccurate in distinguishing between postures that are similar in as far as body positions but are radically different in as far as impact on health (e.g., Lean vs. Slouch). Consequently, the instruction to the users without knowledge on the precise posture in real-time, can be inaccurate, and cause even damage.

There is thus a need for, and it would be highly advantageous to have, a posture detection device that can work in real-time, and provide accurate information to the user related to his or her current posture. SUMMARY OF THE INVENTION

According to one aspect of the present invention there is provided a posture detection device comprising a device body attachable to an individual, the device body including one or more sensors for sensing a body self-motion; and a processing module for processing sensor data related to the body self-motion over time to derive spatiotemporal patterns of movement; and classifying each of the spatiotemporal patterns to a posture to thereby detect a posture of the individual.

According to embodiments of the present invention the device further comprises a component for providing feedback to the individual.

According to embodiments of the present invention the feedback can be tactile (e.g., vibrator) and/or auditory and/or visual.

According to embodiments of the present invention the feedback is a posture correction alert (signal).

According to embodiments of the present invention the device further comprises a wireless communication module for communicating with a remote device.

According to embodiments of the present invention the remote device is a smartphone or a computer.

According to embodiments of the present invention the remote device provides tactile and/or auditory and/or visual feedback to the individual.

According to embodiments of the present invention the feedback indicates a posture and/or posture correction.

According to embodiments of the present invention the feedback forms a part of a training session for training the individual.

According to embodiments of the present invention the sensor is a strain sensor.

According to embodiments of the present invention the sensor is a 3, or 6, or 9-axes sensor (inertial, angle, and magnetic sensors).

According to embodiments of the present invention the processing module classifies the spatiotemporal patterns to detect a slouch posture or a lean posture.

According to embodiments of the present invention the sensor continuously collects body self-motion data.

According to embodiments of the present invention the sensor measures self-motion via acceleration and/or angular velocity.

According to embodiments of the present invention the processing module extracts a set of spatial-temporal features from the body self-motion data. According to embodiments of the present invention the processing module processes the spatial-temporal features to extract the spatiotemporal patterns at least a portion of which represent state transition probabilities.

According to embodiments of the present invention the processing module utilizes the state transition probabilities for posture classification.

According to embodiments of the present invention the processing module utilizes an upright posture of the individual as a reference.

According to another aspect of the present invention there is provided posture detection system comprising a device attachable to an individual, the device including a sensor for sensing a body self-motion; and a processing module being in communication with the device, the processing module being for processing sensor data related to the body self-motion over time to derive spatiotemporal patterns of movement; and classifying each of the spatiotemporal patterns to a posture to thereby detect a posture of the individual.

According to embodiments of the present invention the processing module forms a part of a user device.

According to embodiments of the present invention the user device is a smartphone.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although methods and materials similar or equivalent to those described herein can be used in the practice or testing of the present invention, suitable methods and materials are described below. In case of conflict, the patent specification, including definitions, will control. In addition, the materials, methods, and examples are illustrative only and not intended to be limiting.

Implementation of the method and system of the present invention involves performing or completing selected tasks or steps manually, automatically, or a combination thereof. Moreover, according to actual instrumentation and equipment of preferred embodiments of the method and system of the present invention, several selected steps could be implemented by hardware or by software on any operating system of any firmware or a combination thereof. For example, as hardware, selected steps of the invention could be implemented as a chip or a circuit. As software, selected steps of the invention could be implemented as a plurality of software instructions being executed by a computer using any suitable operating system. In any case, selected steps of the method and system of the invention could be described as being performed by a data processor, such as a computing platform for executing a plurality of instructions. BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S)

The invention is herein described, by way of example only, with reference to the accompanying drawings. With specific reference now to the drawings in detail, it is stressed that the particulars shown are by way of example and for purposes of illustrative discussion of the preferred embodiments of the present invention only, and are presented in the cause of providing what is believed to be the most useful and readily understood description of the principles and conceptual aspects of the invention. In this regard, no attempt is made to show structural details of the invention in more detail than is necessary for a fundamental understanding of the invention, the description taken with the drawings making apparent to those skilled in the art how the several forms of the invention may be embodied in practice.

In the drawings:

FIGs. 1 A-B illustrate various body postures including (left to right) sit upright, slouch, lean forward, lean back, and lean to side (Figure 1A), Figure IB illustrates the difference between lean and slouch.

FIG. 2 is a flowchart outlining the steps of raw data processing, posture recognition and feedback.

FIG. 3 illustrates one embodiment of the present device.

FIG. 4 illustrates the present system including a sensor device connected to a hub (e.g., cellphone) and possibly to a communication network (e.g., cloud). Feedback can be provided by the sensor device, the hub or a third connected feedback device.

FIG. 5 illustrates raw data processing, feature extraction and posture classification.

FIG. 6 illustrates the orientation axes of the sensors and related rotation angles around these axes.

FIG. 7 illustrates states and related possible transitions for sagittal planes postures: straight, slouch, leaning, leaning and slouch, and bending.

FIGs. 8A-C illustrate raw-data single traces from an accelerometer (upper row) and gyroscope (lower row) of a subject performing slouch and lean forward movements (Figure 8A), preprocessing of single traces to derive the sagittal plane angle (Figure 8B) and feature extraction from the pre-processed data (Figure 8C).

FIGs. 9A-B illustrate output of a classifier (Figure 9A) trained on the data shown in Figure 8A and the results of modeling such classification (Figure 9B). DESCRIPTION OF SPECIFIC EMBODIMENTS OF THE INVENTION

The present invention is of a device and system which can be used to determine a posture of an individual. Specifically, the present invention can be used to discern between slouch and lean postures.

The principles and operation of the present invention may be better understood with reference to the drawings and accompanying descriptions.

Before explaining at least one embodiment of the invention in detail, it is to be understood that the invention is not limited in its application to the details set forth in the following description or exemplified by the Examples. The invention is capable of other embodiments or of being practiced or carried out in various ways. Also, it is to be understood that the phraseology and terminology employed herein is for the purpose of description and should not be regarded as limiting.

Good posture is important for maintaining musculoskeletal health. Good posture can be tailored to each individual, and can change during the day. Throughout our daily lives we assume numerous postures. Sitting at a desk an individual can sit straight, slouch, lean forward and back and lean to the side (Figures 1A-B). While leaning or sitting straight are considered good postures that do not overly stress the musculoskeletal system, slouch is considered a poor posture that can stress muscles, joints and ligaments and lead to pain and injury.

Devices for posture identification are well known in the art and typically take the form of a wearable device that measures body position. Although such devices can be effective in identifying poor posture and alerting the user, they cannot accurately discern between lean and slouch postures (Figure IB).

While reducing the present invention to practice, the present inventors devised a posture detection device and system that can accurately discern between lean and slouch postures. As is further described hereinunder, the present invention utilizes raw data processing that enables accurate classification of various postures including lean and slouch.

Thus, according to one aspect of the present invention there is provided a method of detecting posture.

The method is carried out by processing sensor data related to body self-motion over time to derive spatiotemporal patterns of movement (time-dependent movement). As used herein, “body self-motion” refers to any motion of the body that is controlled by the individual, i.e., coordinated by the musculoskeletal system of the individual under his or her control. As used herein “spatiotemporal patterns of movement” refers to patterns that capture the extent and characteristics of movement (as measured by angle, distance etc. in any axis) over time. These patterns can include acceleration and/or angular velocity.

The present method classifies these spatiotemporal patterns of movement to postures, with each posture having a unique signature, to thereby detect a posture of the individual.

As is further described hereinbelow and in the Examples section that follows, extraction of these patterns of movement from raw sensor data (e.g., raw data obtained from a strain sensor, a 3- axis sensor or a 6-axis sensor) enables the present approach to distinguish between lean and slouch postures.

Since slouch is oftentimes a slower movement than lean (lean is a more dynamic transition that is often carried out for a purpose, e.g., lean over to look at something), the transitions of lean and slouch will have different movement temporal signatures.

This unique feature of the present invention enables accurate detection of lean and slouch with a high degree of confidence. Thus, feedback provided to the user (individual monitored) will include far less false positives (lean identified as slouch).

The approach of the present invention can be carried out using a dedicated standalone device (Figure 3) that includes a sensor or sensors for acquiring body motion data, a processing module for processing the raw data and classify postures and a feedback module for providing feedback to the user. Such a device can be worn by the user (as a pendant or a clip) or attached directly to the user’s body, e.g., back (via adhesive).

Alternatively, the present approach can be carried out using a system that includes one or more sensors capable of communicating with each other and a remote processing module via a communication network (Figure 4).

The present approach is described below with reference to a standalone device configuration.

Referring now to the drawings, Figure 3 illustrates a posture detection device which is referred to herein as device 10.

Device 10 includes a device body 12 fabricated from a polymer and/or alloy. Device body 12 can be spherical, ellipsoid or in any shape with dimensions in the range of several cm in length and in diameter.

The external surface of device body 12 can include an attachment element for attaching device 10 to clothing, a chain or directly to the user’s body (via, for example, adhesive tape). Button 13, FED lighting 14 and port 15 (charge, data) are also positioned on the external surface and provide feedback and power functions as well as data transfer capabilities. Device body houses a sensor or sensors 16 (e.g., 6-axis sensor such as BMI160-Bosch or LSM6DSL-ST), a power supply 18 (e.g., rechargeable Li ion battery), a processing module 20 and related circuitry and memory storage, a feedback module 22 (e.g., vibrating element) and optionally a short and/or long range communication module (e.g., with Bluetooth and WiFi capabilities).

Raw body self-motion data is collected by sensor 16 and transferred to processing module 20 via a data link. Processing module 20 executes an algorithm for processing the raw data, deriving the spatiotemporal patterns and estimating the posture. Processing module 20 also controls feedback module 22 and sends an operational signal thereto when appropriate (e.g., detection of a slouch posture).

The algorithm includes three main portions (algorithms), sensor setup and calibration, on going dynamic posture recognition and a feedback assessment (Figure 5).

The system shown in Figure 4 incorporates device 10 (shown mounted on a user’s back) and utilizes a local hub 30 (e.g., smartphone) to connect device 10 to a server and/or a feedback device 34 (e.g., computer) through cloud 32.

Sensor setup and calibration

The posture detection algorithms of the present invention can utilize several parameters that vary in value between different users, and for different use scenarios for the same user. Calibration of these parameters at specific reference time points can be used to increase the accuracy of detection.

These parameters encompass positioning of the device on the user, e.g., their deviation from a world-aligned positioning (a coordinate system aligned with the gravitational force); and they provide a reference point to a subject’s healthy posture, that is, at what set of features (e.g. sagittal angle) does the device measure an upright posture of the user.

The calibration can be performed by having the user interact with the device of Figure 3 and/or the system of Figure 4 (via an App on the user’s local hub, e.g., Smartphone, personal computer etc.). The user is instructed to perform a set of operations for example, the user can be instructed to maintain a straight (upright) posture for a short period of time in order to provide a reference point for the algorithmic state that refers to a ‘straight’ posture (Figure 7). The earth’s gravitational force can serve as an additional essential reference point for, e.g., when the device is static (when the user is not moving).

Presented is one example for such a calibration process at the self-body coordinates, which involve setting the user’s straight angle; and correcting for non-zero yaw angle. A similar calibration process can be used to virtually rotate the device to the world-aligned coordinates. The choice of the coordinate system has no impact on the algorithm.

In particular to the example, after the user provides feedback to the system that they are straight (healthy posture), the statistics S (e.g., median) of the acceleration component measured over a short time window (from sensor 16) can then be estimated as follows:

In cases where the yaw angle (frontal plane, Figure 6) is non-zero, e.g., due to inaccurate sensor placement, the device is rotated to a yaw = 0° position. In particular, the yaw-angle displacement can be calculated from the following:

and the rotation matrix, with which the acceleration and angular velocity are corrected by, can be:

and finally, the rotated sensor’s accelerations and angular velocities can be provided by:

The pitch angle (Sagittal plane, Figure 6) of a user can be estimated at any point in time from the rotated raw data:

A user’s straight angle can be calculated from the rotated acceleration values measured at the calibration phase.

Calibration can also be carried out automatically, through an auto-calibration process. In auto-calibration, the parameters of device and user are automatically extracted using statistical databases and knowledge of physical reference points. For example, the aforementioned rotation matrix can be calculated without input from the user in two steps. In the first step, the reference set of postures is detected using statistical methods, such as non-supervised machine learning. It can exploit statistical priors to automatically determine when a user is at reference posture (e.g. straight upright position). The prior is calculated from either the user or from information gathered from a user population. While in a reference posture, the calibration parameters can be calculated (the second step). For example, the rotation matrix is gleaned at instances in which the user is detected as static, e.g., when the total measured acceleration is similar to Earth gravity.

Posture recognition

The different portions of the algorithm can be computed in real-time, or alternatively, the data collected from the user’s activity can be stored on a local device (e.g., smartphone) or the cloud for offline analysis. While the posture recognition can use multiple set of sensors the procedure below is described for a single device having a single on-board sensor.

Posture detection includes the following processing steps:

(i) Continuous collection of sensor data

(ii) Pre-processing of the data

(iii) Features derivation

(iv) Computation of the state transition probabilities

(v) States recognition

The above steps are shown in Figure 5.

Continuous collection of sensor data

The raw-data and its derivatives are locally stored in buffers (e.g., on device 10) or on a personal device or a server in the cloud. The data sampling frequency and length of the buffers are adjusted as to achieve accurate estimation of the statistics used, e.g., estimating the features the posture classification is based on.

Pre-processing of the data

Pre-processing of the data is carried out in processing module 20 (or on a cloud server or personal computer) and includes the following steps:

(i) Mitigating missing samples - Interpolating over missing data samples based on their time stamps.

(ii) Transferring the coordinate system from world axes to subject axes. This step includes the continuous compensation of sensor location and drift.

(iii) Filtering can be applied, based on prior knowledge of sensor statistics. Such filtering can include: a low pass filter of frequencies over 15 Hz and a high pass filter of low frequencies (for example under 0.5 Hz). It is noted that when using IMUs, high-pass-filters can impair gravity related information that is essential for the calculation of the rotation matrix.

(iv) Calculating angles and exclude biases from measurements by using filters such as Kalman or particle filters.

Features derivation

An essential step is the derivation of the values of a set of pre-determined features from the raw and pre-processed data. A feature can be a time-dependent pattern of movement or any variable that can be extracted from it. The features are calculated by using expert knowledge, i.e., from feature engineering; from advanced machine learning tools such as deep neural-networks; or by a combination of the aforementioned techniques. Feature selection algorithms can be used to exclude redundant features, leading to a more efficient and compact implementation on processing unit 20 of device 10. When the raw data is fed to a neural network, the features can be calculated and selected automatically during the training phase of the neural network.

Computation of the state transition probabilities

The values of the features at any given time point serve as the input for the pre-trained classifier. Its output are the probabilities that form the instantaneous state transition matrix at that time. In particular, the states considered are those that are included in the state map. An example state map is shown in Figure 7. State recognition

The state _f at any given time point is the one that minimizes the loss function |L :

where |p,_; is the transition matrix and $,_{„ Ί} is the estimated state at the previous time point. For example, the simplest loss function would be |ΐ — 1 ~ j for all i states that are directed to from That is, the state [s’_r will be the one with the highest transition probability.

Feedback

Once the output of the algorithm is determined, a feedback is given to the subject in real time. In particular, the feedback is derived from a chosen policy saved to device 10 (or cloud server), which can be individually adjusted for a given user based on preferences (e.g., improving a specific health concern or maximizing mobility); or be individually adjusted by reinforcement learning algorithms that can optimize the reward the user receives.

The feedback can be delivered via any type of stimulus, e.g., tactile (vibration) auditory (e.g., sound) or visual (e.g., lights or graphic).

The feedback can be defined as the maximization of a pre-designed reward function. In mathematical terms:

where \R is the reward function, \f represents the feedback function, and

is the chosen feedback at time

Posture related rewards can be, for example, minimizing overall time a subject spends slouching or balancing between a positive reinforcer provided as feedback for minimizing slouch, and a negative reinforcer provided as feedback for insufficient movement. In addition, the users could choose from the available reward scenarios and have direct influence on the reward and feedback they receive.

In order to maximize the impact of feedback on users’ behavior, different sets of stimuli will be provided for different types of feedback. The choice of stimulus will be in accordance with established psycho-physical prior knowledge. For example, different stimuli for positive and negative postures, different stimulus magnitude as function of how much posture correction is needed and the like.

As used herein the term “about” refers to ± 10 %.

Additional objects, advantages, and novel features of the present invention will become apparent to one ordinarily skilled in the art upon examination of the following examples, which are not intended to be limiting.

EXAMPLES

Reference is now made to the following example, which together with the above descriptions, illustrate the invention in a non-limiting fashion.

Posture Recognition

Raw data from devices worn by a group of subjects of both genders and of all ages was utilized to construct a posture classifier. The data is gathered, following individual calibration of the devices, from epochs in which the subjects perform slouch and lean forward movements, with each motion beginning and ending at an upright position (Figure 8A). The motions can be done while the subjects are in a seated position, or when they are standing. The pre-processing of the data includes steps of filtering, normalizing and interpolating it, and deriving Q , the sagittal plane angle (Figure 8B). Subsequently, the dimensionality of the data is reduced to an optimized set of informative features.

The data is represented as points in a six-dimensional space, which is partially visualized by three two-dimensional planes (Figure 8C). To train machine-learning classifiers, such as SVM, Random Forest and KNN, the represented data in the feature space is randomly divided into training, validation and testing sets. Depicted in Figure 9A is one possible outcome of such modeling, and a performance analysis, summarized in the form of a confusion matrix, on the test set. The specific metric, e.g., maximal precision or maximal recall, by which the model performance is optimized can differ between different usages of the invention and different feedback policies.

Once a classifier with satisfying performance is derived, the complete analysis pipeline, including the pre-processing steps and the optimized set of features, can be integrated into a software running on a detection device (e.g. device 10). In real-time usage, a continuous stream of measurements is analyzed (Figure 5), with the users’ self-motion and response closing the loop of action, to feedback, to response. It is appreciated that certain features of the invention, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the invention, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable subcombination. Although the invention has been described in conjunction with specific embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, it is intended to embrace all such alternatives, modifications and variations that fall within the spirit and broad scope of the appended claims. All publications, patents and patent applications mentioned in this specification are herein incorporated in their entirety by reference into the specification, to the same extent as if each individual publication, patent or patent application was specifically and individually indicated to be incorporated herein by reference. In addition, citation or identification of any reference in this application shall not be construed as an admission that such reference is available as prior art to the present invention.

In addition, any priority document(s) of this application is/are hereby incorporated herein by reference in its/their entirety.

Claims

WHAT IS CLAIMED IS:

1. A posture detection device comprising:

(a) a device body attachable to an individual, said device body including a sensor for sensing a body self-motion; and

(b) a processing module for:

(i) processing sensor data related to said body self-motion over time to derive spatiotemporal patterns of movement; and

(ii) classifying each of said spatiotemporal patterns to a posture to thereby detect a posture of said individual.

2. The device of claim 1, further comprising a component for providing feedback to the individual.

3. The device of claim 2, wherein said feedback is tactile and/or auditory and/or visual.

4. The device of claim 2, wherein said feedback is a posture correction alert.

5. The device of claim 1, further comprising a wireless communication module for communicating with a remote device.

6. The device of claim 5, wherein said remote device is a smartphone or a computer.

7. The device of claim 6, wherein said remote device provides tactile and/or auditory and/or visual feedback to the individual.

8. The device of claim 7, wherein said feedback indicates a posture and/or posture correction.

9. The device of claim 2, wherein said feedback forms a part of a training session for training the individual.

10. The device of claim 1, wherein said sensor is a strain sensor.

11. The device of claim 1, wherein said sensor is a 3- or 6-axis sensor.

12. The device of claim 1, wherein said processing module classifies said spatiotemporal patterns to detect a slouch posture or a lean posture.

13. The device of claim 1, wherein said sensor continuously collects body self-motion data.

14. The device of claim 1, wherein said sensor measures self-motion via acceleration and/or angular velocity.

15. The device of claim 13, wherein said processing module extracts a set of spatial- temporal features from said body self-motion data.

16. The device of claim 15, wherein said processing module processes said spatial- temporal features to extract said spatiotemporal patterns at least a portion of which represent state transition probabilities.

17. The device of claim 16, wherein said processing module utilizes said state transition probabilities for posture classification.

18. The device of claim 1, wherein said a processing module utilizes an upright posture of the individual as a reference for (ii).

19. A posture detection system comprising:

(a) a device attachable to an individual, said device including a sensor for sensing a body self-motion; and

(b) a processing module being in communication with said device, said processing module being for:

20. The system of claim 19, wherein said processing module forms a part of a user device.

21. The system of claim 20, wherein said user device is a smartphone.