WO2021045616A1

WO2021045616A1 - A computer-implemented method and an apparatus for determining body positions and/or sleep stages of a person during sleep.

Info

Publication number: WO2021045616A1
Application number: PCT/NL2020/050541
Authority: WO
Inventors: Ruud Johannes Gerardus VAN SLOUN; Sebastiaan OVEREEM; Merel Marietje VAN GILST
Original assignee: Technische Universiteit Eindhoven
Priority date: 2019-09-03
Filing date: 2020-09-03
Publication date: 2021-03-11

Abstract

The present disclosure relates to a method and an apparatus for body positions and/or sleep stages of a person during sleep. The method comprising the steps of: i) receiving from one or more sensors a sequence of pressure- or displacement signals over time in response to movements of a part of the body of the person during sleep; ii) generating one or multiple time-frequency representations of the pressure- or displacement signals through time-frequency transformation, and iii) analysing the time-frequency representations using one or more machine learning algorithms for determining a body position and/or a sleep stage of the person.

Description

TITLE

A computer-implemented method and an apparatus for determining body positions and/or sleep stages of a person during sleep.

FIELD OF THE INVENTION

The present disclosure relates to a method and an apparatus for body positions and/or sleep stages of a person during sleep. The present disclosure also pertains to a computer program or product comprising instructions which, when the program is executed by a computer, cause the computer to carry out steps of the computer implemented method according to the disclosure. Similarly, the present disclosure also pertains to a computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out steps of the computer implemented method according to the disclosure.

BACKGROUND OF THE INVENTION

Sleep impairment and sleep disorders have been associated with detrimental health consequences including depression, heart attack and stroke. Automated detection of body posture during sleep is relevant for determining the quantity and quality of a sleep pattern and identifying common sleeping disorders such as sleep apnoea, restless leg syndrome, periodic limb movement and carpal tunnel syndrome. The ability to unobtrusively measure sleep stages and body orientation would be an important asset for sleep diagnostics, enabling use of monitoring and diagnostic systems in a home environment.

Polysomnography (PSG) is the most commonly used method for the objective assessment in sleep studies, due to its high accuracy and wealth of information in sleep registrations. It measures many different body functions including brain activity (EEG), eye movements (EOG), muscle activity (EMG) and heart rhythm (ECG) during sleep and PSG might even include video.

There are however some limitations to PSG techniques. Firstly, obtaining measurements is labour intensive and expensive due to requiring specialized equipment, making PSG a difficult to transfer technology into a home environment. Secondly, the patient needs to stay overnight in an often unfamiliar research facility, with many sensors attached to their head and body. This may cause stress or anxiety and affect the measurement results. Non-invasive methods of monitoring sleep body posture include camera-based options, some even able to classify sleep posture with high accuracy using 3D-imaging. The main concern with camera-based approaches is privacy related concerns.

In this disclosure a computer-implemented method and an apparatus are proposed for determining body positions and/or sleep stages of a person during sleep under test and sleep conditions which are unobtrusive to the person (or patient) under test, using the automatic analysis of photoplethysmography data using sensors and machine learning.

BRIEF SUMMARY OF THE INVENTION

In a first example, a method is proposed, in particular a computer- implemented method, capable of determining body positions and/or sleep stages of a person during sleep. Accordingly, the method comprises the steps of: i) receiving from one or more sensors a sequence of pressure- or displacement signals over time in response to movements of a part of the body of the person during sleep; ii) generating one or multiple time-frequency representations of the pressure- or displacement signals through time-frequency transformation, and iii) analysing the time-frequency representations using one or more machine learning algorithms for determining a body position and/or a sleep stage of the person.

Additionally, the computer-implemented method comprises the step of outputting the body position and/or the sleep stage of the person being determined.

Herewith, a technique is presented allowing the acquisition and processing of large amounts of data in real-time, and providing a more efficient, accurate, that is less susceptible to noise, result as to the determination of the body position (or body positions) and/or a sleep stage (or sleep stages) of the person under sleep diagnosis.

In several advantageous examples, the time-frequency transformation can be selected from the group exemplified by but not limited to short-time Fourier transform, wavelet transform, filter bank, or discrete cosine transform. This results in an analysis in the time-frequency domain, as exposing characteristic time-frequency patterns that can be well detected by machine learning algorithms.

Preferably, the one or more machine learning algorithms are selected from the group exemplified by but not limited to an artificial neural network, a decision tree, a regression model, a k-nearest neighbour model, a partial least squares model, a support vector machine, or an ensemble of the models that are integrated to define an algorithm.

In yet, another advantageous example, the machine learning algorithm is a computer-implemented artificial neural network, and wherein step i) is preceded by the steps of: a) training the computer-implemented artificial neural network with polysomnography input data characterizing a training sequence of pressure- or displacement signals over time with known body positions and/or sleep stages; b) applying to the computer-implemented artificial neural network polysomnography input data characterizing at least a test sequence of pressure- or displacement signals over time with an unknown body position and/or sleep stage; c) analysing each applied test sequence of pressure- or displacement signals to generate a predicted body position and/or sleep stage for each test sequence of pressure- or displacement signals.

Optionally, during training the computer-implemented artificial neural network, the method may further encompass the outputting of the predicted body position and/or sleep stage in order to verify the accuracy and allow adaptation of the training steps a)-b)-c).

Using a trained artificial neural network allows for a proper, correct determination of a sleep-stage and/or body orientation/pose of a person whilst sleeping. The artificial neural network is trained on the basis of machine learning techniques, in particular deep learning, for example by back propagation. The inputted training data used may be labels annotated by clinical experts based on complete or partial polysomnography.

In order to allow for a scalable execution of identical operations over massive data streams such as a continuous PSG data stream, the method further comprises the sub-steps of monitoring/splitting the sequence of pressure- or displacement signals over time and/or the training sequence of pressure- or displacement signals over time and/or the test sequence of pressure- or displacement signals over time in subsequent, consecutive data sequence frames of a certain time-frame length, and of generating the one or multiple time-frequency representations by time- frequency transformation of each of the subsequent, consecutive data sequence frames. Splitting the several data streams, either during the actual performance of the method steps according to the disclosure, and/or during the training and testing phase of the computer- implemented artificial neural network into data sequence frames of limited time length and transforming and analysing these data sequence frames in accordance with the several steps of the computer-implemented method according to the disclosure improves the accuracy of the method and apparatus.

In an example of an apparatus for determining body positions and/or sleep stages of a person during sleep, the apparatus comprises: a receiving unit arranged for receiving from one or more sensors a sequence of pressure- or displacement signals over time in response to movements of a part of the body of the person during sleep; at least one processing unit arranged for generating one or multiple time- frequency representations of the pressure- or displacement signals through time-frequency transformation, and for analysing the time-frequency representations using one or more machine learning algorithms for determining a body position and/or a sleep stage of the person.

Herewith, is it avoided to use expensive and specialized equipment requiring labour intensive measurements, which specialized equipment are difficult if not impossible to be transferred or converted into a home environment applicable solution. The apparatus according to an example of the disclosure can be implemented in a home environment with readily simple sensing equipment, limiting the burden to patients.

In particular, an output unit is used configured to output the body position and/or the sleep stage of the person being determined.

As to the time-frequency transformation, in an example it is selected from the group exemplified by but not limited to short-time Fourier transform, wavelet transform, filter bank, or discrete cosine transform, whereas the one or more machine learning algorithms are selected from the group exemplified by but not limited to an artificial neural network, a decision tree, a regression model, a k-nearest neighbour model, a partial least squares model, a support vector machine, or an ensemble of the models that are integrated to define an algorithm.

In a preferred example, the apparatus comprises at least one sensor capable outputting the sequence of pressure- or displacement signals over time in response to movements of a part of the body of the person during sleep. In an useful embodiment, the sequence of pressure- or displacement signals over time being inputted to the time- frequency transformation is a sequence of pressure- or displacement signals obtained from a single sensor. In yet another example, the apparatus implements multiple sensors positioned at different locations relative to the body of the person during sleep, for example a sensor near the shoulder region, a sensor near the abdominal region and a sensor near the hip region.

In particular the apparatus further comprises a mattress provided with the one or more sensors. This allows for a simple and versatile implementation of the method and apparatus according to the disclosure in a home environment with limited adjustments to that environment and thus limited burden to the patient or person under test.

Preferably, the apparatus further comprises: a training unit arranged for training the computer-implemented artificial neural network with polysomnography input data characterizing a training sequence of pressure- or displacement signals over time with known body positions and/or sleep stages and for applying to the computer-implemented artificial neural network polysomnography input data characterizing at least a test sequence of pressure- or displacement signals over time with an unknown body position and/or sleep stage and for analysing each applied test sequence of pressure- or displacement signals to generate a predicted body position and/or sleep stage for each test sequence of pressure- or displacement signals, and for outputting the predicted body position and/or sleep stage.

Using a trained artificial neural network allows for a proper, correct determination of a sleep-stage and/or body orientation/pose of a person whilst sleeping. The artificial neural network is trained on the basis of machine learning techniques, in particular deep learning, for example by back propagation.

In a further example, the receiving unit or the at least one processing unit is further configured to monitor the sequence of pressure- or displacement signals over time and/or the training sequence of pressure- or displacement signals over time and/or the test sequence of pressure- or displacement signals over time in subsequent, consecutive data sequence frames of a certain time-frame length, and wherein the at least one processing unit generates the one or multiple time-frequency representations by time-frequency transformation of each of the subsequent, consecutive data sequence frames. This allows for a scalable execution of identical operations over massive data streams such as a continuous PSG data stream, reducing the probability of errors and improving the accuracy as to the determining body positions and/or sleep stages of the person during sleep.

In other advantageous embodiments, the method of the present disclosure can be embodied in a computer program or product, which computer program or product comprises computer-coded instructions which, when the computer program or product program is executed by a computer, such as a laptop or computer, cause the computer to carry out steps of the computer implemented method disclosed herein.

In a particular embodiment, a computer-readable storage medium is proposed comprising computer-coded instructions stored therein, which computer-coded instructions, when executed by a computer, cause the computer to carry out steps of the computer implemented method disclosed in this application. Such computer-readable storage medium can be a (solid-state) hard drive, or a USB drive, or a (digital) optical disc.

DESCRIPTION OF THE DRAWINGS

The invention will now be discussed with reference to the drawings, which show in:

Figure 1 an example of an apparatus according to the disclosure implementing an example of a method according to the disclosure;

Figure 2 a detail of an example of an apparatus according to the disclosure;

Figure 3 an example of the distribution of sequence of pressure- or displacement signals inputted to an example of an apparatus according to the disclosure implementing an example of a method according to the disclosure

Figure 4 another example of an apparatus according to the disclosure implementing an example of a method according to the disclosure;

Figure 5 data pertaining to automatic sleep stages of two subjects (left, right) using an example of the disclosure (middle row) compared to expert-annotated PSG (bottom row);

Figure 6 data pertaining to the automatic body position detection for two subjects (left, right) using an example of the disclosure (middle row) compared to expert- annotated PSG (bottom row);

Figure 7 a sample of a sequence of pressure- or displacement signals being sensed (top) and a sample of the corresponding normalized spectrogram after time- frequency transformation;

Figure 8 an example of single sensor accuracy and Cohen’s Kappa of DenseNet models with variation in model depth;

Figure 9 an example of a three sensor accuracy and Cohen’s Kappa of DenseNet models with variation in model depth; Figure 10 an example of a full night of inputted data (A), corresponding spectrogram (B) and 5-class body posture predictions using an example of the method according to the disclosure on one test subject (C).

DETAILED DESCRIPTION OF THE INVENTION

For a proper understanding of the invention, in the detailed description below corresponding elements or parts of the invention will be denoted with identical reference numerals in the drawings.

As outlined in this disclosure the invention relates to a computer- implemented method for determining body positions and/or sleep stages of a person during sleep, as well as an apparatus for determining body positions and/or sleep stages of a person during sleep. Therefor it is noted that in this disclosure, the computer-implemented method as outlined can be used for determining body positions of a person during sleep, or for determining sleep stages of a person during sleep, or a combination of both. The same applies to the apparatus, it can be used for determining body positions of a person during sleep, or for determining sleep stages of a person during sleep, or a combination of both.

An example of such apparatus implementing an example of such method is shown in Figures 1 and 2.

As outlined in the introduction of this application sleep impairment and sleep disorders have been associated with detrimental health consequences including depression, heart attack and stroke. Automated detection of body posture during sleep is relevant for determining the quantity and quality of a sleep pattern and identifying common sleeping disorders such as sleep apnoea, restless leg syndrome, periodic limb movement and carpal tunnel syndrome. The ability to unobtrusively measure sleep stages and body orientation would be an important asset for sleep diagnostics, enabling use of monitoring and diagnostic systems in a home environment.

Polysomnography (PSG) is the most commonly used method for the objective assessment in sleep studies, due to its high accuracy and wealth of information in sleep registrations. It measures many different body functions including brain activity (EEG), eye movements (EOG), muscle activity (EMG) and heart rhythm (ECG) during sleep and PSG might even include video. Due to the limitations to known PSG techniques, the present disclosure presents a technique allowing the acquisition and processing of large amounts of data in real-time, and providing a more efficient, accurate, that is less susceptible to noise, result as to the determination of the body position (or body positions) and/or a sleep stage (or sleep stages) of the person under sleep diagnosis. In particular, the disclosure enables an unobtrusive estimation of body positions and/or sleep stages through the extraction and automatic interpretation of spectral information from one or more sensors, that for example can be embedded in or placed on a mattress or mattress pad.

The example of the apparatus according to the disclosure and depicted in Figure 1 is denoted with reference numeral 100. In a detail outlined in Figure 2, the apparatus 100 may comprise a mattress or mattress pad (denoted with reference numeral 10), on which a person (or patient) 5 under test to be placed under sleeping conditions.

The apparatus 100 may comprise one or more sensors 11-12-13 embedded in or placed under or on the mattress 10. The mattress 10 can be placed on a bed of the patient 5 in his or her own home environment, thus avoiding stress or anxiety which might affect the accuracy and reliability of the measurement results, which is often the case when such measurements are performed in unfamiliar research facilities, such as hospitals etc.

In an useful example of the disclosure the apparatus may comprise a single sensor mounted in or on or under the mattress 10 or bed, which single sensor (for example sensor 11 in Figure 2) detects a sequence of pressure- or displacement signals over time, which data sequence is being inputted to the apparatus according to the disclosure implementing the method according to the disclosure.

In another embodiment, the sequence of pressure- or displacement signals generated by the person 5 during sleep are obtained from multiple distinct sensors, which are placed at different distinct locations. An example of such configuration is shown in Figure 2, wherein the apparatus 100 comprises a mattress 10 provided with multiple sensors positioned at different locations relative to the body of the person 5 during sleep, for example a sensor 11 near the shoulder region, a sensor 12 near the abdominal region and a sensor 13 near the hip region. However also other configurations of placement of multiple sensors are possible, e.g. one sensor under the head position and/or leg/feet positions.

Figure 3 shows a diagram depicting the number of data sequences collected, which can be attributed to different body posture orientations of a person whilst sleeping. Also, in Figure 2 the sensors 11-12-13 are being depicted as band or strip like pressure sensitive elements or sensors, which are incorporated in the mattress 10, or are strapped around the mattress 10 perpendicular to the longitudinal orientation of the mattress 10. Preferably, these sensors are covered by mattress cover and/or sheet, thus shielding them from the vision of the person 5 and reducing stress or anxiety. In another example the sensors can also be placed in a diagonal orientation relative to the longitudinal orientation of the mattress 10, thus providing additional information about the person’s movements and orientations during the sleeping stage.

The single or multiple sensors 11-12-13 can be pressure sensitive sensors capable of sensing a sequence of pressure- or displacement signals, which are induced by the person during sleep.

According to a step of the computer-implemented method, the apparatus 100 comprises a receiving unit 101 arranged for receiving from the one or more sensors 11-12- 13 a sequence of pressure- or displacement signals over time in response to movements of a part of the body of the person 5 during sleep.

In a further step of the computer-implemented method according to the disclosure, the apparatus 100 comprises at least one processing unit 102 arranged for generating one or multiple time-frequency representations of the pressure- or displacement signals through at least one time-frequency transformation. The at least one processing unit 102 may be connected with each of the sensors 11-12-13 through suitable wiring, or through a wireless data-communication interface operating in accordance with a network protocol for exchanging data between the sensors and the processing unit 102, such as designated ZigBee™, Bluetooth™, or Wi-Fi based protocols for wireless networks.

The time-frequency transformation of the sequences of pressure- or displacement signals obtained from the sensors is selected from the group exemplified by but not limited to short-time Fourier transform, wavelet transform, filter bank, or discrete cosine transform.

The at least one processing unit is further arranged for analysing the time- frequency representations using one or more machine learning algorithms for determining a body position and/or a sleep stage of the person 5. The at least one processing unit implementing the one or more machine learning algorithms is denoted with reference numeral 103 in Figure 1 and can be a processing unit separate from the processing unit 102 or incorporated with the processing unit 102, thus forming a single processing unit performing several distinct steps of the computer-implemented method according to the disclosure.

The body positions and/or sleep stages of the person 5 as determined by the one or more machine learning algorithms implemented in and executed by the processing unit 103 may be outputted or displayed by means of an output unit 104, for example a computer or laptop display, or a separate display unit.

The one or more machine learning algorithms can be computer-code executed by the processing unit 103 and said machine learning algorithms can be selected from the group exemplified by but not limited to an artificial neural network, a decision tree, a regression model, a k-nearest neighbour model, a partial least squares model, a support vector machine, or an ensemble of the models that are integrated to define an algorithm.

Preferably, one of the machine learning algorithm being used in the computer-implemented method and apparatus according to the disclosure, is a computer- implemented artificial neural network. Such computer-implemented artificial neural network (or deep neural network) needs to be trained, before it is capable of successfully determine a sleep-stage and/or body orientation/pose of a person under sleep.

Therefore, according to a further example of the disclosure the apparatus comprises a training unit capable of training the computer-implemented artificial neural network.

In particular the training unit trains the computer-implemented artificial neural network with polysomnography input data, which data characterize a training sequence of pressure- or displacement signals over time with known body positions and/or sleep stages. Additionally, the training unit applies to the computer-implemented artificial neural network polysomnography input data characterizing at least a test sequence of pressure- or displacement signals over time with an unknown body position and/or sleep stage and is arranged in analysing each applied test sequence of pressure- or displacement signals in order to generate a predicted body position and/or sleep stage for each of the test sequences of pressure- or displacement signals.

Additionally, the training unit may output the predicted body position and/or sleep stage.

In particular, the neural network is used to estimate sleep stages and/or body positions of a person whilst sleeping from spectrograms, for example obtained through a time-frequency transformation such as (short-time) Fourier transform, of pressure sensor data being sensed with the sensors 11-12-13 in a mattress 5, placed in a bed (or from a single pressure sensor as outlined before).

A person’s body undergoes during sleep four distinct sleep stages, which stages may consist of both rapid-eye-movement (REM) or non-rapid eye movement (NREM) sleep. In general, sleep is broken down into five phases from going to sleep till the moment that the person awakes. These five stages are identified as ‘wake’, ‘NT, ‘N2’, ‘N3’, and ‘R’. During sleep a person’s body will cycle on average four to five or six times through these stages, with an average of 90-100 minutes per cycle. The NREM sleep stages are N1 to N3, with the person under sleep progressively going into deeper sleep, and with the larger part of the sleep time being spent in the N2 stage.

The signal spectrograms (e.g. through short-time Fourier transform) are calculated to reveal time-frequency phenomena, and by applying a trained deep neural network directly on the signal spectrograms distinct sleep stages can be determined, such as wake, N1, N2, N3, REM) and body orientations (left, right, belly, back, upright, out of bed).

An example of an apparatus implementing an example of a computer- implemented method, wherein the machine learning algorithm is a computer-implemented artificial neural network is shown in Figure 4. In the example of Figure 4 the receiving unit 101 receives sequences of pressure- or displacement signals over time in response to movements of a part of the body of the person 5 during sleep, for example from the sensors 11-12-13 depicted in Figure 2. The sequences of pressure- or displacement signals over time are time-frequency transformed by the processing unit 102 by means of short-time Fourier transformation.

The further processing unit 103 implements a machine learning algorithm configured as a computer-implemented artificial neural network, which is based on a collection of connected artificial neurons, also called nodes, wherein each connection (also called edge) can transmit a signal from one node to another. Each artificial neuron receiving a signal may process it and transfer it to further artificial neurons connected to it.

In a useful embodiment, the artificial neurons of the artificial neural network implemented in the processing unit 103 are arranged in layers. The signals inputted in the in the processing unit 103, i.e. the intensity values of the 2D or 3D matrix after the time- frequency transformation performed by the processing unit 102, travel from the first layer, also termed the input layer, to the last layer, the output layer. In useful embodiments, the neural network a feed-forward network. The neural network preferably comprises several layers, including hidden layers, and is thus, preferably, a deep network.

The neural network can be trained on the basis of machine learning techniques, in particular deep learning, for example by back propagation. The training data used may be labels annotated by clinical experts based on complete or partial polysomnography data. The training data used may also be labels obtained through automatic annotation by a computer based on complete or partial polysomnography. In an embodiment the neural network is trained to minimize the categorical cross-entropy between its outputs and the labels.

According to a preferred embodiment, the trained artificial neural network comprises at least one two-dimensional (2D) convolutional layer (CONV). A 2D convolutional layer applies one or more filter kernels over the entire input layer, so that the neurons inside the layer are connected to only a small region of the layer before it. Each filter kernel is convolved across the 2D of the input volume, producing a feature map of that filter. Stacking the feature maps for all filter kernels along the depth dimension forms the full output volume of the convolutional layer. In a useful embodiment, the convolutional layer or layers of the neural network comprise convolutional filter kernels having a size of 3x3, or5x5. In another useful embodiment, the trained neural network comprises at least one three-dimensional (3D) convolutional filter.

In an example, the depth of each convolutional layer (CONV) is between 8 and 64, preferably 16-32, most preferred 32, meaning that 32 different feature maps, produced by different filter kernels are part of each convolutional layer. Such convolutional layer having a depth of 32 different feature maps is disclosed in the example of Figure 4. If two convolutional layers follow one another as shown in Figure 4, preferably their depth remains the same, wherein each of the 32 feature maps of the subsequent layer are produced by adding together the result of convoluting each of the feature maps of the previous layer with a separately trained filter kernel.

Accordingly, a deep convolutional layer having a depth of 32, and connected to a subsequent layer having a depth of 32, requires 32x32 filter kernels to train.

According to a further useful embodiment, the trained neural network comprises at least one pooling (down-sampling) layer, preferably a 2x2 max-pooling operation, which reduces a kernel of four pixels to one by projecting only the highest value onto the subsequent layer. According to a further example, the neural network comprises at least one skip or residual connection from the input of a layer to the output of another layer. The skip or residual connections can be achieved either by element-wise summation, or through concatenation of the feature maps.

Also, the trained neural network may comprise at least one recurrent layer, preferably a long-short-term memory (LSTM) layer. In useful examples, such LSTM layer contains 16-256 hidden units, i.e. the dimensionality of its hidden state. In a preferred embodiment, it contains 64 hidden units as depicted in the example of Figure 4.

Additionally, the trained neural network may comprise at least one fully connected layer.

Figures 5-10 depict several data pertaining to an example of the computer- implemented method and apparatus according to the disclosure, as described above in relation to Figure 1-2 and 3.

The sensors 11-12-13 for detecting the movements of a part of the body of the person 5 during sleep generate a fluctuating signal over time, which signal is both periodic and aperiodic in not very well defined occurrence patterns. This complicates the task of body posture classification. However, by transforming this fluctuating signal (the sequence of pressure- or displacement signals over time) using a time-frequency transformation (e.g. short-term Fourier transformation) a spectrogram is obtained, which represents the sensor data two-dimensionally as a function of frequency and time. Such sensor data and its corresponding spectrogram is for example depicted in Figure 3.

The data acquisition can be performed with the apparatus according to Figure 1 and its detail in Figure 2 using three pressure sensors 11-12-13 located under the shoulder region, under the abdominal region and under the hip region. The sensor output data (= the sequence of pressure- or displacement signals over time) is logged at a sampling frequency of 100 Hz to a storage unit (not shown) of the apparatus 100. The storage unit of the apparatus 100 can be a hard disc unit or a solid state storage unit, or a removable storage device such as a USB storage unit mounted in a laptop or computer implementing computer code performing the computer-implemented method according to the disclosure.

The apparatus according to the disclosure can be (part of) a laptop or computer, which may be implementing computer code performing the computer- implemented method according to the disclosure. For example, the method of the present disclosure can be embodied in a computer program or product, which computer program or product comprises computer-coded instructions which, when the computer program or product program is executed by a computer, such as a laptop or computer, cause the computer to carry out steps of the computer implemented method disclosed herein.

The sensor output data is representative for the global change in pressure exerted by the person 5 on the sensors 11-12-13 during sleep.

The sensor output data contain sleeping data of several persons (subjects) with a variety of sleeping disorders, which subjects all undergo a PSG, resulting in individual data recordings for each person, each PSG data recording consisting of a single night of patient sleep. These subjects undergo a PSG which is part of standard clinical care in a dedicated sleeping centre. Full video-PSG will be performed according to the standards of the American Academy of Sleep Medicine. The multiple PSG datasets are split up into two parts, one set (approx two-third of the PSG recordings) is used for training the artificial neural network, and one third is used as test and validation set.

During the recording of each PSG dataset, the continuous PSG data sequence being detected over time (and in real time) by the sensors (one or multiple sensors 11-12-13) is continuously monitored per time frame of a certain time frame length. The time frame can have for example a time frame length of 30 seconds or one minute (or even shorter, for example 20 seconds). The continuous PSG data sequence being monitored during that specific time frame (for example having a 30 second frame length) is labelled (or marked) for two scenario’s. The marking pertains to labelling the PSG data sequence monitored during that specific time frame as indicative to one of the four (left/right, back, front or up- right) or one of the five (left, right, back, front or upright) different body postures (positions) of the patient 5.

The classification of specific PSG data sequences, each detected during a subsequent time frames (e.g. each of a 30 second time frame length) and labelled as or corresponding to a certain body posture allows for a proper training of the artificial neural network. For training the computer-implemented artificial neural network the PSG training data sequences, each detected during subsequent, consecutive time frames and labelled with one of the four (or five) corresponding body postures (positions) serve as the polysomnography input data characterizing a training sequence of pressure- or displacement signals over time with known body positions and/or sleep stages.

The monitoring or splitting per time frame of a certain time frame length of a continuous data sequence over time in consecutive time frames of a limited time length, allows for a scalable execution of identical operations over massive data streams as PSG data. When training the computer-implemented artificial neural network the overall, continuous PSG training data sequence is monitored or split in subsequent, consecutive data sequence frames of a certain time-frame length. Each (training) data sequence frame is transformed or generated into a subsequent, corresponding time-frequency representation by time-frequency transformation and characterized (marked or labelled) with one of the four or five body positions, which body position has been identified for example by means of video.

Each marked or labelled time-frequency representation or transformation represents a (training) data sequence frame of pressure- or displacement signals of a limited time length and with a known body position and/or sleep stage as identified during that time-frame length and is presented to the computer-implemented artificial neural network for training purposes.

Similarly, for testing the computer-implemented artificial neural network, a PSG test data sequence is used, which test data sequence of pressure- or displacement signals over time is monitored or split in subsequent, consecutive test data sequence frames of a certain, preferably similar time-frame length as during the training stage. Each (test) data sequence frame is similarly transformed or generated into a subsequent, corresponding time-frequency representation by time-frequency transformation. Such time- frequency representation or transformation represents a (test) data sequence frame of pressure- or displacement signals of a limited time length with an unknown body position and/or sleep stage.

The time-frequency representation of the (test) data sequence frame is analysed by the computer-implemented artificial neural network, in order to generate a predicted body position and/or sleep stage for that test data sequence frame. If the computer-implemented artificial neural network is well trained with a large number of (training) data sequence frames of pressure- or displacement signals of a limited time length and with a known body position and/or sleep stage, the accuracy of the correct prediction of the body position and/or sleep stage for that test data sequence frame can be improved. Likewise, when analysing an actual PSG data sequence of pressure- or displacement signals over time in response to movements of a part of the body of the person during sleep with a well-trained computer-implemented artificial neural network, the actual monitored PSG data sequence of pressure- or displacement signals over time is monitored or split in subsequent, consecutive data sequence frames of a time-frame length, preferably of the same time length as used during training and/or testing the computer- implemented artificial neural network. Each (actual) data sequence frame has likewise an unknown body position and/or sleep stage, and is in a similar manner also transformed or generated into a corresponding time-frequency representation by means of time-frequency transformation described in this disclosure.

The thus generated time-frequency representation having an unknown body position and/or sleep stage is analysed by the one or more machine learning algorithms, such as a computer-implemented artificial neural network, in order to determine the body position and/or the sleep stage of the person during that actual time-frame length of his or her sleeping time.

According to the disclosure, in an example of the apparatus 100, the receiving unit 101 or at least one of the processing units 102 or 103 are configured to monitor the sequence of pressure- or displacement signals over time in subsequent, consecutive data sequence frames of a certain time-frame length, e.g. of a time-frame length of 20, 30 or 60 seconds.

In an example, the apparatus 100 may comprise a further processing unit (not shown), which monitors the sequence of pressure- or displacement signals over time being detected by the sensors 11-12-13 and received by the receiving unit 101 in the subsequent, consecutive data sequence frames of the certain time-frame length, and feeds the separate, consecutive data sequence frames of the certain time-frame length to the processing unit 102, where the one or multiple time-frequency representations are generated based on the separate, consecutive data sequence frames by means of time- frequency transformation.

In another example, both the monitoring of the sequence of pressure- or displacement signals over time being detected by the sensors 11-12-13 and received by the receiving unit 101 in the subsequent, consecutive data sequence frames of the certain time-frame length as well as the generation of the one or multiple time-frequency representations based on the separate, consecutive data sequence frames by means of time-frequency transformation are performed by one, single processing unit 102. In a similar fashion, during training and testing the computer-implemented artificial neural network 103 being implemented by the apparatus 100 according to the disclosure, the monitoring in subsequent, consecutive data sequence frames of a certain time-frame length of both the training sequence of pressure- or displacement signals over time and the test sequence of pressure- or displacement signals over time and the subsequent generation of the one or multiple time-frequency representations based on the separate, consecutive training and/or test data sequence frames by means of time- frequency transformation are performed by either the receiving unit 101, or the processing unit 102, or by a separate processing unit. The posture/position of the respective patient 5 may be determined by the combination of data sequences of video-PSG, which is performed in accordance to the standards of the American Academy of Sleep Medicine in combination with a body position sensor attached to the patient 5 or in the mattress 10. After the label (classification or marking) indicative to one of the four (or five) body postures mentioned above is assigned to the PSG data sequence per each subsequent time frame, the categorical features indicating that one of the four (or five) possible body postures are one-hot encoded to speed up the neural network.

The body posture classifications were imbalanced. The imbalanced class distribution of the validation set depicting the distribution of the body postures (positions) labelled or classified to the number of the PSG data sequences per time frame is shown in Figure 4, which depicts the classification distribution of five body positions. Class imbalance can have detrimental effects in a classification problem, especially when some classes have a significantly higher number of examples in the training set than other classes. Class imbalance affects convergence during the training phase and generalization of a model on a test set. New training data batches are generated randomly from the training dataset. From each class an equal number of random samples is chosen in a training batch to reduce the effects of class-imbalance.

The PSG sensor data is transformed using a short-time Fourier transform (STFT), to represent the signal as a function of frequency and time in a spectrogram. A STFT is used to determine the frequency and phase content as the signal changes over time. The procedure to compute a spectrogram is to split up a longer time signal into shorter segments and compute the Fourier transform separately for every shorter segment.

Before the spectrogram data is used in the model the data is normalized to speed up the learning process of the neural network. Sensor data is normalized between zero and one as described in the following equation: X_normaiized = [(x-min(x)) / (max(x)-min(x)]

, where x is the data input value. Normalization is done on the basis of a full night of data, which ensures no relevant information is lost, but patient-specific features such as average sensor value, which is partly determined by patient weight are disregarded.

Figure 8 and 9 depict the (distribution of the) results of the posture classifications assigned to the subsequent PSG data sequences measured for each specific time frame (e.g. of a 30 second frame length) for training the artificial neural network to the training dataset for either an example (Figure 8) of an apparatus according to the disclosure implementing one, single sensor and an example (Figure 9) of the apparatus according to the disclosure implementing three sensors as in Figure 2. For the single sensor approach (Figure 8), only the detection results of the shoulder sensor (reference numeral 11 in Figure 2) are depicted.

Two metrics were used to measure the performance of the classifier, accuracy and Cohen’s Kappa. Accuracy measures the percentage of agreement with the ground truth, while Cohen’s Kappa coefficient k takes into account the possibility of agreement occurring by chance making this a more robust metric. In the single sensor embodiment the data shows an improvement in accuracy of 0.0140 and an improvement in terms of Cohen’s Kappa of 0.0210 compared to the best single sensor embodiment. The three sensor ensemble embodiment shows an improvement in accuracy of 0.0480 and an improvement in terms of Cohen’s Kappa of 0.0650 compared to the best three sensor individual model.

An illustrative example of a full night PSG data recording using the three sensor embodiment is shown in Figure 10, displaying as input data the sequence of pressure- or displacement signals over time in response to movements of a part of the body of the person 5 during sleep as detected by the sensors 11-12-13 and received by the receiving unit 101 , the spectrogram or time-frequency transformation generated by the processing unit 102 and the body posture prediction by the artificial neural network 103.

Claims

1. A computer-implemented method for determining body positions and/or sleep stages of a person during sleep, the method comprising the steps of: i) receiving from one or more sensors a sequence of pressure- or displacement signals over time in response to movements of a part of the body of the person during sleep; ii) generating one or multiple time-frequency representations of the pressure- or displacement signals by time-frequency transformation, and iii) analysing the time-frequency representations using one or more machine learning algorithms for determining a body position and/or a sleep stage of the person.

2. The computer-implemented method of claim 1, wherein the time-frequency transformation is selected from the group exemplified by but not limited to short-time Fourier transform, wavelet transform, filter bank, or discrete cosine transform.

3. The computer-implemented method of claim 1 or 2, wherein the one or more machine learning algorithms are selected from the group exemplified by but not limited to an artificial neural network, a decision tree, a regression model, a k-nearest neighbour model, a partial least squares model, a support vector machine, or an ensemble of the models that are integrated to define an algorithm.

4. The computer-implemented method according to any one of the preceding claims, wherein the machine learning algorithm is a computer-implemented artificial neural network, and wherein step i) is preceded by the steps of: a) training the computer-implemented artificial neural network with polysomnography input data characterizing a training sequence of pressure- or displacement signals over time with known body positions and/or sleep stages; b) applying to the computer-implemented artificial neural network polysomnography input data characterizing at least a test sequence of pressure- or displacement signals over time with an unknown body position and/or sleep stage; c) analysing each applied test sequence of pressure- or displacement signals to generate a predicted body position and/or sleep stage for each test sequence of pressure- or displacement signals.

5. The computer-implemented method according to according to 4, wherein the computer-implemented artificial neural network is a deep neural network comprising 2D, and/or 3D convolutional layers, and/or recurrent layers.

6. The computer-implemented method according to any one of the preceding claims, further comprises the sub-steps of monitoring the sequence of pressure- or displacement signals over time and/or the training sequence of pressure- or displacement signals over time and/or the test sequence of pressure- or displacement signals over time in subsequent, consecutive data sequence frames of a certain time-frame length, and generating the one or multiple time-frequency representations by time- frequency transformation of each of the subsequent, consecutive data sequence frames.

7. An apparatus for determining body positions and/or sleep stages of a person during sleep, the apparatus comprising: a receiving unit configured to receive from one or more sensors a sequence of pressure- or displacement signals over time in response to movements of a part of the body of the person during sleep; at least one processing unit configured to generate one or multiple time- frequency representations of the pressure- or displacement signals through time-frequency transformation, and for analysing the time-frequency representations using one or more machine learning algorithms for determining a body position and/or a sleep stage of the person.

8. The apparatus according to claim 7, further comprising: an output unit configured to output the body position and/or the sleep stage of the person being determined.

9. The apparatus according to claim 7 or 8, where the time-frequency transformation is selected from the group exemplified by but not limited to short-time Fourier transform, wavelet transform, filter bank, or discrete cosine transform.

10. The apparatus according to any one or more of the claims 7-9, wherein the one or more machine learning algorithms are selected from the group exemplified by but not limited to an artificial neural network, a decision tree, a regression model, a k-nearest neighbour model, a partial least squares model, a support vector machine, or an ensemble of the models that are integrated to define an algorithm.

11. The apparatus according to any one or more of the claims 7-10, wherein the apparatus comprises at least one sensor capable outputting the sequence of pressure- or displacement signals over time in response to movements of a part of the body of the person during sleep;

12. The apparatus according to claim 11 , wherein the apparatus implements multiple sensors positioned at different locations relative to the body of the person during sleep, for example a sensor near the shoulder region, a sensor near the abdominal region and a sensor near the hip region.

13. The apparatus according to claim 11 or 12, further comprising a mattress provided with the one or more sensors.

14. The apparatus according to any one or more of the claims 7-13, wherein the apparatus further comprises: a training unit configured to train the computer-implemented artificial neural network with polysomnography input data characterizing a training sequence of pressure- or displacement signals over time with known body positions and/or sleep stages and for applying to the computer-implemented artificial neural network polysomnography input data characterizing at least a test sequence of pressure- or displacement signals over time with an unknown body position and/or sleep stage and for analysing each applied test sequence of pressure- or displacement signals to generate a predicted body position and/or sleep stage for each test sequence of pressure- or displacement signals, and for outputting the predicted body position and/or sleep stage.

15. The apparatus according to any one or more of the claims 7-14, wherein the receiving unit or the at least one processing unit is further configured to monitor the sequence of pressure- or displacement signals over time and/or the training sequence of pressure- or displacement signals over time and/or the test sequence of pressure- or displacement signals over time during (in) subsequent, consecutive data sequence frames of a certain time-frame length, and wherein the at least one processing unit generates the one or multiple time-frequency representations by time-frequency transformation of each of the subsequent, consecutive data sequence frames.

16. A computer program or product comprising instructions which, when the program is executed by a computer, cause the computer to carry out steps of the computer implemented method according to any one or more of the claims 1-6.

17. A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out steps of the computer implemented method according to any one or more of the claims 1-6.