WO2017140696A1

WO2017140696A1 - Device, system and method for determining a subject's breathing rate

Info

Publication number: WO2017140696A1
Application number: PCT/EP2017/053335
Authority: WO
Inventors: Benoit Marie BALMAEKERS; Vincent Jeanne
Original assignee: Koninklijke Philips N.V.
Priority date: 2016-02-15
Filing date: 2017-02-15
Publication date: 2017-08-24

Abstract

The present invention relates to a device, system and method for determining a subject's breathing rate. For determining a subject's breathing rate without active interaction of the user, without the use of a dedicated device and with improved user comfort and ease of use, the device comprises a signal input (21) for obtaining one or more sensor signals representing orientation changes and/or motion of a sensor, which is configured to measure orientation changes and/or motion of the sensor and to generate said one or more sensor signals. Further, the device comprises a selector (41) for determining combination values of said one or more sensor signals and for selecting one or more of said sensor signals either having the most extreme combination values or that most likely coincide with the direction of a breathing motion, which direction is determined from said combination values, a converter (22) for converting the selected one or more sensor signals into the frequency domain to obtain one or more frequency-domain sensor signals, and a peak detector (24) for performing maximum peak detection in said one or more frequency-domain sensor signals within a predetermined frequency range and selecting the frequency corresponding to the detected maximum peak as breathing rate.

Description

Device, system and method for determining a subject's breathing rate

FIELD OF THE INVENTION

The present invention relates to a device, system and method for determining a subject's breathing rate. BACKGROUND OF THE INVENTION

Recently, there has been a clear increase in interest in personal health monitoring, as evidenced by numerous consumer-oriented devices to track health-related metrics such as heart rate, oxygenation levels, breathing rate and pedometry. Breathing rate detection currently requires either a respiration belt to be applied around the chest, or electrodes to be attached to the chest. There are methods to determine breathing rate through video without other sensors, but these do not allow for mobile application, and still require some setup for correct framing of the camera image. Sensorless detection is also possible using radio -frequency wave reflection, but this still requires dedicated hardware.

Hence, the known methods and devices have several disadvantages. In particular, measuring breathing rate currently requires a dedicated device, i.e. a user would either have to purchase a device specifically for just this purpose, or go to a clinical facility. Further, conventional breathing rate detection systems require the user to wear a sensor on their body, reducing comfort and freedom of movement. Sensorless methods do not allow for mobile measurements, for example while walking. Finally, current systems require the user to actively setup and initiate a measurement.

US 2015/265161 Al discloses methods and apparatus for physiological parameter estimation. A gyroscope, an accelerometer and a camera gather sensor data indicative of motion of a human head. The gyroscope, accelerometer and camera are each housed in, or attached to, headwear that is worn on the head. In some cases, the headwear comprises a headband, hat, cap, or structure similar to an eyeglasses frame. A computer takes the sensor data as input and calculates a heart rate and respiration rate of the human. In some cases, a computer also calculates heart rate variability. The head motion being measured is caused by forces that are transmitted, at least in part, from the chest, through the neck, and to the head. This head motion is caused, at least in part, by respiration, by heart beats, or by blood flow caused by the heart beats.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide a device, system and method for determining a subject's breathing rate without active interaction of the user, without the use of a dedicated device and with improved user comfort and ease of use.

In a first aspect of the present invention a device for determining a subject's breathing rate is presented, said device comprising:

- a signal input for obtaining one or more sensor signals representing orientation changes and/or motion of a sensor, which is configured to measure orientation changes and/or motion of the sensor and to generate said one or more sensor signals,

a selector for determining combination values of said one or more sensor signals and for selecting one or more of said sensor signals either having the most extreme combination values or that most likely coincide with the direction of a breathing motion, which direction is determined from said combination values,

a converter for converting the selected one or more sensor signals into the frequency domain to obtain one or more frequency-domain sensor signals, and

a peak detector for performing maximum peak detection in said one or more frequency-domain sensor signals within a predetermined frequency range and selecting the frequency corresponding to the detected maximum peak as breathing rate.

In a further aspect of the present invention a corresponding method for determining a subject's breathing rate is presented.

In still a further aspect of the present invention a system for determining a subject's breathing rate is presented, said system comprising:

a sensor for measuring orientation changes and/or motion of the sensor and for generating one or more sensor signals representing the measured orientation changes and/or motion of the sensor, and

a device as disclosed herein for determining a subject's breathing rate from said one or more sensor signals.

In yet further aspects of the present invention, there are provided a computer program which comprises program code means for causing a computer to perform the steps of the method disclosed herein when said computer program is carried out on a computer as well as a non-transitory computer-readable recording medium that stores therein a computer program product, which, when executed by a processor, causes the method disclosed herein to be performed.

Preferred embodiments of the invention are defined in the dependent claims. It shall be understood that the claimed method, system, computer program and medium have similar and/or identical preferred embodiments as the claimed device and as defined in the dependent claims.

The present invention is based on the idea to make use of sensors that are already available in a handheld or wearable user device, such as a smartphone or camera. Such sensor may include a gyroscope and/or accelerometer. By use of the sensor signals (representing separate sensor channels) acquired by one or more of such sensors the breathing rate of the subject using the user device can be determined. This can be done without user interaction and unobtrusively, particularly while the user is actually using the user device, i.e. while the user device is actually in operation (e.g. while the user is making a phone call with a smartphone or taking a picture with a camera).

By use of the device, system and method according to the present invention, a user can assess his breathing rate comfortably at home without having to affix any specific sensors to their body, but using an available user device. The breathing rate can also be monitored when the user is walking, either on a fitness device or outside. Still further, by detecting the orientation and movement of the user device, measurements can automatically be performed whenever applicable without active user interaction.

In the case of an accelerometer as sensor, as proposed in an embodiment, the (raw) sensor signals represent forces. If the accelerometer is static, its sensor signal values correspond to a specific orientation with respect to gravity. Hence, the sensor signal values change as a result of orientation changes and/or sensor motion changes (possibly due to respiration), but for this particular purpose it should be noted that orientation itself is also of interest and may be used independent of motion.

A suitable frequency range, within which the peak detection is performed, may be a frequency range around typical breathing rates, such as a frequency range of 0 to 30 BPM (beats per minute) or 10 to 30 BPM. The device further comprises a selector for determining combination values, in particular the mean values, of said one or more sensor signals and for selecting one or more of said sensor signals having the most extreme combination values, in particular the highest mean value, wherein said converter is configured to convert the selected one or more sensor signals, in particular only the selected one or more sensor signals, into the frequency domain to obtain one or more frequency- domain sensor signals. The mean values are particularly used to determine orientation, which may then be used to map the different sensor signals onto the most likely breathing vector (which may be predetermined, although possibly adapted to the detected activity, e.g. sitting or making a phone call). This leads to a particular linear combination of these sensor signals into a single signal, which is then converted into the frequency domain. One option is to select the largest frequency peak within a suitable frequency range across sensor signals. Respiration motion and disturbances might affect several sensor signals, and a linear combination of two or more sensor signal may further improve the accuracy. Generally, for the combination a PC A (principal component analysis), ICA (independent component analysis), linear or non-linear combination of the sensor signals can be used.

The selector is further configured for selecting the one or more sensor signals that most likely coincide with the direction of a breathing motion, which direction is determined from the combination values, in particular mean values, of said one or more sensor signals, wherein said converter is configured to convert only the selected one or more sensor signals into the frequency domain to obtain one or more frequency-domain sensor signals. Hence, according to an embodiment use is made of the mean value of three raw sensor channels to determine orientation of a device (with respect to gravity) and of a given relative vector expected to contain the (main) breathing displacement. Further, in an embodiment, rather than then selecting one or more channels (i.e. sensor signals) for further individual processing, all channels may be mapped onto this vector, thereby combining the relevant signal contributions of all these channels. In case the subject is walking (which could be detected using the same signals), these embodiments may reduce the contributions of the periodic artefacts in the signals resulting from the stepping.

In an embodiment said signal input is configured to obtain a sensor signal per direction of three orthogonal directions in which the orientation changes and/or motion of the sensor are measured. This improves the accuracy and reliability of the detection of the breathing rate.

In another embodiment the device further comprises a controller for generating a control signal for controlling the sensor to start measuring orientation changes and/or motion of the sensor and generating said one or more sensor signals and for controlling the device to start determining the breathing rate based on the generated one or more sensor signals. The controller is particularly configured to generate said control signal based on the orientation of the sensor and/or an indication that a system, which comprises the sensor, is in use. For instance, if the system is implemented in a telephone, the start of the determination of the breathing rate may thus be started when the telephone is in use (i.e. is held in the typical position close to the ear for making a phone call).

In a further embodiment the device further comprises a signal processor for applying independent component analysis or principal component analysis to obtain information for distinguishing breathing rate from noise.

Independent component analysis may be applied to several sensor channels (or possible even several sensors, in which case some preprocessing might have to be applied). This embodiment finds the set of linear coefficients that result in an equal number of components as there are input signals, and where the components are as independent as possible (based on Gaussianity). This way, respiration motion can be separated from disturbances. However, this method does not provide an indication of which component contains the respiration signal. While it is possible to select the component with the largest frequency peak, a more robust way would be to select the component that results from the coefficients closest to the predicted respiration vector. This combines the location/activity prediction based on the device orientation, and the ICA method for correcting small errors between the predicted and actual respiration vector.

Principal component analysis (PCA) also computes coefficients that can be used to construct components. However, these components are orthogonal, and chosen such that they can reconstruct each input signal with minimal error. One or more these

components will only reflect a distinct source (be it respiration or disturbance) if this source significantly affects all sensor channels. This means that PCA is less suitable for source separation when the breathing vector aligns with one of the physical sensor axes (which can be determined from the orientation), but might provide more robust noise rejection in the opposite case.

The device may further comprise a signal output for outputting the determined breathing rate, in particular for transmitting the determined breathing rate to an external entity in a wireless or wired manner and/or for issuing the determined breathing rate for reception by a user. The signal output may e.g. comprises a transmitter for transmitting the determined breathing rate in a wireless or wired manner, e.g. via a wired or wireless network or communication link. In another embodiment the signal output may comprise a user interface, such as a display.

In one embodiment said sensor is configured to measure orientation changes and/or motion of the sensor in three orthogonal directions and to output a sensor signal per direction. In a practical implementation, the sensor may comprise a gyroscope for measuring orientation changes of the sensor and/or a motion sensor, in particular an accelerometer, for measuring motion of the sensor.

The system may be integrated into a handheld or wearable user device, in particular a camera, smartphone, mobile phone, glasses, hearing aid, virtual reality device, or augmented reality device. Generally, however, it may also be implemented in other ways.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter. In the following drawings

Fig. 1 shows a schematic diagram of a first embodiment of a system and device according to the present invention,

Fig. 2 shows a diagram of a second embodiment of a system and device according to the present invention,

Fig. 3 shows a diagram of signals for explaining details of the present invention,

Fig. 4 shows a schematic diagram of a third embodiment of a system and device according to the present invention,

Fig. 5 shows a schematic diagram of a fourth embodiment of a system and device according to the present invention, and

Fig. 6 shows a schematic diagram of a fifth embodiment of a system and device according to the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Fig. 1 shows a schematic diagram of a first embodiment of a system 1 and device 20 according to the present invention for determining the breathing rate of a subject, e.g. a patient in a hospital or care center, an elderly person at home (e.g. in home healthcare) or a baby in a baby monitoring station. The system 1 generally comprises a sensor 10 for measuring orientation changes and/or motion of the sensor and for generating one or more sensor signals 11 representing the measured orientation changes and/or motion of the sensor 10. The system 1 further comprises a device 20 for determining a subject's breathing rate from said one or more sensor signals 11.

Fig. 2 shows a particular implementation of the system 1 in the form of a smartphone 3, according to which the elements of the system 1 are integrated into the smartphone 3, which may already comprises elements that can be used (partly after modification) for determining the subject's breathing rate as proposed. For instance, the device 20 may be implemented in soft- and/or hardware, e.g. in the form of an application program ('app') running on the smartphone's processor 30.

The sensor 10 may include a gyroscope 12 for measuring orientation changes 13 of the sensor 10 and/or a motion sensor 14, e.g. an accelerometer, for measuring motion 15 of the sensor. In a smartphone 3 shown in Fig. 2 such a gyroscope 12 and motion sensor 14 are generally available as standard equipment, which can thus be used for the purpose of the present invention. Generally, each of the gyroscope 12 and the motion sensor 14 measure the respective signal in three orthogonal directions and output a corresponding sensor signal per direction, i.e. six sensor signals may be available, three for the orientation and three for the motion. Hence, orientation and/or motion as well as changes of orientation and/or motion of the respective sensor as well as the complete smartphone 3 can be determined and corresponding information (via the sensor signals) can be provided to the device 20, either continuously or at certain (regular or irregular) intervals or on demand.

The device 20 comprises a signal input 21 for obtaining (i.e. retrieving or receiving) one or more sensor signals 11 from the sensor 10, e.g. three sensor signals 13 from the gyroscope 12 representing orientation changes and/or three sensor signals 15 from the accelerometer 14 representing motion. A converter 22 converts the one or more sensor signals 11 into the frequency domain, e.g. using an FFT or wavelet transformation, to obtain one or more frequency-domain sensor signals 23. A peak detector 24 performs maximum peak detection in said one or more frequency-domain sensor signals 23 within a

predetermined frequency range and selects the frequency corresponding to the detected maximum peak as breathing rate 25.

An output unit 26 may be provided for outputting the determined breathing rate 25. The output unit 26 may include a display 31, such as the display of the smartphone 3, for issuing the determined breathing rate directly for information and/or use by the user of the smartphone 3. Alternatively or additionally, the output unit 26 may include a transmission unit 32, such as a WiFi interface, LAN interface or Bluetooth interface, for transmitting the determined breathing rate to a third entity, e.g. a user device (e.g. a smartphone or tablet) of a doctor, caregiver or nurse, or a central computer or database of a hospital.

Thus, with such an embodiment a fast Fourier analysis may be performed on data from a three-axis gyroscope and accelerometer embedded in a user device, such as a smartphone. When the user is holding the phone to his ear, the breathing rate can be detected by maximum peak detection in a suitable frequency range, across the different sensor channels. Thus, no additional sensors, in particular contact sensors mounted to the user's body, are required, but an already available user device is modified (e.g. programmed) and used accordingly. The breathing rate can thus also be monitored when the user is walking, either on a fitness device or active outside. Measurements can automatically be performed whenever applicable without active user interaction.

In a more advanced implementation, since the signal channels most dominantly containing the motion resulting from breathing can depend on the exact device orientation, the sensor signals 15 from the motion sensor 14, e.g. mean accelerometer values, can be used to determine this orientation and select the candidate channels, i.e. one or more sensor signals 11, for further processing. For the selected channels, the largest peak in the frequency spectrum within a suitable frequency range may be interpreted as indicating the breathing rate. For this purpose, as shown in the embodiment of the system 2 and device 40 depicted in Fig. 3, a selector 41 is provided for determining combination values 42, in particular the mean values, of said one or more sensor signals 11, in particular of the sensor signals 15 from the motion sensor 14, and for selecting one or more of said sensor signals having the most extreme combination values, in particular the highest mean value. The converter 22 is configured to convert only the selected one or more sensor signals into the frequency domain to obtain one or more frequency-domain sensor signals.

Additionally, through detection of the orientation, determining when to perform measurements may be performed in the background. This way, measurements can be performed whenever possible without further user interaction. For this purpose, a controller 43 is provided for generating a control signal 44 for controlling the sensor 10 to start measuring orientation changes and/or motion of the sensor and generating said one or more sensor signals. Further, the controller 43 may control the device 40 to start determining the breathing rate based on the generated one or more sensor signals. Particularly the orientation of the sensor 10 is used for generating the control signal 44 since the orientation allows determining if the user is actually in use or not, e.g. if the smartphone is actually held at the user's ear or not.

Alternatively, an indication that the system 2 is in use, e.g. that the smartphone is currently used for making a call, may be used for generating the control signal 44. Such an indicating may be easily provided by the smartphone and may be evaluated by the controller 43.

For further processing of the sensor signals e.g. of the raw three-axis accelerometer data and/or gyroscope data, a signal processor 45 may be provided, as shown in Figs. 5 and 6 depicting a fourth embodiment of a system 3 and device 40a and a fifth embodiment of a system 4 and device 40b according to the present invention. The signal processor 45 is particularly provided for applying an algorithm such as ICA or PCA to further improve signal separation and improving signal-to-noise ratio, which improved sensor signals 46 may then be used in the subsequent processing. Hereby, the signal processor 45 may e.g. immediately follow the signal input 21 or the selector 41.

Additionally, the coefficients resulting from ICA or PCA may be compared to those in the selector. Similarity then indicates that the components extracted by ICA or PCA are likely to represent breathing. Otherwise, the decision can be made, either to ignore the ICA or PCA results (and instead use the direct projection along the expected breathing vector given, possibly, the detected activity), or to interpret this to indicate an unsuitable

measurement moment or configuration.

Results of the processing as explained above may be validated by inducing different breathing rates while walking. Fig. 4 shows the frequency spectrum of an accelerometer. The x-axis indicates the breathing rate for induced breathing rates of 15 BPM (Fig. 4A), 20 BPM (Fig. 4B) and 10 BPM (Fig. 4C), respectively. Also visible are the main peaks indicating the walking rate (120 steps per minute) and, to a lesser extent, half the walking rate.

The presented system, device and method are rather simple in terms of signal processing and only require one or more sensors, such as a motion sensor (e.g. an

accelerometer) or an orientation sensor, such as a gyroscope. It can therefore potentially be applied in all types of consumer devices that already include these sensors, such as smartphones, or wearables such as fitness devices, or augmented reality devices that are held or positioned such that they are exposed to the motion of the upper body, such as a phone while making a phone call, or a device integrated into glasses or other head mounted accessories. It may also be used in devices that are upgraded with a required sensor. As a smartphone process running in the background, the present invention can be used for automatic breathing rate detection whenever appropriate, without active user action.

It shall be noted that the invention can be used in other embodiments of systems and devices than the ones shown in the figures. Particularly the elements of the various devices may be used in other combinations.

While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims.

In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single element or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

A computer program may be stored/distributed on a suitable non-transitory medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.

Any reference signs in the claims should not be construed as limiting the scope.

Claims

CLAIMS:

1. Device for determining a subject's breathing rate, said device comprising:

a signal input (21) for obtaining one or more sensor signals representing orientation changes and/or motion of a sensor, which is configured to measure orientation changes and/or motion of the sensor and to generate said one or more sensor signals, - a selector (41) for determining combination values of said one or more sensor signals and for selecting one or more of said sensor signals either having the most extreme combination values or that most likely coincide with the direction of a breathing motion, which direction is determined from said combination values,

a converter (22) for converting the selected one or more sensor signals into the frequency domain to obtain one or more frequency-domain sensor signals, and

a peak detector (24) for performing maximum peak detection in said one or more frequency-domain sensor signals within a predetermined frequency range and selecting the frequency corresponding to the detected maximum peak as breathing rate.

2. Device as claimed in claim 1,

wherein said signal input (22) is configured to obtain a sensor signal per direction of three orthogonal directions in which the orientation changes and/or motion of the sensor are measured.

3. Device as claimed in claim 1,

wherein said selector (41) is configured to determine the mean values of said one or more sensor signals as combination values and to select one or more of said sensor signals having the highest mean values.

4. Device as claimed in claim 1,

wherein said selector (41) is configured to determine the mean values of said one or more sensor signals as combination values and to determine the direction of breathing motion from said mean values of said one or more sensor signals.

5. Device as claimed in claim 1,

further comprising a controller (43) for generating a control signal for controlling the sensor to start measuring orientation changes and/or motion of the sensor and generating said one or more sensor signals and for controlling the device to start determining the breathing rate based on the generated one or more sensor signals.

6. Device as claimed in claim 5,

wherein said controller (43) is configured to generate said control signal based on the orientation of the sensor and/or an indication that a system, which comprises the sensor, is in use.

7. Device as claimed in claim 1,

further comprising a signal processor (45) for applying independent component analysis or principal component analysis to obtain information for distinguishing breathing rate from noise.

8. Device as claimed in claim 1,

further comprising a signal output (26) for outputting the determined breathing rate, in particular for transmitting the determined breathing rate to an external entity in a wireless or wired manner and/or for issuing the determined breathing rate for reception by a user.

9. System for determining a subject's breathing rate, said system comprising:

a sensor (10) for measuring orientation changes and/or motion of the sensor and for generating one or more sensor signals representing the measured orientation changes and/or motion of the sensor, and

a device (20, 40) as claimed in claim 1 for determining a subject's breathing rate from said one or more sensor signals.

10. System as claimed in claim 9,

wherein said sensor (10) is configured to measure orientation changes and/or motion of the sensor in three orthogonal directions and to output a sensor signal per direction.

11. System as claimed in claim 9,

wherein said sensor comprises a gyroscope (12) for measuring orientation changes of the sensor.

12. System as claimed in claim 9,

wherein said sensor comprises a motion sensor (14), in particular an accelerometer, for measuring motion of the sensor.

13. System as claimed in claim 9,

wherein said system is integrated into a handheld or wearable user device, in particular a camera, smartphone, mobile phone, glasses, hearing aid, virtual reality device, or augmented reality device.

14. Method for determining a subject's breathing rate, said device comprising: - obtaining one or more sensor signals representing orientation changes and/or motion of a sensor, which is configured to measure orientation changes and/or motion of the sensor and to generate said one or more sensor signals,

determining combination values of said one or more sensor signals, selecting one or more of said sensor signals either having the most extreme combination values or that most likely coincide with the direction of a breathing motion, which direction is determined from said combination values,

converting the selected one or more sensor signals into the frequency domain to obtain one or more frequency-domain sensor signals,

performing maximum peak detection in said one or more frequency-domain sensor signals within a predetermined frequency range, and

selecting the frequency corresponding to the detected maximum peak as breathing rate.

15. Computer program comprising program code means for causing a computer to carry out the steps of the method as claimed in claim 14 when said computer program is carried out on the computer.