WO2020141999A1 - Breath monitoring devices, systems and methods - Google Patents

Breath monitoring devices, systems and methods Download PDF

Info

Publication number
WO2020141999A1
WO2020141999A1 PCT/SG2018/050641 SG2018050641W WO2020141999A1 WO 2020141999 A1 WO2020141999 A1 WO 2020141999A1 SG 2018050641 W SG2018050641 W SG 2018050641W WO 2020141999 A1 WO2020141999 A1 WO 2020141999A1
Authority
WO
WIPO (PCT)
Prior art keywords
breathing
patient
breath
sound signal
ear
Prior art date
Application number
PCT/SG2018/050641
Other languages
French (fr)
Inventor
Dimuthu Lakmal KARIYAWASAN JALATH THANTHRIGE
Buddhi Ayesha Rathnayaka HEWA KIRINDAGE DON
Original Assignee
Vulcand (Private) Limited
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Vulcand (Private) Limited filed Critical Vulcand (Private) Limited
Priority to PCT/SG2018/050641 priority Critical patent/WO2020141999A1/en
Publication of WO2020141999A1 publication Critical patent/WO2020141999A1/en

Links

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/08Detecting, measuring or recording devices for evaluating the respiratory organs
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/85Providing additional services to players
    • A63F13/86Watching games played by other players
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/08Detecting, measuring or recording devices for evaluating the respiratory organs
    • A61B5/0803Recording apparatus specially adapted therefor
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/08Detecting, measuring or recording devices for evaluating the respiratory organs
    • A61B5/0816Measuring devices for examining respiratory frequency
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/68Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6813Specially adapted to be attached to a specific body part
    • A61B5/6814Head
    • A61B5/6815Ear
    • A61B5/6817Ear canal
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235Details of waveform analysis
    • A61B5/7264Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/74Details of notification to user or communication with user or patient ; user input means
    • A61B5/742Details of notification to user or communication with user or patient ; user input means using visual displays
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B7/00Instruments for auscultation
    • A61B7/003Detecting lung or respiration noise
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/30Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
    • A63F13/35Details of game servers
    • A63F13/355Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/45Controlling the progress of the video game
    • A63F13/49Saving the game status; Pausing or ending the game
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/50Controlling the output signals based on the game progress
    • A63F13/53Controlling the output signals based on the game progress involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
    • A63F13/533Controlling the output signals based on the game progress involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game for prompting the player, e.g. by displaying a game menu
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2560/00Constructional details of operational features of apparatus; Accessories for medical measuring apparatus
    • A61B2560/02Operational features
    • A61B2560/0242Operational features adapted to measure environmental factors, e.g. temperature, pollution
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2560/00Constructional details of operational features of apparatus; Accessories for medical measuring apparatus
    • A61B2560/02Operational features
    • A61B2560/0242Operational features adapted to measure environmental factors, e.g. temperature, pollution
    • A61B2560/0247Operational features adapted to measure environmental factors, e.g. temperature, pollution for compensation or correction of the measured physiological value
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2560/00Constructional details of operational features of apparatus; Accessories for medical measuring apparatus
    • A61B2560/04Constructional details of apparatus
    • A61B2560/0406Constructional details of apparatus specially shaped apparatus housings
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2560/00Constructional details of operational features of apparatus; Accessories for medical measuring apparatus
    • A61B2560/04Constructional details of apparatus
    • A61B2560/0462Apparatus with built-in sensors
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2562/00Details of sensors; Constructional details of sensor housings or probes; Accessories for sensors
    • A61B2562/02Details of sensors specially adapted for in-vivo measurements
    • A61B2562/0204Acoustic sensors
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2562/00Details of sensors; Constructional details of sensor housings or probes; Accessories for sensors
    • A61B2562/02Details of sensors specially adapted for in-vivo measurements
    • A61B2562/0271Thermal or temperature sensors
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2562/00Details of sensors; Constructional details of sensor housings or probes; Accessories for sensors
    • A61B2562/02Details of sensors specially adapted for in-vivo measurements
    • A61B2562/029Humidity sensors
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/08Detecting, measuring or recording devices for evaluating the respiratory organs
    • A61B5/082Evaluation by breath analysis, e.g. determination of the chemical composition of exhaled breath
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/08Detecting, measuring or recording devices for evaluating the respiratory organs
    • A61B5/0823Detecting or evaluating cough events
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7203Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235Details of waveform analysis
    • A61B5/7264Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A61B5/7267Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61MDEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
    • A61M15/00Inhalators
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61MDEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
    • A61M15/00Inhalators
    • A61M15/0065Inhalators with dosage or measuring devices
    • A61M15/0068Indicating or counting the number of dispensed doses or of remaining doses
    • A61M15/008Electronic counters
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61MDEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
    • A61M2205/00General characteristics of the apparatus
    • A61M2205/33Controlling, regulating or measuring
    • A61M2205/3303Using a biosensor
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61MDEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
    • A61M2230/00Measuring parameters of the user
    • A61M2230/40Respiratory characteristics
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20Input arrangements for video game devices
    • A63F13/21Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/212Input arrangements for video game devices characterised by their sensors, purposes or types using sensors worn by the player, e.g. for measuring heart beat or leg activity
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/30Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
    • A63F13/35Details of game servers
    • A63F13/352Details of game servers involving special game server arrangements, e.g. regional servers connected to a national server or a plurality of servers managing partitions of the game world

Definitions

  • TECHNICAL FIELD The present disclosure relates to breath monitoring and in particular to devices, systems and methods for monitoring the breathing of a patient such as a person suffering from asthma.
  • Asthma is a widespread disease around the globe, where it is ranked third as the main cause for hospitalization, for children under the age of 15. Asthma can be a serious respiratory condition which has a serious impact on a person’s quality of life and health and can lead to death. Average figures suggest that 383,000 people die due to asthma every year. From these 202,990 are children.
  • asthma creates and causes many secondary ailments leading into different sicknesses and or amplifying the adverse effects of other diseases.
  • a major problem faced by patients especially children and doctors alike is the effective administration of asthmatic medication which to a large extent are inhaler-based medication. It has been estimated that 93% of children suffering from asthma do not know how to use inhalers properly and they also suffer from severe breathing problems. The main reason for this is that children do not like to use inhalers as treatments due to the lack of motivation in the treatment process and a proper tracking mechanism to see the progress of the asthma condition of the child. Most asthma patients are unaware of correct breathing exercises and unwilling to do breathing exercises since there is no motivation factor.
  • a device for detecting breath sounds of a subject comprises: a housing having a portion configured to be insertab!e in a first ear of the subject; a first microphone arranged within the housing to capture sounds within the ear of the subject; and a second microphone arranged within the housing to capture sounds external to the ear of the subject.
  • the device further comprises a processing module configured to combine signals from the first microphone and the second microphone to generate a breathing sound signal for the subject.
  • the processing module is configured to apply a noise reduction algorithm to the signals from the first microphone and the second microphone.
  • the noise reduction algorithm may be a normalized least mean squares algorithm.
  • the device further comprises a communication interface configured to transmit an indication of the breathing sound of the subject to a user device.
  • the second microphone is arranged within the housing such that second microphone faces the pinna or outer ear of the first ear of the subject when the portion of the housing configured to be insertable in the first ear of the subject is inserted in the first ear of the subject.
  • the device further comprises a second housing portion configured to be insertable in a second ear of the subject, the second housing portion having a speaker arranged therein.
  • a method of monitoring breathing of a subject comprises: receiving on an indication of breathing sound signal from a breath monitoring device; identifying breathing events by classifying parts of the sound signal; determining a temporal length of an identified breathing event; and generating a breathing event indication.
  • the breathing events are inhaling and / or exhaling.
  • generating a breathing event indication comprises controlling a displayed object on a display as part of a game.
  • the game may be configured to simulate breathing exercises.
  • classifying parts of the sound signal comprise using a machine learning classifier to classify parts of the sound signal.
  • the method further comprises generating a mel-scaled spectrogram from the indication of the breathing signal and wherein classifying parts of the sound signal comprises using the mel-scaled spectrogram.
  • the breathing events comprise inhaler usage and wherein generating a breathing event indication comprises estimating an inhaled dosage.
  • an electronic device comprising a processor and a storage device.
  • the storage device stores computer executable instructions for causing the processor to carry out a method comprising: receiving on an indication of breathing sound signal from a breath monitoring device; identifying breathing events by classifying parts of the sound signal; determining a temporal length of an identified breathing event; and generating a breathing event indication.
  • the breathing events are inhaling and / or exhaling.
  • generating a breathing event indication comprises controlling a displayed object on a display as part of a game.
  • the game may be configured to simulate a breathing exercise.
  • classifying parts of the sound signal comprises using a machine learning classifier to classify parts of the sound signal.
  • the method further comprises generating a mel-scaled spectrogram from the indication of the breathing signal and wherein classifying parts of the sound signal comprises using the mel-scaled spectrogram.
  • the breathing events comprise inhaler usage and wherein generating a breathing event indication comprises estimating an inhaled dosage.
  • a breath monitoring system comprising: a user device configured to monitoring breath sounds of a patient and generate a symptom diary for the patient, the symptom diary comprising indications of breathing events for the patient; a breath monitoring and analysis server configured to receive the symptom diary from the user device and to store indications of the breathing events in a database; and a physician terminal configured to allow a doctor to access the indications of breathing events stored on the database.
  • the breath monitoring system further comprises an air quality measuring device comprising a plurality of sensors configured to measure indications of environmental conditions in the vicinity of the patient, wherein the breath monitoring and analysis server is configured to receive indications of the environmental conditions in the vicinity of the patient and store the indications of the environmental conditions in the database.
  • the plurality of sensors may comprise at least one of the following: a particulate matter sensor, a volatile organic compound sensor, a carbon dioxide sensor, a temperature sensor and a humidity sensor.
  • the user device is an electronic device as described above.
  • Embodiments of the present invention provide a product that assists Asthma Patients in particular children and gives a perfect motivation for the patients to carry out breathing exercises properly and keep on using the inhalers accordingly.
  • the product comes with a headset that will identify the breathing patterns of the user, and mobile games, that motivate the children to use the inhaler and get healthier as they play.
  • the game also supports single player as well as multi-player features.
  • Embodiments of the present invention facilitate enhanced administration of specific inhaler-based medicines for asthma simultaneously having the ability to assimilate and evaluate data to support a symptom diary, identification of preventive measures.
  • Figure 1 shows a breath monitoring system according to an embodiment of the present invention
  • Figures 2a and 2b show an earpiece breath monitoring device according to an embodiment of the present invention
  • Figures 3a and 3b illustrate a processing of signals captured by an earpiece breath monitoring device according to an embodiment of the present invention
  • Figure 4 shows a block diagram of an earpiece breath monitoring device according to an embodiment of the present invention
  • Figure 5 shows a block diagram of a user device for processing breathing sounds of a user according to an embodiment of the present invention
  • Figure 6 is a flowchart showing a method of breath monitoring according to an embodiment of the present invention.
  • Figure 7 shows a mel spectrogram used in embodiments of the present invention to classify sounds
  • Figure 8 shows an example of analysis of breathing sounds carried out in an embodiment of the present invention
  • Figure 9 is a block diagram showing an air quality measuring device according to an embodiment of the present invention.
  • Figure 10 shows a breath monitoring system according to an embodiment of the present invention.
  • Embodiments of the present invention provide a system for monitoring breathing and inhaler usage by patients such as children who are asthma suffers.
  • the system may include games that give motivation for children to carry out breathing exercises properly and keep on using the inhalers.
  • FIG. 1 shows a breath monitoring system according to an embodiment of the present invention.
  • the breath monitoring system 100 comprises an earpiece breath monitoring device 110 which is inserted into the ear of the patient in a similar manner to a normal earpiece.
  • the earpiece breath monitoring device 110 comprises microphones arranged to capture and isolate the breath sounds of the patient.
  • the breath monitoring system 100 further comprises a user device 120 such as a smart phone or tablet computer.
  • the user device 120 may be coupled to the earpiece breath monitoring device 110 via a wireless network connection such as a Bluetooth connection.
  • the user device 120 and the breath monitoring device are coupled by a wired connection.
  • the user device 120 runs a software application which identifies breathing patterns of the patient from the signals captured by the earpiece breath monitoring device 110.
  • the software application may include games that are controlled by the breathing of the patient. Such a game motivates children to use the breathing inhaler and carry breathing exercises properly and get healthier as they play.
  • the games may also support single player as well as multi-player features.
  • the software application may be configured to identify inhaler usage in addition to analyzing breathing patterns of the patient.
  • the breath monitoring system 100 further comprises an air quality measuring device 130 which is located close to the patient and comprises a plurality of sensors for detecting air pollutants which may be causes of asthma.
  • the air quality measuring device 130 may be portable and may be carried with the patient.
  • the user device 120 and the air quality measuring device 130 are connected to a network 140, which may be for example the internet.
  • a monitoring and analysis server 150 is connected to the network 140 and receives data from the user device 120 and the air quality measuring device 130 over the network 140.
  • the monitoring and analysis server 150 may implement a cloud platform for monitoring and analyzing the breathing patterns of the patient over time. For example, the monitoring and analysis server 150 may assimilate and evaluate data to support a symptom diary for the patient.
  • a physician terminal 160 is coupled to the monitoring and analysis server 150 and allows a doctor to access data for the patient, for example to review inhaler usage or breathing patterns of the patient.
  • FIGs 2a and 2b show an earpiece breath monitoring device according to an embodiment of the present invention.
  • the earpiece breath monitoring device 200 comprises two microphones. The signals from the two microphones are used to isolate the breathing sounds of a patient from other noise sounds.
  • the earpiece breath monitoring device 200 comprises a housing 210 which has a cylindrical portion 212 that is sized to fit into the ear canal of the patient.
  • a soft rubber bush 214 is arranged over the cylindrical portion 212.
  • a cable outlet 216 extends downward from the housing 210.
  • the earpiece breath monitoring device 200 comprises an exterior microphone 220 arranged within the housing 200 to collect sound signals outside the ear of the patient.
  • the earpiece breath monitoring device 200 shown in Figure 2a is intended to be inserted into the right ear of a patient with the cable outlet 216 facing downwards.
  • the exterior microphone 220 will face backwards towards the outer ear or pinna of the patient when the earpiece breath monitoring device 200 is in use. This minimizes the breathing sounds captured by the exterior microphone 220 because it faces away from the patient’s mouth.
  • the exterior microphone 220 captures a relatively large amount of exterior noise and a relatively small amount of the patient’s breathing sound. Therefore the efficiency of isolation of the patient’s breathing sounds by combining the signal captured by the two microphones is maximized.
  • Figure 2b shows a view of the earpiece breath monitoring device 200 with the soft rubber bush 214 removed.
  • the cylindrical portion 212 comprises a ridge 218 to hold the soft rubber bush in place.
  • An in-ear microphone 230 is located with the cylindrical portion 212 to capture sounds within the patient’s ear canal.
  • the in-ear microphone 230 captures sounds from within the patient’s inner ear and the exterior microphone 220 captures sounds outside the patient’s ear.
  • the signals captured by the two microphones are used to isolate the patient’s breathing sounds from other noise sounds. This process is described in below with reference to Figures 3a and 3b.
  • Figures 3a and 3b illustrate a processing of signals captured by an earpiece breath monitoring device according to an embodiment of the present invention.
  • a breathing sound signal 310 is captured by the in-ear microphone 230.
  • the in-ear microphone 230 also captures a noise sound signal 320.
  • the exterior microphone 220 captures the noise sound signal 320.
  • the outputs from the in-ear microphone 230 and the external microphone 220 are provided to a processing module 330 which isolates the breathing sound signal 310 by applying a noise cancellation algorithm such as the normalized least mean squared (NLMS) algorithm.
  • NLMS normalized least mean squared
  • the in-ear signal 315 received by the in-ear microphone 230 is a combination of the breathing sound signal 310 and the noise sound signal 320.
  • the exterior signal 325 received by the exterior microphone 220 is the noise sound signal 320 alone.
  • the processing module 330 applies a noise cancellation algorithm to reduce the noise in the in-ear signal 315 to provide a filtered breathing sound signal 340.
  • This filtered breathing sound signal 340 is used in the later processing to analyze the breathing of the patient and / or to control the game.
  • FIG 4 shows a block diagram of an earpiece breath monitoring device according to an embodiment of the present invention.
  • the earpiece breath monitoring device 400 shown in Figure 4 comprises two earpieces: a monitoring earpiece 410 which comprises two microphones as described above with reference to Figures 2a and 2b, and a speaker earpiece 420 which is configured to produce sound in the same manner as a conventional earpiece.
  • the monitoring earpiece 410 and the speaker earpiece 420 are connected to a processing and interface component 430.
  • the processing an interface component 430 comprises a processing module 432 and a communication interface 434.
  • the processing module 432 is configured to process the signals received from the monitoring earpiece 410.
  • the communication interface 434 allows the earpiece breath monitoring device 400 to send and receive signals from device such as a smart phone or tablet computing device.
  • the communication interface 434 may be implemented as a wireless interface such as a Bluetooth interface.
  • the communication interface 434 may be implemented as a wired interface such as an analogue interface provided by a 3.5mm audio jack or a digital interface such as a universal serial bus (USB) interface.
  • the processing carried out by the processing module 432 comprises a noise reduction algorithm such as that described above with reference to Figures 3a and 3b.
  • the communication interface 434 may transmit a filtered breathing sound signal to a user device for processing and may receive indications of sound signals from the user device for transmission to the user through the speaker earpiece 420.
  • FIG. 5 shows a block diagram of a user device for processing breathing sounds of a user according to an embodiment of the present invention.
  • the user device 500 may be for example a smart phone device, tablet computing device, or other computing device.
  • the user device 500 comprises a processor 510, a communication interface, a wireless network interface 530, a display 540 and a program storage 550.
  • the processor 510 is a central processing unit (CPU) which is capable of executing computer executable instructions.
  • the communication interface 520 allows communication with devices such as the earpiece breath monitoring device described above.
  • the communication interface 520 may be implemented as a wireless interface such as a Bluetooth interface.
  • the communication interface may be a wired interface such as an analogue interface provided by a 3.5 mm audio jack or a digital interface such as a universal serial bus (USB) interface.
  • the wireless network interface 530 allows the user device 500 to connect to wireless networks and thereby connect with the internet through a wireless router.
  • the program storage 550 stores computer program instructions which may be in the form of computer program or application which can be executed by the processor 510.
  • the computer program storage 550 stores a sound classification module 552, a breath analysis module 554 and a game module 556.
  • the sound classification module comprises computer program instructions for classifying sounds received from the earpiece breath monitoring. The sounds may be classified as an inhale breathing sound, an exhale breathing sound, an inhale using an inhaler, or noise sounds.
  • the sound classification module 552 may also comprise computer program instructions for identifying asthma symptoms and events such as coughing, wheezing and other breathing related sounds.
  • the breath analysis module 554 comprises computer program instructions for analyzing the breathing sounds classified by the sound classification module 552.
  • the breath analysis module 554 may comprise computer program instructions for determining the length of time of breathing actions of the patient, such as the length of time of an exhale or inhale.
  • the breath analysis module may comprise computer program instructions for determining a dose of medicine or drug inhaled by the patient using an inhaler.
  • the game module 556 comprises computer program instructions for a breathing based games.
  • the games may involve displaying a character or other object on the display 540 of the user device 500. From the analysis of breathing signals of the patient determined by the breath analysis module 554, the movement of the character may be controlled. For example, by breathing in, the patient controls the character to move upwards and by breathing out the patient controls the character to move downwards. Thus, by playing the game, a child can learn to control their breathing.
  • games can motivate the patients, particularly children, to use inhalers properly.
  • the games may simulate recommended breathing exercises.
  • FIG. 6 is a flowchart showing a method of breath monitoring according to an embodiment of the present invention. The method 600 shown in Figure 6 is carried out on the user device 500 shown in Figure 5.
  • the communication interface 520 of the user device 500 receives an indication of a breath sound signal from the earpiece breath monitoring device.
  • a noise reduction algorithm is applied by the earpiece breath monitoring device before the indication of the breath sound signal is transmitted to the user device 500.
  • the sound classification module 552 of the user device 500 classifies sounds in the breath sound signal.
  • the classification module 552 may classify sounds as breath sounds or noise sounds. It is noted that while the processing carried out on the earpiece breath monitoring device will remove some of the non breath sounds from the capture signals, some noise signals may still remain. For example, the voice of the patient travels through facial muscles and thus cannot be filtered out by the noise reduction algorithm implemented on the earpiece breath monitoring device.
  • the sound classification module 552 may implement a machine learning based classification algorithm.
  • the sound classification module generates a mel-spectrogram from the input breath sound signal.
  • Figure 7 shows an example of a mel-spectrogram.
  • Figure 7 shows a mel-spectrogram used in embodiments of the present invention to classify sounds.
  • a mel-spectrogram may be used to train a deep neural network.
  • a deep neural network is built.
  • the neural network is trained by breathing and non-breathing samples.
  • Inputs for the model are mel-spectrograms of those thousands of sound samples.
  • Deep neural network iteratively learns the features of mel-spectrogram which can be used to identify inhaling and exhaling sounds among others.
  • By feeding mel-spectrogram as an input to the trained model it can output whether the sound sample is an inhaling sound, an exhaling sound or another sound.
  • portions of the sound are identified as an inhale sound 710 and an exhale sound 720.
  • the breath analysis module 554 of the user device 500 determines a temporal length of breathing events classified in step 604. For example, the breath analysis module 554 may determine the length of inhale and / or exhale events.
  • Figure 8 shows an example of analysis of breathing sounds carried out in an embodiment of the present invention. As shown in Figure 8, parts of the sound are classified as an inhaling part 810 and an exhaling part 820 of a breathing cycle 830.
  • the breath analysis module 554 determines the temporal length of the inhaling part 810 and the exhaling part 820.
  • the breath analysis module 554 may determine a dose of medication inhaled by the patient. This may comprise determining length of an inhale event for an inhaler and from this determining the amount of medication inhaled. The dose may be determined from a cumulative count of the number of puffs taken on the inhaler. Inhalers generally have a specified amount of medication delivered in a single puff or actuation. Doctors recommend a minimum time window to keep inhaling in order to enter full medication into lungs. This differs according to inhaler type.
  • the medication delivered in each puff is d
  • the prescribed inhaling time period for a single puff is tr
  • the total number of puffs detected is n
  • the actual recorded inhaling time periods are tai, ta2, ... ,ta n .
  • the total dose Dtotai can then be calculated as:
  • the breath analysis module may use the equations above to estimate the dose of medication inhaled by the patient.
  • the patient may enter details of the inhaler used and the medication and this data may be stored on the user device 500 to allow calculation of the dose of medication.
  • the user device 500 generates an event notification. This may involve the game module 556 controlling an object displayed on the display 540 of the user device. Step 608 may also involve recording breathing events and sending an indication of the recorded events to the monitoring and analysis server 150 using the wireless network interface 530 of the user device 500.
  • FIG 9 is a block diagram showing an air quality measuring device according to an embodiment of the present invention.
  • the air quality measuring device 130 comprises a particulate matter sensor r 910, a volatile organic compound sensor 920, a carbon dioxide sensor 930, a temperature sensor 940, a humidity sensor 950, and a communication interface 960.
  • the air quality measuring device 130 is configured to detect and measure several air pollutants which are causes of asthma and also to monitor environmental conditions.
  • the particulate matter sensor 910 may be a smoke particle detector which detects particles having a diameter of 2.5 microns or less.
  • the communication interface 960 may be a wireless network interface and allows the outputs of the sensors to be transmitted to the monitoring an analysis server 150.
  • FIG. 10 shows a breath monitoring system according to an embodiment of the present invention.
  • the breath monitoring system 1000 shown in Figure 10 may include the features of the breath monitoring system 100 shown in Figure 1.
  • the breath monitoring system 1000 comprises a monitoring and analysis server 150.
  • the monitoring and analysis server 150 is coupled via a network to a user device 120 and a physician terminal 160.
  • the breath monitoring and analysis system 1000 further comprises an air quality measuring device 130 which may be configured as shown in Figure 9.
  • the air quality measuring device comprises a communication interface 960 and sensors 1032.
  • the sensors 1032 may comprise the sensors shown in Figure 9.
  • the user device 120 may be a mobile device and may include the functionality described above with reference to Figure 5.
  • the user device 120 runs games 1022 and a patient app 1024.
  • the games 1022 are designed to encourage the user to perform breathing exercises and use inhales properly.
  • the patient app 1024 implements a symptom diary which stores information about asthma symptoms such as coughing, wheezing and asthma attacks.
  • the symptom diary may also record inhaler usage and medication dosages. Additionally, the symptom diary may record breathing exercises carried out by the user and information on the results of such breathing exercises.
  • the symptom diary may also record environmental data captured by the air quality measuring device 130.
  • Information captured by the symptom diary of the patient app 1024 is sent to the breath monitoring and analysis server 150 which stores the data in a database 1052.
  • the breath monitoring and analysis server 150 may execute machine learning algorithms to predict asthma attacks from the patient diary data.
  • the breath monitoring and analysis server 150 may receive environmental data directly from the air quality measuring device 130.
  • the physician terminal 160 is coupled to the breath monitoring and analysis server 150 and allows a doctor to access and analyze patient data using a doctor app 1062 which runs on the physician terminal.
  • the patient app 1024 may send the symptom diary data to the breath monitoring and analysis server 150 on a daily basis.
  • the symptom diary data may include number of puffs of medication inhaled, time spent on breathing exercises and other data.
  • the data is collected by the breath monitoring and analysis server 150 and stored in the database 1052.
  • a doctor can view the symptom diary data using the doctor app 1062 running on the physician terminal 160.
  • the symptom diary data may be protected and encrypted to ensure that access is limited to a doctor or a group of doctors authorized to access the patient data. This invention facilitates enhanced administration of specific inhaler-based medicines for asthma simultaneously having the ability to assimilate and evaluate data to support a symptom diary, identification of preventive measures.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • Public Health (AREA)
  • Biomedical Technology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Surgery (AREA)
  • Molecular Biology (AREA)
  • Medical Informatics (AREA)
  • Veterinary Medicine (AREA)
  • General Health & Medical Sciences (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Pulmonology (AREA)
  • Physiology (AREA)
  • Otolaryngology (AREA)
  • Artificial Intelligence (AREA)
  • Fuzzy Systems (AREA)
  • Evolutionary Computation (AREA)
  • Mathematical Physics (AREA)
  • Signal Processing (AREA)
  • Psychiatry (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Optics & Photonics (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

A system for monitoring breathing of a patient such as an asthma sufferer is described. The system comprises an earpiece device which captures breath sounds of the patient. The earpiece device may comprise two microphones arranged to capture sounds from within the ear of the patient and external to the ear of the patient respectively. The signals from the two microphones are used in a noise reduction algorithm. A user device processes the breathing signals and may implement a game controlled by the patient's breathing. The user device may also generate entries for a symptom diary for the patient which is stored on a monitoring and analysis server. Air quality measurements may also be stored on the monitoring and analysis server.

Description

BREATH MONITORING DEVICES, SYSTEMS AND METHODS
TECHNICAL FIELD The present disclosure relates to breath monitoring and in particular to devices, systems and methods for monitoring the breathing of a patient such as a person suffering from asthma.
BACKGROUND
Many children suffer from various illnesses and diseases throughout their childhood and some consequences of them are tragic. Asthma is a widespread disease around the globe, where it is ranked third as the main cause for hospitalization, for children under the age of 15. Asthma can be a serious respiratory condition which has a serious impact on a person’s quality of life and health and can lead to death. Average figures suggest that 383,000 people die due to asthma every year. From these 202,990 are children.
Additionally, asthma creates and causes many secondary ailments leading into different sicknesses and or amplifying the adverse effects of other diseases. A major problem faced by patients especially children and doctors alike is the effective administration of asthmatic medication which to a large extent are inhaler-based medication. It has been estimated that 93% of children suffering from asthma do not know how to use inhalers properly and they also suffer from severe breathing problems. The main reason for this is that children do not like to use inhalers as treatments due to the lack of motivation in the treatment process and a proper tracking mechanism to see the progress of the asthma condition of the child. Most asthma patients are unaware of correct breathing exercises and unwilling to do breathing exercises since there is no motivation factor. SUMMARY OF THE INVENTION
According to a first aspect of the present disclosure, a device for detecting breath sounds of a subject is provided. The device comprises: a housing having a portion configured to be insertab!e in a first ear of the subject; a first microphone arranged within the housing to capture sounds within the ear of the subject; and a second microphone arranged within the housing to capture sounds external to the ear of the subject.
In an embodiment, the device further comprises a processing module configured to combine signals from the first microphone and the second microphone to generate a breathing sound signal for the subject. in an embodiment, the processing module is configured to apply a noise reduction algorithm to the signals from the first microphone and the second microphone. The noise reduction algorithm may be a normalized least mean squares algorithm.
In an embodiment, the device further comprises a communication interface configured to transmit an indication of the breathing sound of the subject to a user device.
In an embodiment the second microphone is arranged within the housing such that second microphone faces the pinna or outer ear of the first ear of the subject when the portion of the housing configured to be insertable in the first ear of the subject is inserted in the first ear of the subject.
In an embodiment the device further comprises a second housing portion configured to be insertable in a second ear of the subject, the second housing portion having a speaker arranged therein.
According to a second aspect of the present disclosure, a method of monitoring breathing of a subject is provided. The method comprises: receiving on an indication of breathing sound signal from a breath monitoring device; identifying breathing events by classifying parts of the sound signal; determining a temporal length of an identified breathing event; and generating a breathing event indication.
In an embodiment the breathing events are inhaling and / or exhaling.
In an embodiment generating a breathing event indication comprises controlling a displayed object on a display as part of a game. The game may be configured to simulate breathing exercises.
In an embodiment classifying parts of the sound signal comprise using a machine learning classifier to classify parts of the sound signal.
In an embodiment, the method further comprises generating a mel-scaled spectrogram from the indication of the breathing signal and wherein classifying parts of the sound signal comprises using the mel-scaled spectrogram.
In an embodiment the breathing events comprise inhaler usage and wherein generating a breathing event indication comprises estimating an inhaled dosage.
According to a third aspect of the present invention an electronic device comprising a processor and a storage device is provided. The storage device stores computer executable instructions for causing the processor to carry out a method comprising: receiving on an indication of breathing sound signal from a breath monitoring device; identifying breathing events by classifying parts of the sound signal; determining a temporal length of an identified breathing event; and generating a breathing event indication.
In an embodiment, the breathing events are inhaling and / or exhaling.
In an embodiment, generating a breathing event indication comprises controlling a displayed object on a display as part of a game. The game may be configured to simulate a breathing exercise. In an embodiment, classifying parts of the sound signal comprises using a machine learning classifier to classify parts of the sound signal.
In an embodiment, the method further comprises generating a mel-scaled spectrogram from the indication of the breathing signal and wherein classifying parts of the sound signal comprises using the mel-scaled spectrogram.
In an embodiment, the breathing events comprise inhaler usage and wherein generating a breathing event indication comprises estimating an inhaled dosage.
According to a fourth aspect of the present disclosure a breath monitoring system is provided. The breath monitoring system comprises: a user device configured to monitoring breath sounds of a patient and generate a symptom diary for the patient, the symptom diary comprising indications of breathing events for the patient; a breath monitoring and analysis server configured to receive the symptom diary from the user device and to store indications of the breathing events in a database; and a physician terminal configured to allow a doctor to access the indications of breathing events stored on the database. In an embodiment, the breath monitoring system further comprises an air quality measuring device comprising a plurality of sensors configured to measure indications of environmental conditions in the vicinity of the patient, wherein the breath monitoring and analysis server is configured to receive indications of the environmental conditions in the vicinity of the patient and store the indications of the environmental conditions in the database.
The plurality of sensors may comprise at least one of the following: a particulate matter sensor, a volatile organic compound sensor, a carbon dioxide sensor, a temperature sensor and a humidity sensor.
In an embodiment, the user device is an electronic device as described above.
Embodiments of the present invention provide a product that assists Asthma Patients in particular children and gives a perfect motivation for the patients to carry out breathing exercises properly and keep on using the inhalers accordingly. The product comes with a headset that will identify the breathing patterns of the user, and mobile games, that motivate the children to use the inhaler and get healthier as they play. The game also supports single player as well as multi-player features. Embodiments of the present invention facilitate enhanced administration of specific inhaler-based medicines for asthma simultaneously having the ability to assimilate and evaluate data to support a symptom diary, identification of preventive measures.
BRIEF DESCRIPTION OF THE DRAWINGS
In the following, embodiments of the present invention will be described as non limiting examples with reference to the accompanying drawings in which:
Figure 1 shows a breath monitoring system according to an embodiment of the present invention;
Figures 2a and 2b show an earpiece breath monitoring device according to an embodiment of the present invention; Figures 3a and 3b illustrate a processing of signals captured by an earpiece breath monitoring device according to an embodiment of the present invention;
Figure 4 shows a block diagram of an earpiece breath monitoring device according to an embodiment of the present invention;
Figure 5 shows a block diagram of a user device for processing breathing sounds of a user according to an embodiment of the present invention;
Figure 6 is a flowchart showing a method of breath monitoring according to an embodiment of the present invention;
Figure 7 shows a mel spectrogram used in embodiments of the present invention to classify sounds; Figure 8 shows an example of analysis of breathing sounds carried out in an embodiment of the present invention; Figure 9 is a block diagram showing an air quality measuring device according to an embodiment of the present invention; and
Figure 10 shows a breath monitoring system according to an embodiment of the present invention.
DETAILED DESCRIPTION
Embodiments of the present invention provide a system for monitoring breathing and inhaler usage by patients such as children who are asthma suffers. The system may include games that give motivation for children to carry out breathing exercises properly and keep on using the inhalers.
Figure 1 shows a breath monitoring system according to an embodiment of the present invention. The breath monitoring system 100 comprises an earpiece breath monitoring device 110 which is inserted into the ear of the patient in a similar manner to a normal earpiece. The earpiece breath monitoring device 110 comprises microphones arranged to capture and isolate the breath sounds of the patient. The breath monitoring system 100 further comprises a user device 120 such as a smart phone or tablet computer. The user device 120 may be coupled to the earpiece breath monitoring device 110 via a wireless network connection such as a Bluetooth connection. In alternative embodiments, the user device 120 and the breath monitoring device are coupled by a wired connection. The user device 120 runs a software application which identifies breathing patterns of the patient from the signals captured by the earpiece breath monitoring device 110. The software application may include games that are controlled by the breathing of the patient. Such a game motivates children to use the breathing inhaler and carry breathing exercises properly and get healthier as they play. The games may also support single player as well as multi-player features. The software application may be configured to identify inhaler usage in addition to analyzing breathing patterns of the patient.
The breath monitoring system 100 further comprises an air quality measuring device 130 which is located close to the patient and comprises a plurality of sensors for detecting air pollutants which may be causes of asthma. The air quality measuring device 130 may be portable and may be carried with the patient. The user device 120 and the air quality measuring device 130 are connected to a network 140, which may be for example the internet.
A monitoring and analysis server 150 is connected to the network 140 and receives data from the user device 120 and the air quality measuring device 130 over the network 140. The monitoring and analysis server 150 may implement a cloud platform for monitoring and analyzing the breathing patterns of the patient over time. For example, the monitoring and analysis server 150 may assimilate and evaluate data to support a symptom diary for the patient.
A physician terminal 160 is coupled to the monitoring and analysis server 150 and allows a doctor to access data for the patient, for example to review inhaler usage or breathing patterns of the patient.
Figures 2a and 2b show an earpiece breath monitoring device according to an embodiment of the present invention. The earpiece breath monitoring device 200 comprises two microphones. The signals from the two microphones are used to isolate the breathing sounds of a patient from other noise sounds. As shown in Figure 2a, the earpiece breath monitoring device 200 comprises a housing 210 which has a cylindrical portion 212 that is sized to fit into the ear canal of the patient. A soft rubber bush 214 is arranged over the cylindrical portion 212. A cable outlet 216 extends downward from the housing 210. The earpiece breath monitoring device 200 comprises an exterior microphone 220 arranged within the housing 200 to collect sound signals outside the ear of the patient. The earpiece breath monitoring device 200 shown in Figure 2a is intended to be inserted into the right ear of a patient with the cable outlet 216 facing downwards. Thus, the exterior microphone 220 will face backwards towards the outer ear or pinna of the patient when the earpiece breath monitoring device 200 is in use. This minimizes the breathing sounds captured by the exterior microphone 220 because it faces away from the patient’s mouth. Thus the exterior microphone 220 captures a relatively large amount of exterior noise and a relatively small amount of the patient’s breathing sound. Therefore the efficiency of isolation of the patient’s breathing sounds by combining the signal captured by the two microphones is maximized.
Figure 2b shows a view of the earpiece breath monitoring device 200 with the soft rubber bush 214 removed. As shown in Figure 2b, the cylindrical portion 212 comprises a ridge 218 to hold the soft rubber bush in place. An in-ear microphone 230 is located with the cylindrical portion 212 to capture sounds within the patient’s ear canal.
As described above, the in-ear microphone 230 captures sounds from within the patient’s inner ear and the exterior microphone 220 captures sounds outside the patient’s ear. In embodiments of the present invention the signals captured by the two microphones are used to isolate the patient’s breathing sounds from other noise sounds. This process is described in below with reference to Figures 3a and 3b. Figures 3a and 3b illustrate a processing of signals captured by an earpiece breath monitoring device according to an embodiment of the present invention. As shown in Figure 3a, a breathing sound signal 310 is captured by the in-ear microphone 230. The in-ear microphone 230 also captures a noise sound signal 320. The exterior microphone 220 captures the noise sound signal 320. The outputs from the in-ear microphone 230 and the external microphone 220 are provided to a processing module 330 which isolates the breathing sound signal 310 by applying a noise cancellation algorithm such as the normalized least mean squared (NLMS) algorithm.
As shown in Figure 3b, the in-ear signal 315 received by the in-ear microphone 230 is a combination of the breathing sound signal 310 and the noise sound signal 320. The exterior signal 325 received by the exterior microphone 220 is the noise sound signal 320 alone. The processing module 330 applies a noise cancellation algorithm to reduce the noise in the in-ear signal 315 to provide a filtered breathing sound signal 340. This filtered breathing sound signal 340 is used in the later processing to analyze the breathing of the patient and / or to control the game.
Figure 4 shows a block diagram of an earpiece breath monitoring device according to an embodiment of the present invention. The earpiece breath monitoring device 400 shown in Figure 4 comprises two earpieces: a monitoring earpiece 410 which comprises two microphones as described above with reference to Figures 2a and 2b, and a speaker earpiece 420 which is configured to produce sound in the same manner as a conventional earpiece. The monitoring earpiece 410 and the speaker earpiece 420 are connected to a processing and interface component 430. The processing an interface component 430 comprises a processing module 432 and a communication interface 434. The processing module 432 is configured to process the signals received from the monitoring earpiece 410. The communication interface 434 allows the earpiece breath monitoring device 400 to send and receive signals from device such as a smart phone or tablet computing device. The communication interface 434 may be implemented as a wireless interface such as a Bluetooth interface. Alternatively, the communication interface 434 may be implemented as a wired interface such as an analogue interface provided by a 3.5mm audio jack or a digital interface such as a universal serial bus (USB) interface. The processing carried out by the processing module 432 comprises a noise reduction algorithm such as that described above with reference to Figures 3a and 3b. The communication interface 434 may transmit a filtered breathing sound signal to a user device for processing and may receive indications of sound signals from the user device for transmission to the user through the speaker earpiece 420.
Figure 5 shows a block diagram of a user device for processing breathing sounds of a user according to an embodiment of the present invention. The user device 500 may be for example a smart phone device, tablet computing device, or other computing device. The user device 500 comprises a processor 510, a communication interface, a wireless network interface 530, a display 540 and a program storage 550. The processor 510 is a central processing unit (CPU) which is capable of executing computer executable instructions. The communication interface 520 allows communication with devices such as the earpiece breath monitoring device described above. In some embodiments, the communication interface 520 may be implemented as a wireless interface such as a Bluetooth interface. Alternatively, the communication interface may be a wired interface such as an analogue interface provided by a 3.5 mm audio jack or a digital interface such as a universal serial bus (USB) interface. The wireless network interface 530 allows the user device 500 to connect to wireless networks and thereby connect with the internet through a wireless router.
The program storage 550 stores computer program instructions which may be in the form of computer program or application which can be executed by the processor 510. The computer program storage 550 stores a sound classification module 552, a breath analysis module 554 and a game module 556. The sound classification module comprises computer program instructions for classifying sounds received from the earpiece breath monitoring. The sounds may be classified as an inhale breathing sound, an exhale breathing sound, an inhale using an inhaler, or noise sounds. The sound classification module 552 may also comprise computer program instructions for identifying asthma symptoms and events such as coughing, wheezing and other breathing related sounds. The breath analysis module 554 comprises computer program instructions for analyzing the breathing sounds classified by the sound classification module 552. For example, the breath analysis module 554 may comprise computer program instructions for determining the length of time of breathing actions of the patient, such as the length of time of an exhale or inhale. The breath analysis module may comprise computer program instructions for determining a dose of medicine or drug inhaled by the patient using an inhaler. The game module 556 comprises computer program instructions for a breathing based games. The games may involve displaying a character or other object on the display 540 of the user device 500. From the analysis of breathing signals of the patient determined by the breath analysis module 554, the movement of the character may be controlled. For example, by breathing in, the patient controls the character to move upwards and by breathing out the patient controls the character to move downwards. Thus, by playing the game, a child can learn to control their breathing.
There are two main benefits of using games. Firstly, games can motivate the patients, particularly children, to use inhalers properly. There are standard inhaler breathing techniques and the games may be designed to simulate those breathing techniques. Secondly, the games may simulate recommended breathing exercises.
For example, doctors recommend a breathing exercise called belly breathing (diaphragm breathing). It helps to use diaphragm effectively for breathing. In diaphragm breathing, patient need to take a slow deep breath and exhale slowly through the nose. Patient needs to stretch the stomach rather than the chest. The deep breath can be simulated through the games. When patient is using a metered dose inhaler, patient needs to take a deep breath in order to take medication into the mouth. Then he/she needs to hold the breath for about 10 secs and slowly breath out. This technique can be simulated via our games while using the inhalers. Figure 6 is a flowchart showing a method of breath monitoring according to an embodiment of the present invention. The method 600 shown in Figure 6 is carried out on the user device 500 shown in Figure 5.
In step 602, the communication interface 520 of the user device 500 receives an indication of a breath sound signal from the earpiece breath monitoring device. As described above, a noise reduction algorithm is applied by the earpiece breath monitoring device before the indication of the breath sound signal is transmitted to the user device 500. In step 604, the sound classification module 552 of the user device 500 classifies sounds in the breath sound signal. The classification module 552 may classify sounds as breath sounds or noise sounds. It is noted that while the processing carried out on the earpiece breath monitoring device will remove some of the non breath sounds from the capture signals, some noise signals may still remain. For example, the voice of the patient travels through facial muscles and thus cannot be filtered out by the noise reduction algorithm implemented on the earpiece breath monitoring device. Additionally, some external noise may not be completely canceled out by the noise reduction. The sound classification module 552 may implement a machine learning based classification algorithm. In an embodiment, the sound classification module generates a mel-spectrogram from the input breath sound signal. Figure 7 shows an example of a mel-spectrogram.
Figure 7 shows a mel-spectrogram used in embodiments of the present invention to classify sounds. Such a mel-spectrogram may be used to train a deep neural network. Firstly, a deep neural network is built. The neural network is trained by breathing and non-breathing samples. Inputs for the model are mel-spectrograms of those thousands of sound samples. Deep neural network iteratively learns the features of mel-spectrogram which can be used to identify inhaling and exhaling sounds among others. By feeding mel-spectrogram as an input to the trained model, it can output whether the sound sample is an inhaling sound, an exhaling sound or another sound. As shown in Figure 7, portions of the sound are identified as an inhale sound 710 and an exhale sound 720.
Returning to Figure 6, in step 606, the breath analysis module 554 of the user device 500 determines a temporal length of breathing events classified in step 604. For example, the breath analysis module 554 may determine the length of inhale and / or exhale events.
Figure 8 shows an example of analysis of breathing sounds carried out in an embodiment of the present invention. As shown in Figure 8, parts of the sound are classified as an inhaling part 810 and an exhaling part 820 of a breathing cycle 830. The breath analysis module 554 determines the temporal length of the inhaling part 810 and the exhaling part 820.
In some embodiments, the breath analysis module 554 may determine a dose of medication inhaled by the patient. This may comprise determining length of an inhale event for an inhaler and from this determining the amount of medication inhaled. The dose may be determined from a cumulative count of the number of puffs taken on the inhaler. Inhalers generally have a specified amount of medication delivered in a single puff or actuation. Doctors recommend a minimum time window to keep inhaling in order to enter full medication into lungs. This differs according to inhaler type.
In an example, the medication delivered in each puff is d, the prescribed inhaling time period for a single puff is tr, the total number of puffs detected is n, and the actual recorded inhaling time periods are tai, ta2, ... ,tan.
Then in the ith actuation, if tr>tai, the actual dose di inhaled will be: d
di—— x to-i
tr
Otherwise, (if tr£ tai) the actual dose di inhaled in the ith actuation will be:
di = d
The total dose Dtotai can then be calculated as:
Dtotai ¾ d + d2 H - l· dn
Figure imgf000015_0001
Where:
Figure imgf000015_0002
d-i— d (if tr ^ tcif).
The breath analysis module may use the equations above to estimate the dose of medication inhaled by the patient.
In a set up phase, the patient may enter details of the inhaler used and the medication and this data may be stored on the user device 500 to allow calculation of the dose of medication. In step 608, the user device 500 generates an event notification. This may involve the game module 556 controlling an object displayed on the display 540 of the user device. Step 608 may also involve recording breathing events and sending an indication of the recorded events to the monitoring and analysis server 150 using the wireless network interface 530 of the user device 500.
Figure 9 is a block diagram showing an air quality measuring device according to an embodiment of the present invention. As shown in Figure 9, the air quality measuring device 130 comprises a particulate matter sensor r 910, a volatile organic compound sensor 920, a carbon dioxide sensor 930, a temperature sensor 940, a humidity sensor 950, and a communication interface 960. The air quality measuring device 130 is configured to detect and measure several air pollutants which are causes of asthma and also to monitor environmental conditions. The particulate matter sensor 910 may be a smoke particle detector which detects particles having a diameter of 2.5 microns or less. The communication interface 960 may be a wireless network interface and allows the outputs of the sensors to be transmitted to the monitoring an analysis server 150.
Figure 10 shows a breath monitoring system according to an embodiment of the present invention. The breath monitoring system 1000 shown in Figure 10 may include the features of the breath monitoring system 100 shown in Figure 1.
The breath monitoring system 1000 comprises a monitoring and analysis server 150. The monitoring and analysis server 150 is coupled via a network to a user device 120 and a physician terminal 160.
The breath monitoring and analysis system 1000 further comprises an air quality measuring device 130 which may be configured as shown in Figure 9. The air quality measuring device comprises a communication interface 960 and sensors 1032. The sensors 1032 may comprise the sensors shown in Figure 9. The user device 120 may be a mobile device and may include the functionality described above with reference to Figure 5. The user device 120 runs games 1022 and a patient app 1024. The games 1022 are designed to encourage the user to perform breathing exercises and use inhales properly. The patient app 1024 implements a symptom diary which stores information about asthma symptoms such as coughing, wheezing and asthma attacks. The symptom diary may also record inhaler usage and medication dosages. Additionally, the symptom diary may record breathing exercises carried out by the user and information on the results of such breathing exercises. The symptom diary may also record environmental data captured by the air quality measuring device 130.
Information captured by the symptom diary of the patient app 1024 is sent to the breath monitoring and analysis server 150 which stores the data in a database 1052. The breath monitoring and analysis server 150 may execute machine learning algorithms to predict asthma attacks from the patient diary data. In some embodiments, the breath monitoring and analysis server 150 may receive environmental data directly from the air quality measuring device 130. The physician terminal 160 is coupled to the breath monitoring and analysis server 150 and allows a doctor to access and analyze patient data using a doctor app 1062 which runs on the physician terminal.
The patient app 1024 may send the symptom diary data to the breath monitoring and analysis server 150 on a daily basis. The symptom diary data may include number of puffs of medication inhaled, time spent on breathing exercises and other data. The data is collected by the breath monitoring and analysis server 150 and stored in the database 1052. A doctor can view the symptom diary data using the doctor app 1062 running on the physician terminal 160. The symptom diary data may be protected and encrypted to ensure that access is limited to a doctor or a group of doctors authorized to access the patient data. This invention facilitates enhanced administration of specific inhaler-based medicines for asthma simultaneously having the ability to assimilate and evaluate data to support a symptom diary, identification of preventive measures.
Whilst the foregoing description has described exemplary embodiments, it will be understood by those skilled in the art that many variations of the embodiments can be made within the scope and spirit of the present invention.

Claims

1. A device for detecting breath sounds of a subject, the device comprising:
a housing having a portion configured to be insertable in a first ear of the subject;
a first microphone arranged within the housing to capture sounds within the ear of the subject; and
a second microphone arranged within the housing to capture sounds external to the ear of the subject.
2. A device according to claim 1 , further comprising a processing module configured to combine signals from the first microphone and the second microphone to generate a breathing sound signal for the subject.
3. A device according to claim 2, wherein the processing module is configured to apply a noise reduction algorithm to the signals from the first microphone and the second microphone.
4. A device according to claim 3, wherein the noise reduction algorithm comprises a normalized least mean squares algorithm.
5. A device according to any one of claims 2 to 4, further comprising a
communication interface configured to transmit an indication of the breathing sound of the subject to a user device.
6. A device according to any preceding claim, wherein the second microphone is arranged within the housing such that second microphone faces the pinna or outer ear of the first ear of the subject when the portion of the housing configured to be insertable in the first ear of the subject is inserted in the first ear of the subject.
7. A device according to any preceding claim further comprising a second housing portion configured to be insertable in a second ear of the subject, the second housing portion having a speaker arranged therein.
8. A method of monitoring breathing of a subject, the method comprising:
receiving on an indication of breathing sound signal from a breath monitoring device;
identifying breathing events by classifying parts of the sound signal;
determining a temporal length of an identified breathing event; and
generating a breathing event indication.
9. A method according to claim 8, wherein the breathing events are inhaling and / or exhaling.
10. A method according to claim 8 or claim 9 wherein generating a breathing event indication comprises controlling a displayed object on a display as part of a game.
11. A method according to claim 10 wherein the game is configured to simulate a breathing exercise.
12. A method according to any one of claims 8 to 11 , wherein classifying parts of the sound signal comprise using a machine learning classifier to classify parts of the sound signal.
13. A method according to any one of claims 8 to 12, further comprising generating a mel-scaled spectrogram from the indication of the breathing signal and wherein classifying parts of the sound signal comprises using the mel-scaled spectrogram.
14. A method according to any one of claims 8 to 13, wherein the breathing events comprise inhaler usage and wherein generating a breathing event indication comprises estimating an inhaled dosage.
15. An electronic device comprising a processor and a storage device, the storage device storing computer executable instructions for causing the processor to carry out a method comprising: receiving on an indication of breathing sound signal from a breath monitoring device;
identifying breathing events by classifying parts of the sound signal;
determining a temporal length of an identified breathing event; and generating a breathing event indication.
16. An electronic device according to claim 15, wherein the breathing events are inhaling and / or exhaling.
17. An electronic device according to claim 15 or 16, wherein generating a breathing event indication comprises controlling a displayed object on a display as part of a game.
18. An electronic device according to claim 17 wherein the game is configured to simulate a breathing exercise.
19. An electronic device according to any one of claims 15 to 18, wherein classifying parts of the sound signal comprise using a machine learning classifier to classify parts of the sound signal.
20. An electronic device according to any one of claims 15 to 19, wherein the method further comprises generating a mel-scaled spectrogram from the indication of the breathing signal and wherein classifying parts of the sound signal comprises using the mel-scaled spectrogram.
21. An electronic device according to any one of claims 15 to 20, wherein the breathing events comprise inhaler usage and wherein generating a breathing event indication comprises estimating an inhaled dosage.
22. A breath monitoring system comprising:
a user device configured to monitoring breath sounds of a patient and generate a symptom diary for the patient, the symptom diary comprising indications of breathing events for the patient; a breath monitoring and analysis server configured to receive the symptom diary from the user device and to store indications of the breathing events in a database; and
a physician terminal configured to allow a doctor to access the indications of breathing events stored on the database.
23. A breath monitoring system according to claim 22, further comprising an air quality measuring device comprising a plurality of sensors configured to measure indications of environmental conditions in the vicinity of the patient, wherein the breath monitoring and analysis server is configured to receive indications of the environmental conditions in the vicinity of the patient and store the indications of the environmental conditions in the database.
24. A breath monitoring system according to claim 23, wherein the plurality of sensors comprises at least one of the following:
a particulate matter sensor, a volatile organic compound sensor, a carbon dioxide sensor, a temperature sensor and a humidity sensor.
25. A breath monitoring system according to any one of claims 22 to 24 wherein the user device is an electronic device according to any one of claims 15 to 21.
PCT/SG2018/050641 2018-12-31 2018-12-31 Breath monitoring devices, systems and methods WO2020141999A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/SG2018/050641 WO2020141999A1 (en) 2018-12-31 2018-12-31 Breath monitoring devices, systems and methods

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/SG2018/050641 WO2020141999A1 (en) 2018-12-31 2018-12-31 Breath monitoring devices, systems and methods

Publications (1)

Publication Number Publication Date
WO2020141999A1 true WO2020141999A1 (en) 2020-07-09

Family

ID=71407094

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/SG2018/050641 WO2020141999A1 (en) 2018-12-31 2018-12-31 Breath monitoring devices, systems and methods

Country Status (1)

Country Link
WO (1) WO2020141999A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023229429A1 (en) * 2022-05-27 2023-11-30 Samsung Electronics Co., Ltd. System and method for deep audio spectral processing for respiration rate and depth estimation using smart earbuds
CN117357094A (en) * 2023-11-15 2024-01-09 读取(北京)科技有限公司 Respiratory state verification system using sound intensity and carbon dioxide detection

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020143242A1 (en) * 2001-03-30 2002-10-03 Nemirovski Guerman G. Sensor for detecting changes within a human ear and producing a signal corresponding to thought, movement, biological function and/or speech
US20080146890A1 (en) * 2006-12-19 2008-06-19 Valencell, Inc. Telemetric apparatus for health and environmental monitoring
US20090016541A1 (en) * 2007-05-04 2009-01-15 Personics Holdings Inc. Method and Device for Acoustic Management Control of Multiple Microphones
US20170372691A1 (en) * 2016-06-23 2017-12-28 Mediatek Inc. Speech enhancement for headsets with in-ear microphones

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020143242A1 (en) * 2001-03-30 2002-10-03 Nemirovski Guerman G. Sensor for detecting changes within a human ear and producing a signal corresponding to thought, movement, biological function and/or speech
US20080146890A1 (en) * 2006-12-19 2008-06-19 Valencell, Inc. Telemetric apparatus for health and environmental monitoring
US20090016541A1 (en) * 2007-05-04 2009-01-15 Personics Holdings Inc. Method and Device for Acoustic Management Control of Multiple Microphones
US20170372691A1 (en) * 2016-06-23 2017-12-28 Mediatek Inc. Speech enhancement for headsets with in-ear microphones

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023229429A1 (en) * 2022-05-27 2023-11-30 Samsung Electronics Co., Ltd. System and method for deep audio spectral processing for respiration rate and depth estimation using smart earbuds
CN117357094A (en) * 2023-11-15 2024-01-09 读取(北京)科技有限公司 Respiratory state verification system using sound intensity and carbon dioxide detection

Similar Documents

Publication Publication Date Title
Holmes et al. Acoustic analysis of inhaler sounds from community-dwelling asthmatic patients for automatic assessment of adherence
Shi et al. Theory and application of audio-based assessment of cough
US20190328278A1 (en) System and method for monitoring respiration
CN109308443A (en) Method and apparatus for detecting breathing pattern
US10219740B2 (en) Systems, methods and apparatuses for the alleviation and outcome monitoring of sleep disordered breathing
Huq et al. Acoustic breath-phase detection using tracheal breath sounds
CN107529991A (en) For the heart of detection object and/or the equipment, system and method for respiratory disorder
Zubaydi et al. MobSpiro: Mobile based spirometry for detecting COPD
JP2022530963A (en) Inhaler system
Amoh et al. Technologies for developing ambulatory cough monitoring devices
Chen et al. Detection of mouse cough based on sound monitoring and respiratory airflow waveforms
WO2020141999A1 (en) Breath monitoring devices, systems and methods
CN108366756A (en) The devices, systems, and methods of the respiratory characteristic of object are determined based on breathing gas
Romano et al. Respiratory rate estimation during walking and running using breathing sounds recorded with a microphone
JP2013123496A (en) Discrimination device, electronic stethoscope, electronic auscultating system, discrimination method, discrimination program, and computer-readable recording medium
AU2020332707A1 (en) A method and apparatus for processing asthma patient cough sound for application of appropriate therapy
Al Rasyid et al. Design and development of portable spirometer
US20210045657A1 (en) Device and method for measuring respiratory air flow
Ahmed et al. Mouth breathing detection using audio captured through earbuds
CN107847186A (en) System and method and respiratory pressure for analyzing epithelium healing support system
US20210315517A1 (en) Biomarkers of inflammation in neurophysiological systems
WO2021188786A1 (en) Systems and methods for using a sound-producing breathing device to perform breathing exercises, and/or determine pulmonary function
US11925762B2 (en) Portable inhalator device
McNulty et al. Automatic Audio-Based Classification of Patient Inhaler Use: A Pharmacy Based Study
Hsu et al. A dual-purpose deep learning model for auscultated lung and tracheal sound analysis based on mixed set training

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18945373

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18945373

Country of ref document: EP

Kind code of ref document: A1