WO2023058946A1 - System and method for predicting respiratory disease prognosis through time-series measurements of cough sounds, respiratory sounds, recitation sounds and vocal sounds - Google Patents

System and method for predicting respiratory disease prognosis through time-series measurements of cough sounds, respiratory sounds, recitation sounds and vocal sounds Download PDF

Info

Publication number
WO2023058946A1
WO2023058946A1 PCT/KR2022/014058 KR2022014058W WO2023058946A1 WO 2023058946 A1 WO2023058946 A1 WO 2023058946A1 KR 2022014058 W KR2022014058 W KR 2022014058W WO 2023058946 A1 WO2023058946 A1 WO 2023058946A1
Authority
WO
WIPO (PCT)
Prior art keywords
sound
respiratory disease
data
disease prognosis
time
Prior art date
Application number
PCT/KR2022/014058
Other languages
French (fr)
Korean (ko)
Inventor
전진희
김경남
민충기
김태진
한상훈
문경민
Original Assignee
주식회사 웨이센
울산대학교 산학협력단
전진희
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from KR1020220037496A external-priority patent/KR102624637B1/en
Application filed by 주식회사 웨이센, 울산대학교 산학협력단, 전진희 filed Critical 주식회사 웨이센
Publication of WO2023058946A1 publication Critical patent/WO2023058946A1/en

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/08—Detecting, measuring or recording devices for evaluating the respiratory organs
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B7/00—Instruments for auscultation
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G10L25/66—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for extracting parameters related to health condition
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/30—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for calculating health indices; for individual health risk assessment
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/50—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for simulation or modelling of medical disorders
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients

Definitions

  • the present invention relates to a system and method for predicting the prognosis of respiratory diseases, and more particularly, by time-series analysis of voice data through time-series cough sound, breathing sound, reading sound, and vocalization sound measurement, and based on the analyzed information, the diagnosis of respiratory disease
  • it relates to a method for predicting the prognosis of respiratory diseases through the measurement of time-series cough sounds, breathing sounds, reading sounds, and vocalization sounds so that users can respond in advance.
  • Respiratory disease is classically diagnosed by a physician's chest auscultation, imaging tests, and the like. Coughing and wheezing accompany most respiratory diseases, and as an early detectable symptom, it is an important factor in a doctor's diagnosis. Depending on the respiratory disease, there are differences in the presence or absence of wheezing, the location, type, and time of coughing, and the presence or absence of sputum.
  • machine learning models based on deep learning have been continuously researched and developed, and such machine learning models are used in various voice analysis fields. Since there is a limit to directly hearing and analyzing abnormal breathing sounds by medical staff, it can play a role in supporting (assisting) the doctor's decision (diagnosis) for the patient by distinguishing them through a machine learning model.
  • a highly contagious disease such as COVID19
  • self-isolation there is no way to confirm whether the prognosis worsens or improves, so an opportunity to respond in a timely manner may be missed.
  • Patent Document 1 discloses a "cough sound analysis method using a disease signature for diagnosing respiratory diseases", thereby diagnosing one or more diseases of the patient's airways.
  • a method for doing this includes acquiring a cough sound from the patient; processing the cough sound to generate cough sound feature signals representing one or more cough sound features from cough segments; obtaining one or more disease signatures based on the cough sound feature signals; and classifying the one or more disease signatures to consider that the cough segments represent one or more of the diseases, wherein obtaining the one or more disease signatures based on the cough sound feature signals comprises: applying cough sound features to each of one or more pre-trained disease signature determination machines, each of which classifies the cough sound features as corresponding to a particular disease or non-disease state or to a first particular disease or It is characterized in that it is trained in advance to classify as corresponding to a second specific disease different from the first specific disease.
  • the cough sound features are applied to disease signature decision machines, and each decision machine is trained to classify the cough sound features as those corresponding to a specific disease or non-disease state, so that the patient's cough sound
  • each decision machine uses only the feature signal of the patient's cough sound, so other sounds of the patient, such as breathing sounds, reading sounds, or vocalizations, can be used to diagnose the respiratory tract. Predicting the prognosis of a disease contains problems that are difficult to apply.
  • the present invention was created in consideration of the above matters comprehensively, and analyzes voice data in a time series through periodic measurement of the user's cough sound, breathing sound, reading sound, and pronunciation sound, and based on the analyzed information, respiratory
  • An object of the present invention is to provide a respiratory disease prognosis prediction system and method through time-series coughing, breathing, reading, and vocalization measurements that allow users to respond appropriately in advance by predicting the prognosis of the disease.
  • the respiratory disease prognosis prediction system through the measurement of time-series cough sound, breathing sound, reading sound, and vocalization sound according to the present invention
  • a voice measuring unit that periodically measures cough sounds, breath sounds, reading sounds, and vocal sounds from the user
  • a data collection unit for collecting learning data obtained by evaluating a respiratory disease severity score with respect to the data measured by the voice measurement unit
  • model learning that trains a respiratory disease prognosis prediction model that takes the time series measurement data at any K time as an input value of the respiratory disease prognosis prediction model and uses the respiratory disease severity evaluation score after M hours as an output value. wealth;
  • a respiratory disease prognosis prediction unit that inputs the time-series measurement data at time K to the learned respiratory disease prognosis prediction model and predicts a respiratory disease prognosis based on a respiratory disease prognosis prediction value output by the respiratory disease prognosis prediction model after M time ;
  • the status of the data collection unit, the model learning unit, and the respiratory disease prognosis prediction unit are checked, the respiratory disease severity is evaluated by the data collection unit and classified into a specific class, and the respiratory disease prognosis prediction model is learned by the model learning unit. And, it is characterized in that it includes a control unit that transmits each control command for predicting the respiratory disease prognosis by the respiratory disease prognosis prediction unit.
  • the cough sound measured by the voice measurement unit can be obtained by repeatedly recording the patient's cough sound multiple times using a recording function of a smartphone.
  • the breathing sound can be obtained by repeatedly recording the patient's inhalation and exhalation multiple times using a recording function of a smartphone.
  • the reading sound may be obtained by recording a sound read aloud in a normal tone of speech using a recording function of a smartphone.
  • the vocalized sound can be obtained by recording a voice through vocalization produced while singing a pitched sound according to a familiar melody using a recording function of a smartphone.
  • the cough sound, breath sound, reading sound, and vocal sound can be measured using a smartphone at a period of 1 hour to N hours.
  • the data collection unit may evaluate the severity of respiratory disease with respect to the measured data, and classify it into normal (0), caution (1), and borderline (2) classes.
  • the model learning unit learns the respiratory disease prognosis prediction model, fine dust (PM10, PM2.5), temperature / humidity, CO 2 , Personal environment data and public environment data including VOCs (volatile organic compounds), and blood pressure/heart rate data may be learned together.
  • the model learning unit extracts the characteristics of the measured cough sound, breathing sound, reading sound, and vocalization sound using Mel-spectrogram, and extracts voice features from it. It can be trained using CNN (Convolutional Neural Network) algorithm by attaching data and public environment data.
  • CNN Convolutional Neural Network
  • M_(x ⁇ m ⁇ n) multi-channel data is formed by configuring x number of channels for cough sound, breath sound, reading sound, and vocal sound data and each data of the personal biometric, environmental data, and public environment data created, and the output of the CNN can be configured as a regression model of the respiratory disease severity evaluation score.
  • the method for predicting the prognosis of respiratory diseases through the measurement of time-series cough sound, breathing sound, reading sound, and vocalization sound includes a voice measurement unit, a data collection unit, a model learning unit, and a respiratory disease prognosis
  • a respiratory disease prognosis prediction method based on a respiratory disease prognosis prediction system through time series cough sound, breathing sound, reading sound, and vocalization sound measurement including a prediction unit,
  • Respiratory disease prognosis prediction in which the model learning unit takes time-series measurement data at any K time as an input value of a respiratory disease prognosis prediction model based on the collected learning data, and the respiratory disease severity evaluation score after M time as an output value training the model;
  • the respiratory disease prognosis prediction unit inputs the time-series measurement data of the K time to the learned respiratory disease prognosis prediction model, and the respiratory disease prognosis is based on the respiratory disease prognosis prediction value output by the respiratory disease prognosis prediction model after M time Its feature is that it includes the step of predicting .
  • the cough sound can be obtained by repeatedly recording the patient's cough sound multiple times using a recording function of a smart phone.
  • the breathing sound can be obtained by repeatedly recording the patient's inhalation and exhalation multiple times using a recording function of a smartphone.
  • the reading sound may be obtained by recording a sound read aloud in a normal tone of speech using a recording function of a smartphone.
  • the vocalized sound can be obtained by recording a voice through vocalization produced while singing a pitched sound according to a familiar melody using a recording function of a smartphone.
  • the cough sound, breath sound, reading sound, and vocal sound can be measured using a smartphone at a period of 1 hour to N hours.
  • the respiratory disease severity may be evaluated for the data measured in step b), and classified into normal (0), caution (1), and borderline (2) classes.
  • fine dust PM10, PM2.5
  • temperature / humidity in the same time zone as the cough sound, breathing sound, reading sound, and pronunciation sound measurement time
  • CO 2 personal environment data and public environment data including VOCs (volatile organic compounds), and blood pressure/heart rate data may be learned together.
  • the measured cough sound, breathing sound, reading sound, and vocalization sound are extracted using the Mel-spectrogram, and the characteristics of the voice are extracted, and personal biopsies, It can be trained using CNN (Convolutional Neural Network) algorithm by attaching environmental data and public environment data.
  • CNN Convolutional Neural Network
  • M_(x ⁇ m ⁇ n) multi-channel data is formed by configuring x number of channels for cough sound, breath sound, reading sound, and vocal sound data and each data of the personal biometric, environmental data, and public environment data and the output of the CNN can be configured as a regression model of the respiratory disease severity evaluation score.
  • the voice data is analyzed in a time series through periodic measurement of cough sounds, breath sounds, reading sounds, and pronunciation sounds, and the prognosis of respiratory diseases is predicted based on the analyzed information, so that the user can It has the advantage of being able to respond appropriately.
  • FIG. 1 is a diagram schematically showing the configuration of a respiratory disease prognosis prediction system through time-series cough sound, breathing sound, reading sound, and vocal sound measurement according to the present invention.
  • FIG. 2 is a flowchart showing the execution process of the method for predicting the prognosis of respiratory diseases through the measurement of time-series cough sounds, breathing sounds, reading sounds, and vocal sounds according to the present invention.
  • FIG. 3 is a diagram showing an outline of measuring cough sound, breathing sound, reading sound, and vocalization sound using a recording function of a smartphone.
  • Figure 4 is a diagram showing the evaluation of the severity of respiratory diseases and classification into normal, caution, and alert classes.
  • FIG. 5 is a diagram showing personal environment data, public environment data, and blood pressure/heart rate data that a respiratory disease prognosis prediction model learns together.
  • FIG. 6 is a diagram showing an overview of extracting characteristics of measured cough sounds, breath sounds, reading sounds, and pronunciation sounds, and learning by attaching personal biometric data, environmental data, and public environment data.
  • FIG. 1 is a diagram schematically showing the configuration of a respiratory disease prognosis prediction system through time-series cough sound, breathing sound, reading sound, and vocal sound measurement according to an embodiment of the present invention.
  • the respiratory disease prognosis prediction system 100 through the measurement of time-series cough sound, breathing sound, reading sound, and vocalization sound according to the present invention includes a voice measurement unit 110, a data collection unit 120, and model learning It is configured to include a unit 130, a respiratory disease prognosis prediction unit 140, and a control unit 150.
  • the voice measurement unit 110 periodically measures cough sounds, breathing sounds, reading sounds, and vocal sounds from the user.
  • a smartphone may be used as such a voice measurement unit 110 .
  • the cough sound measured by the voice measurement unit 110 as described above is a patient's cough sound multiple times (eg, 3 to 5 times) can be obtained by repeatedly recording.
  • the breathing sound can be obtained by repeatedly recording the patient's inhalation and exhalation multiple times (eg, 3 to 5 times) using a recording function of a smartphone.
  • the reading sound is a sentence presented using the recording function of the smartphone (eg, 'I agree to record voices for a healthier society') in a normal tone. It can be obtained by recording the sound you read.
  • the voiced sound is a pitched sound (for example, a pitched sound such as 'Do Re Mi Fa Sol La Si Do', a global common scale) using the recording function of a smartphone to a familiar melody. It can be obtained by recording the voice through vocalization while singing along.
  • cough sounds, breath sounds, reading sounds, and vocal sounds as described above can be measured using a smartphone at intervals of 1 hour to N hours.
  • the data collection unit 120 collects learning data by evaluating the data measured by the voice measurement unit 110 as a respiratory disease severity score.
  • the data collection unit 120 evaluates the respiratory disease severity as a score for the measured data.
  • the model learning unit 130 sets the time-series measurement data of any K time as an input value of the respiratory disease prognosis prediction model 160 as shown in FIG. 4, and the respiratory disease after M time A respiratory disease prognosis prediction model 160 having a disease severity evaluation score as an output value is trained.
  • the model learning unit 130 learns the respiratory disease prognosis prediction model 160, as shown in FIG.
  • personal environment data and public environment data including fine dust (PM10, PM2.5), temperature/humidity, CO 2 , VOCs (volatile organic compounds) of the time zone, and blood pressure/heart rate data can be learned together.
  • the model learning unit 140 as described above, as shown in FIG.
  • M_(x ⁇ m ⁇ n) multi-channel data is formed by configuring x number of channels for cough sound, breath sound, reading sound, and vocal sound data and each data of the personal biometric, environmental data, and public environment data created, and the output of the CNN can be configured as a respiratory disease severity evaluation score regression model.
  • the respiratory disease prognosis prediction unit 140 inputs the time-series measurement data of the K time to the learned respiratory disease prognosis prediction model 160, and after M time, the respiratory disease prognosis prediction model ( 160) predicts the respiratory disease prognosis based on the respiratory disease prognosis prediction value output.
  • the respiratory disease prognosis prediction unit 140 compares the respiratory disease prognosis prediction value output by the respiratory disease prognosis prediction model 160 with a preset reference value (reference range) so that the predicted value is a reference value (reference range) (eg , 0 to 1 point of the reference value), "steady state", if the predicted value belongs to the first threshold value (for example, 2 to 4 points of the reference value) for the reference value (reference range), "attention state", If it falls within 2 thresholds (eg, 5 points of the reference value), the respiratory disease prognosis is predicted as "alert state".
  • a respiratory disease prognosis prediction unit 140 may be configured with a microprocessor or microcontroller.
  • the control unit 150 checks the status of the data collection unit 120, the model learning unit 130, and the respiratory disease prognosis prediction unit 140 and controls their operations, and determines the severity of respiratory disease by the data collection unit 120.
  • Such control unit 150 may be composed of a microprocessor or microcontroller.
  • the data collection unit 120, the model learning unit 130, the respiratory disease prognosis prediction unit 140, and the control unit 150 as described above may be integrated into one computer system.
  • FIG. 2 is a flowchart illustrating an execution process of a method for predicting the prognosis of a respiratory disease through measurement of time-series coughing sound, breathing sound, reading sound, and vocalization sound according to an embodiment of the present invention.
  • the method for predicting the prognosis of respiratory diseases through the measurement of time-series cough sound, breathing sound, reading sound, and vocalization sound includes the voice measurement unit 110, the data collection unit 120, Respiratory disease based on respiratory disease prognosis prediction system 100 through measurement of time-series cough sound, breathing sound, reading sound, and vocalization sound including model learning unit 130, respiratory disease prognosis prediction unit 140, and control unit 150
  • the voice measuring unit 110 ie, using a user terminal (eg, a smartphone)
  • the cough sound can be obtained by repeatedly recording the patient's cough sound multiple times (eg, 3 to 5 times) using a recording function of a smartphone, as shown in FIG. 3 (a).
  • the breathing sound can be obtained by repeatedly recording the patient's inhalation and exhalation multiple times (eg, 3 to 5 times) using a recording function of a smartphone.
  • the reading sound is a sentence presented using the recording function of the smartphone (eg, 'I agree to record voices for a healthier society') in a normal tone. It can be obtained by recording the sound you read.
  • the voiced sound is a pitched sound (for example, a pitched sound such as 'Do Re Mi Fa Sol La Si Do', a global common scale) using the recording function of a smartphone to a familiar melody. It can be obtained by recording the voice through vocalization while singing along.
  • cough sounds, breath sounds, reading sounds, and vocal sounds as described above can be measured using a smartphone at intervals of 1 hour to N hours.
  • the data collection unit 120 evaluates the measured data as a respiratory disease severity score and collects learning data.
  • a data collection unit 120 as shown in Figure 4, can evaluate the respiratory disease severity score with respect to the measured data.
  • the model learning unit 130 sets the time series measurement data of arbitrary K hours as an input value of the respiratory disease prognosis prediction model 160 based on the collected learning data, and M hours later A respiratory disease prognosis prediction model 160 having a respiratory disease severity evaluation score as an output value is trained (step S203).
  • fine dust PM10, PM2
  • temperature/humidity CO 2
  • blood pressure/heart rate data can be learned together.
  • the model learning unit 130 learns the respiratory disease prognosis prediction model 160, as shown in FIG. 6, the measured cough sound, breathing sound, reading sound, and vocalization sound are Mel-spectrogram It is possible to extract features of voice using , and attach personal biometric data, environmental data, and public environment data to it, and learn them using a Convolutional Neural Network (CNN) algorithm.
  • CNN Convolutional Neural Network
  • the personal biometric data, environment data, and public environment data all data of 24 to 48 hours can be used.
  • the time point influencing symptoms can affect the current time or the situation N hours ago can affect the present, data of 24 to 48 hours It is preferable to use all of them.
  • M_(x ⁇ m ⁇ n) multi-channel data is formed by configuring x number of channels for cough sound, breath sound, reading sound, and vocal sound data and each data of the personal biometric, environmental data, and public environment data created, and the output of the CNN can be configured as a regression model that outputs respiratory disease severity evaluation scores.
  • the respiratory disease prognosis prediction unit 140 inputs the time-series measurement data of time K into the learned respiratory disease prognosis prediction model 160, and predicts the respiratory disease prognosis after M time.
  • a respiratory disease prognosis is predicted based on the respiratory disease prognosis prediction value output by the model 160 (step S204).
  • the respiratory disease prognosis prediction unit 140 compares the respiratory disease prognosis prediction value output by the respiratory disease prognosis prediction model 160 with a preset reference value (reference range) so that the predicted value is a reference value (reference range) (eg , 0 to 1 point of the reference value), "steady state”, if the predicted value belongs to the first threshold value (for example, 2 to 4 points of the reference value) for the reference value (reference range), "attention state", If it falls within 2 thresholds (eg, 5 points of the reference value), the respiratory disease prognosis is predicted as "alert state".
  • a reference value reference range
  • the system and method for predicting the prognosis of respiratory diseases through the measurement of time-series cough sound, breathing sound, reading sound, and vocalization sound provide voice through periodic measurement of cough sound, breathing sound, reading sound, and vocalization sound.

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Public Health (AREA)
  • Medical Informatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Pathology (AREA)
  • Data Mining & Analysis (AREA)
  • Epidemiology (AREA)
  • Databases & Information Systems (AREA)
  • Primary Health Care (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Veterinary Medicine (AREA)
  • Physics & Mathematics (AREA)
  • Molecular Biology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Biophysics (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Acoustics & Sound (AREA)
  • Computational Linguistics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physiology (AREA)
  • Pulmonology (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

A method for predicting respiratory disease prognosis through time-series measurements of cough sounds, respiratory sounds, recitation sounds, and vocal sounds, according to the present invention, comprises steps in which: a voice measurement unit periodically measures cough sounds, respiratory sounds, recitation sounds, and vocal sounds of a user; a data collection unit collects training data in which a respiratory disease severity score is evaluated with respect to the measurement data; a model training unit trains, on the basis of the collected training data, a respiratory disease prognosis prediction model of which an input value is random K-hour time-series measurement data and of which an output value is a respiratory disease severity evaluation score after M hours; and a respiratory disease prognosis prediction unit inputs the K-hour time-series measurement data into the trained respiratory disease prognosis prediction model, and predicts a respiratory disease prognosis on the basis of the respiratory disease prognosis prediction value outputted from the respiratory disease prognosis prediction model after M hours.

Description

시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측시스템 및 방법System and method for predicting the prognosis of respiratory diseases through time-series cough sound, breath sound, reading sound, and vocal sound measurement
본 발명은 호흡기 질환 예후 예측 시스템 및 방법에 관한 것으로서, 더 상세하게는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통하여 음성데이터를 시계열로 분석하고, 분석된 정보를 기반으로 호흡기 질환의 예후를 예측함으로써, 사용자가 사전에 대응할 수 있도록 하는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측방법에 관한 것이다.The present invention relates to a system and method for predicting the prognosis of respiratory diseases, and more particularly, by time-series analysis of voice data through time-series cough sound, breathing sound, reading sound, and vocalization sound measurement, and based on the analyzed information, the diagnosis of respiratory disease By predicting the prognosis, it relates to a method for predicting the prognosis of respiratory diseases through the measurement of time-series cough sounds, breathing sounds, reading sounds, and vocalization sounds so that users can respond in advance.
호흡기 질환은 고전적으로 의사의 흉부 청진, 영상 검사 등을 통해 진단된다. 기침과 천명음은 대부분의 호흡기 질환에 동반되며, 초기에 발견 가능한 증상으로 의사의 진단에 중요한 요소이다. 호흡기 질병에 따라 천명음의 유무, 기침의 위치, 종류, 시간, 객담의 유무 등의 차이가 있다.Respiratory disease is classically diagnosed by a physician's chest auscultation, imaging tests, and the like. Coughing and wheezing accompany most respiratory diseases, and as an early detectable symptom, it is an important factor in a doctor's diagnosis. Depending on the respiratory disease, there are differences in the presence or absence of wheezing, the location, type, and time of coughing, and the presence or absence of sputum.
상기 기침/호흡음의 차이를 이용하여 질병을 분류하고자 하는 연구가 지속되고 있다. COVID19의 경우, 열을 동반한 짧고 마른 기침이 나오며, 미각 소실 특징이 있다. 따라서 이를 이용하여 질병을 분류하거나 기타 다른 연구를 진행할 수 있다. COVID19의 유행으로 원격 진료 혹은 비대면 진료의 필요성 및 관심이 증대되면서, 스마트폰을 이용한 검진 및 진단에 대한 연구가 활발하다. 따라서, 호흡기 질병의 조기 진단과, 기침과 호흡음의 진단적 가치를 높이는 노력이 필요하다. 여기서, 비정상 호흡음의 종류로는 천명음, 나음, 수포음, 협착음, 흉막마찰음이 있다. 날숨(호기) 때 소리가 나는지 아니면 들숨(흡기) 때 소리가 나는지, 아니면 둘 다 소리가 나는지, 소리의 크기는 작은지 큰지, 소리의 높이는 낮은지 높은지, 어느 부위에서 소리가 나는지에 따라 진단이 달라진다.Studies to classify diseases using the cough/breath sound difference are ongoing. In the case of COVID19, a short, dry cough accompanied by a fever appears and is characterized by loss of taste. Therefore, it can be used to classify diseases or conduct other studies. As the need for and interest in telemedicine or non-face-to-face treatment increases due to the COVID19 epidemic, research on examination and diagnosis using smartphones is active. Therefore, it is necessary to make efforts to improve the early diagnosis of respiratory diseases and the diagnostic value of cough and breath sounds. Here, the types of abnormal breath sounds include wheezing, wheezing, rales, stridor, and pleural friction sounds. Diagnosis is based on whether the sound is heard during exhalation (exhalation) or inhalation (inhalation), or both, whether the sound is small or loud, whether the sound is low or high, and where the sound comes from. It varies.
최근 딥 러닝(deep learning) 기반의 기계학습 모델이 지속적으로 연구 및 개발되고 있으며, 이와 같은 기계학습 모델이 다양한 음성 분석 분야에 사용되고 있다. 비정상 호흡음을 의료진이 직접 듣고 이를 분석하는 것에는 한계가 있으므로, 기계학습 모델을 통해 이를 구분하여 환자에 대한 의사의 결정(진단)을 지원(보조)하는 역할을 할 수 있다. COVID19 같은 전파력이 강한 질병의 경우, 외부와 차단된 격리시설 또는 자가격리를 통해 증상이 호전될 때까지 관찰해야 하는데, 이를 항시 확인하는 것에는 어려움이 있다. 특히, 자가격리의 경우 예후가 악화되는지 호전되는지를 확인하는 방법이 없어 적시에 대응할 기회를 놓칠 수 있다.Recently, machine learning models based on deep learning have been continuously researched and developed, and such machine learning models are used in various voice analysis fields. Since there is a limit to directly hearing and analyzing abnormal breathing sounds by medical staff, it can play a role in supporting (assisting) the doctor's decision (diagnosis) for the patient by distinguishing them through a machine learning model. In the case of a highly contagious disease such as COVID19, it is necessary to observe until symptoms improve through quarantine facilities or self-isolation that are blocked from the outside world, but it is difficult to check this at all times. In particular, in the case of self-isolation, there is no way to confirm whether the prognosis worsens or improves, so an opportunity to respond in a timely manner may be missed.
한편, 한국 공개특허공보 제10-2020-0122301호(특허문헌 1)에는 "호흡기 질병 진단을 위한 질병 시그니처를 이용한 기침 소리 분석 방법"이 개시되어 있는 바, 이에 따른 환자 기도의 하나 이상의 질병을 진단하기 위한 방법은, 상기 환자로부터 기침 소리를 취득하는 단계; 기침 세그먼트들로부터 하나 이상의 기침 소리 피처를 나타내는 기침 소리 피처 신호들을 생성하기 위해 상기 기침 소리를 처리하는 단계; 상기 기침 소리 피처 신호들에 기초하여 하나 이상의 질병 시그니처를 획득하는 단계; 및 상기 기침 세그먼트들이 상기 질병들 중 하나 이상을 나타내는 것으로 간주하도록 상기 하나 이상의 질병 시그니처를 분류하는 단계를 포함하고, 상기 기침 소리 피처 신호들에 기초하여 상기 하나 이상의 질병 시그니처를 획득하는 단계는, 상기 기침 소리 피처들을 하나 이상의 미리 훈련된 질병 시그니처 결정 머신 각각에 적용하는 단계를 포함하고, 각각의 상기 결정 머신은 상기 기침 소리 피처들을 특정한 질병 또는 비질병 상태에 대응하는 것으로서 분류하거나 제1 특정한 질병 또는 상기 제1 특정한 질병과는 상이한 제2 특정한 질병에 대응하는 것으로서 분류하도록 미리 훈련되는 것을 특징으로 한다.On the other hand, Korean Patent Publication No. 10-2020-0122301 (Patent Document 1) discloses a "cough sound analysis method using a disease signature for diagnosing respiratory diseases", thereby diagnosing one or more diseases of the patient's airways. A method for doing this includes acquiring a cough sound from the patient; processing the cough sound to generate cough sound feature signals representing one or more cough sound features from cough segments; obtaining one or more disease signatures based on the cough sound feature signals; and classifying the one or more disease signatures to consider that the cough segments represent one or more of the diseases, wherein obtaining the one or more disease signatures based on the cough sound feature signals comprises: applying cough sound features to each of one or more pre-trained disease signature determination machines, each of which classifies the cough sound features as corresponding to a particular disease or non-disease state or to a first particular disease or It is characterized in that it is trained in advance to classify as corresponding to a second specific disease different from the first specific disease.
이상과 같은 특허문헌 1의 경우, 기침 소리 피처들을 질병 시그니처 결정 머신들에 적용하여 각각의 결정 머신은 상기 기침 소리 피처들을 특정한 질병 또는 비질병 상태에 대응하는 것으로서 분류하도록 훈련함으로써, 환자의 기침 소리 피처 신호로부터 하나 이상의 질병을 진단할 수 있는 장점이 있기는 하나, 이는 환자의 기침 소리 피처 신호만을 이용하고 있어, 환자의 다른 음, 예를 들면, 호흡음이나 낭독음 또는 발성음을 이용하여 호흡기 질환의 예후를 예측하는 것에는 적용하기 어려운 문제점을 내포하고 있다.In the case of Patent Document 1 as described above, the cough sound features are applied to disease signature decision machines, and each decision machine is trained to classify the cough sound features as those corresponding to a specific disease or non-disease state, so that the patient's cough sound Although it has the advantage of being able to diagnose one or more diseases from the feature signal, it uses only the feature signal of the patient's cough sound, so other sounds of the patient, such as breathing sounds, reading sounds, or vocalizations, can be used to diagnose the respiratory tract. Predicting the prognosis of a disease contains problems that are difficult to apply.
본 발명은 상기와 같은 사항을 종합적으로 감안하여 창출된 것으로서, 사용자의 기침음, 호흡음, 낭독음, 발성음의 주기적인 측정을 통해 음성데이터를 시계열로 분석하고, 분석된 정보를 기반으로 호흡기 질환의 예후를 예측함으로써, 사용자가 사전에 적절히 대응할 수 있도록 하는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 시스템 및 방법을 제공함에 그 목적이 있다.The present invention was created in consideration of the above matters comprehensively, and analyzes voice data in a time series through periodic measurement of the user's cough sound, breathing sound, reading sound, and pronunciation sound, and based on the analyzed information, respiratory An object of the present invention is to provide a respiratory disease prognosis prediction system and method through time-series coughing, breathing, reading, and vocalization measurements that allow users to respond appropriately in advance by predicting the prognosis of the disease.
상기의 목적을 달성하기 위하여 본 발명에 따른 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 시스템은,In order to achieve the above object, the respiratory disease prognosis prediction system through the measurement of time-series cough sound, breathing sound, reading sound, and vocalization sound according to the present invention,
사용자로부터 기침음, 호흡음, 낭독음 및 발성음을 주기적으로 측정하는 음성 측정부와;a voice measuring unit that periodically measures cough sounds, breath sounds, reading sounds, and vocal sounds from the user;
상기 음성 측정부에 의해 측정된 데이터에 대하여 호흡기 질환 중증도 점수를 평가한 학습용 데이터를 수집하는 데이터 수집부와;a data collection unit for collecting learning data obtained by evaluating a respiratory disease severity score with respect to the data measured by the voice measurement unit;
상기 수집된 학습용 데이터를 바탕으로 임의의 K 시간의 시계열 측정 데이터를 호흡기 질환 예후 예측모델의 입력값으로 하고, M 시간 후의 호흡기 질환 중증도 평가점수를 출력값으로 하는 호흡기 질환 예후 예측모델을 학습시키는 모델 학습부와; Based on the collected training data, model learning that trains a respiratory disease prognosis prediction model that takes the time series measurement data at any K time as an input value of the respiratory disease prognosis prediction model and uses the respiratory disease severity evaluation score after M hours as an output value. wealth;
상기 학습된 호흡기 질환 예후 예측모델에 상기 K 시간의 시계열 측정 데이터를 입력하고, M 시간 후에 상기 호흡기 질환 예후 예측모델이 출력하는 호흡기 질환 예후 예측값을 바탕으로 호흡기 질환 예후를 예측하는 호흡기 질환 예후 예측부; 및 A respiratory disease prognosis prediction unit that inputs the time-series measurement data at time K to the learned respiratory disease prognosis prediction model and predicts a respiratory disease prognosis based on a respiratory disease prognosis prediction value output by the respiratory disease prognosis prediction model after M time ; and
상기 데이터 수집부, 모델 학습부, 호흡기 질환 예후 예측부의 상태를 체크하고, 상기 데이터 수집부에 의한 호흡기 질환 중증도 평가 및 특정 클래스로의 분류와, 상기 모델 학습부에 의한 호흡기 질환 예후 예측모델의 학습과, 상기 호흡기 질환 예후 예측부에 의한 호흡기 질환 예후 예측을 위한 각각의 제어 명령을 송출하는 제어부를 포함하는 점에 그 특징이 있다.The status of the data collection unit, the model learning unit, and the respiratory disease prognosis prediction unit are checked, the respiratory disease severity is evaluated by the data collection unit and classified into a specific class, and the respiratory disease prognosis prediction model is learned by the model learning unit. And, it is characterized in that it includes a control unit that transmits each control command for predicting the respiratory disease prognosis by the respiratory disease prognosis prediction unit.
여기서, 상기 음성 측정부에 의해 측정되는 기침음은 스마트폰의 녹음 기능을 이용하여 환자의 기침 소리를 다수회 반복적으로 녹음함으로써 획득할 수 있다.Here, the cough sound measured by the voice measurement unit can be obtained by repeatedly recording the patient's cough sound multiple times using a recording function of a smartphone.
또한, 상기 호흡음은 스마트폰의 녹음 기능을 이용하여 환자의 들숨 및 날숨을 다수회 반복적으로 녹음함으로써 획득할 수 있다.In addition, the breathing sound can be obtained by repeatedly recording the patient's inhalation and exhalation multiple times using a recording function of a smartphone.
또한, 상기 낭독음은 스마트폰의 녹음 기능을 이용하여 제시된 문장을 평소 말투로 소리 내어 읽은 음을 녹음함으로써 획득할 수 있다.In addition, the reading sound may be obtained by recording a sound read aloud in a normal tone of speech using a recording function of a smartphone.
또한, 상기 발성음은 스마트폰의 녹음 기능을 이용하여 음의 높낮이가 있는 음을 익숙한 멜로디에 따라 부르면서 나오는 발성을 통한 목소리를 녹음함으로써 획득할 수 있다.In addition, the vocalized sound can be obtained by recording a voice through vocalization produced while singing a pitched sound according to a familiar melody using a recording function of a smartphone.
또한, 상기 기침음, 호흡음, 낭독음, 발성음은 1시간∼N시간 주기로 스마트폰을 이용하여 측정할 수 있다.In addition, the cough sound, breath sound, reading sound, and vocal sound can be measured using a smartphone at a period of 1 hour to N hours.
또한, 상기 데이터 수집부는 상기 측정된 데이터에 대하여 호흡기 질환 중증도를 평가하고, 이를 정상(0), 주의(1), 경계(2) 클래스로 분류할 수 있다.In addition, the data collection unit may evaluate the severity of respiratory disease with respect to the measured data, and classify it into normal (0), caution (1), and borderline (2) classes.
또한, 상기 모델 학습부는 상기 호흡기 질환 예후 예측모델을 학습시킴에 있어서, 상기 기침음, 호흡음, 낭독음, 발성음 측정 시간과 동일한 시간대의 미세먼지(PM10, PM2.5), 온/습도, CO2, VOCs(휘발성 유기화합물)를 포함하는 개인 환경 데이터 및 공공 환경 데이터와, 혈압/심박수 데이터를 함께 학습시킬 수 있다.In addition, the model learning unit learns the respiratory disease prognosis prediction model, fine dust (PM10, PM2.5), temperature / humidity, CO 2 , Personal environment data and public environment data including VOCs (volatile organic compounds), and blood pressure/heart rate data may be learned together.
또한, 상기 모델 학습부는 상기 호흡기 질환 예후 예측모델을 학습시킴에 있어서, 측정된 기침음, 호흡음, 낭독음, 발성음을 Mel-spectrogram을 이용하여 음성의 특징을 추출하고, 그것에 개인 생체, 환경 데이터와 공공 환경 데이터를 붙여 CNN(Convolutional Neural Network) 알고리즘을 이용하여 학습시킬 수 있다.In addition, in learning the respiratory disease prognosis prediction model, the model learning unit extracts the characteristics of the measured cough sound, breathing sound, reading sound, and vocalization sound using Mel-spectrogram, and extracts voice features from it. It can be trained using CNN (Convolutional Neural Network) algorithm by attaching data and public environment data.
이때, 상기 개인 생체, 환경 데이터 및 공공 환경 데이터는 24시간∼48시간의 데이터를 모두 이용할 수 있다.At this time, as the personal biometric data, environment data, and public environment data, all data of 24 hours to 48 hours may be used.
이때, 또한 기침음, 호흡음, 낭독음, 발성음 데이터와 상기 개인 생체, 환경 데이터와 공공 환경 데이터의 각각의 데이터에 대하여 x개의 채널로 구성하여 M_(x×m×n) 멀티채널 데이터를 만들고, CNN의 출력은 호흡기 질환 중증도 평가 점수의 회귀 모델로 구성할 수 있다.At this time, M_(x×m×n) multi-channel data is formed by configuring x number of channels for cough sound, breath sound, reading sound, and vocal sound data and each data of the personal biometric, environmental data, and public environment data created, and the output of the CNN can be configured as a regression model of the respiratory disease severity evaluation score.
또한, 상기의 목적을 달성하기 위하여 본 발명에 따른 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 방법은, 음성 측정부, 데이터 수집부, 모델 학습부, 호흡기 질환 예후 예측부를 포함하는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 시스템에 기반한 호흡기 질환 예후 예측 방법으로서,In addition, in order to achieve the above object, the method for predicting the prognosis of respiratory diseases through the measurement of time-series cough sound, breathing sound, reading sound, and vocalization sound according to the present invention includes a voice measurement unit, a data collection unit, a model learning unit, and a respiratory disease prognosis A respiratory disease prognosis prediction method based on a respiratory disease prognosis prediction system through time series cough sound, breathing sound, reading sound, and vocalization sound measurement including a prediction unit,
a) 상기 음성 측정부에 의해 사용자로부터 기침음, 호흡음, 낭독음 및 발성음을 주기적으로 측정하는 단계와;a) periodically measuring cough sound, breathing sound, reading sound, and vocalization sound from the user by the voice measuring unit;
b) 상기 데이터 수집부가 상기 측정된 데이터에 대하여 호흡기 질환 중증도점수를 평가한 학습용 데이터를 수집하는 단계와;b) collecting, by the data collection unit, learning data for evaluating respiratory disease severity scores with respect to the measured data;
c) 상기 모델 학습부가 상기 수집된 학습용 데이터를 바탕으로 임의의 K 시간의 시계열 측정 데이터를 호흡기 질환 예후 예측모델의 입력값으로 하고, M 시간 후의 호흡기 질환 중증도 평가점수를 출력값으로 하는 호흡기 질환 예후 예측모델을 학습시키는 단계; 및c) Respiratory disease prognosis prediction in which the model learning unit takes time-series measurement data at any K time as an input value of a respiratory disease prognosis prediction model based on the collected learning data, and the respiratory disease severity evaluation score after M time as an output value training the model; and
d) 상기 호흡기 질환 예후 예측부가 상기 학습된 호흡기 질환 예후 예측모델에 상기 K 시간의 시계열 측정 데이터를 입력하고, M 시간 후에 상기 호흡기 질환 예후 예측모델이 출력하는 호흡기 질환 예후 예측값을 바탕으로 호흡기 질환 예후를 예측하는 단계를 포함하는 점에 그 특징이 있다.d) The respiratory disease prognosis prediction unit inputs the time-series measurement data of the K time to the learned respiratory disease prognosis prediction model, and the respiratory disease prognosis is based on the respiratory disease prognosis prediction value output by the respiratory disease prognosis prediction model after M time Its feature is that it includes the step of predicting .
여기서, 상기 단계 a)에서 상기 기침음은 스마트폰의 녹음 기능을 이용하여 환자의 기침 소리를 다수회 반복적으로 녹음함으로써 획득할 수 있다.Here, in step a), the cough sound can be obtained by repeatedly recording the patient's cough sound multiple times using a recording function of a smart phone.
또한, 상기 호흡음은 스마트폰의 녹음 기능을 이용하여 환자의 들숨 및 날숨을 다수회 반복적으로 녹음함으로써 획득할 수 있다.In addition, the breathing sound can be obtained by repeatedly recording the patient's inhalation and exhalation multiple times using a recording function of a smartphone.
또한, 상기 낭독음은 스마트폰의 녹음 기능을 이용하여 제시된 문장을 평소 말투로 소리 내어 읽은 음을 녹음함으로써 획득할 수 있다.In addition, the reading sound may be obtained by recording a sound read aloud in a normal tone of speech using a recording function of a smartphone.
또한, 상기 발성음은 스마트폰의 녹음 기능을 이용하여 음의 높낮이가 있는 음을 익숙한 멜로디에 따라 부르면서 나오는 발성을 통한 목소리를 녹음함으로써 획득할 수 있다.In addition, the vocalized sound can be obtained by recording a voice through vocalization produced while singing a pitched sound according to a familiar melody using a recording function of a smartphone.
또한, 상기 기침음, 호흡음, 낭독음, 발성음은 1시간∼N시간 주기로 스마트폰을 이용하여 측정할 수 있다.In addition, the cough sound, breath sound, reading sound, and vocal sound can be measured using a smartphone at a period of 1 hour to N hours.
또한, 상기 단계 b)에서 상기 측정된 데이터에 대하여 호흡기 질환 중증도를 평가하고, 이를 정상(0), 주의(1), 경계(2) 클래스로 분류할 수 있다.In addition, the respiratory disease severity may be evaluated for the data measured in step b), and classified into normal (0), caution (1), and borderline (2) classes.
또한, 상기 단계 c)에서 상기 호흡기 질환 예후 예측모델을 학습시킴에 있어서, 상기 기침음, 호흡음, 낭독음, 발성음 측정 시간과 동일한 시간대의 미세먼지(PM10, PM2.5), 온/습도, CO2, VOCs(휘발성 유기화합물)를 포함하는 개인 환경 데이터 및 공공 환경 데이터와, 혈압/심박수 데이터를 함께 학습시킬 수 있다.In addition, in learning the respiratory disease prognosis prediction model in step c), fine dust (PM10, PM2.5), temperature / humidity in the same time zone as the cough sound, breathing sound, reading sound, and pronunciation sound measurement time , CO 2 , personal environment data and public environment data including VOCs (volatile organic compounds), and blood pressure/heart rate data may be learned together.
또한, 상기 단계 c)에서 상기 호흡기 질환 예후 예측모델을 학습시킴에 있어서, 측정된 기침음, 호흡음, 낭독음, 발성음을 Mel-spectrogram을 이용하여 음성의 특징을 추출하고, 그것에 개인 생체, 환경 데이터와 공공 환경 데이터를 붙여 CNN (Convolutional Neural Network) 알고리즘을 이용하여 학습시킬 수 있다.In addition, in learning the respiratory disease prognosis prediction model in step c), the measured cough sound, breathing sound, reading sound, and vocalization sound are extracted using the Mel-spectrogram, and the characteristics of the voice are extracted, and personal biopsies, It can be trained using CNN (Convolutional Neural Network) algorithm by attaching environmental data and public environment data.
이때, 상기 개인 생체, 환경 데이터 및 공공 환경 데이터는 24시간∼48시간의 데이터를 모두 이용할 수 있다.At this time, as the personal biometric data, environment data, and public environment data, all data of 24 hours to 48 hours may be used.
이때, 또한 기침음, 호흡음, 낭독음, 발성음 데이터와 상기 개인 생체, 환경 데이터와 공공 환경 데이터의 각각의 데이터에 대하여 x개의 채널로 구성하여 M_(x×m×n) 멀티채널 데이터를 만들고, CNN의 출력은 호흡기 질환 중증도 평가점수의 회귀모델로 구성할 수 있다. At this time, M_(x×m×n) multi-channel data is formed by configuring x number of channels for cough sound, breath sound, reading sound, and vocal sound data and each data of the personal biometric, environmental data, and public environment data and the output of the CNN can be configured as a regression model of the respiratory disease severity evaluation score.
이와 같은 본 발명에 의하면, 기침음, 호흡음, 낭독음, 발성음의 주기적인 측정을 통해 음성데이터를 시계열로 분석하고, 분석된 정보를 기반으로 호흡기 질환의 예후를 예측함으로써, 사용자가 사전에 적절히 대응할 수 있도록 하는 장점이 있다.According to the present invention, the voice data is analyzed in a time series through periodic measurement of cough sounds, breath sounds, reading sounds, and pronunciation sounds, and the prognosis of respiratory diseases is predicted based on the analyzed information, so that the user can It has the advantage of being able to respond appropriately.
또한, 개인의 심박음 등의 생체 데이터와, 개인 환경 데이터 및 공공 환경 데이터를 함께 이용하여 주변 환경의 변화에 따른 호흡기 질환의 예후를 예측함으로써, 호흡기 질환의 예후 예측의 정확도를 한층 더 높일 수 있는 장점이 있다. In addition, by predicting the prognosis of respiratory diseases according to changes in the surrounding environment by using biometric data such as individual heartbeats, personal environment data and public environment data together, the accuracy of predicting the prognosis of respiratory diseases can be further improved There are advantages.
도 1은 본 발명에 따른 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 시스템의 구성을 개략적으로 나타낸 도면이다. 1 is a diagram schematically showing the configuration of a respiratory disease prognosis prediction system through time-series cough sound, breathing sound, reading sound, and vocal sound measurement according to the present invention.
도 2는 본 발명에 따른 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 방법의 실행 과정을 나타낸 흐름도이다.2 is a flowchart showing the execution process of the method for predicting the prognosis of respiratory diseases through the measurement of time-series cough sounds, breathing sounds, reading sounds, and vocal sounds according to the present invention.
도 3은 스마트폰의 녹음 기능을 이용하여 기침음, 호흡음, 낭독음, 발성음을 측정하는 개요를 나타낸 도면이다.3 is a diagram showing an outline of measuring cough sound, breathing sound, reading sound, and vocalization sound using a recording function of a smartphone.
도 4는 호흡기 질환 중증도에 대한 평가를 하여 정상, 주의, 경계 클래스로 분류하는 것을 나타낸 도면이다.Figure 4 is a diagram showing the evaluation of the severity of respiratory diseases and classification into normal, caution, and alert classes.
도 5는 호흡기 질환 예후 예측모델이 함께 학습하는 개인 환경 데이터 및 공공 환경 데이터와 혈압/심박수 데이터를 나타낸 도면이다.5 is a diagram showing personal environment data, public environment data, and blood pressure/heart rate data that a respiratory disease prognosis prediction model learns together.
도 6은 측정된 기침음, 호흡음, 낭독음, 발성음의 특징을 추출하고, 개인 생체, 환경 데이터와 공공 환경 데이터를 붙여 학습하는 개요를 나타낸 도면이다.6 is a diagram showing an overview of extracting characteristics of measured cough sounds, breath sounds, reading sounds, and pronunciation sounds, and learning by attaching personal biometric data, environmental data, and public environment data.
이하 첨부된 도면을 참조하여 본 발명의 실시예를 상세히 설명한다.Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
도 1은 본 발명의 실시예에 따른 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 시스템의 구성을 개략적으로 나타낸 도면이다.1 is a diagram schematically showing the configuration of a respiratory disease prognosis prediction system through time-series cough sound, breathing sound, reading sound, and vocal sound measurement according to an embodiment of the present invention.
도 1을 참조하면, 본 발명에 따른 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 시스템(100)은 음성 측정부(110), 데이터 수집부(120), 모델 학습부(130), 호흡기 질환 예후 예측부(140) 및 제어부(150)를 포함하여 구성된다.Referring to FIG. 1, the respiratory disease prognosis prediction system 100 through the measurement of time-series cough sound, breathing sound, reading sound, and vocalization sound according to the present invention includes a voice measurement unit 110, a data collection unit 120, and model learning It is configured to include a unit 130, a respiratory disease prognosis prediction unit 140, and a control unit 150.
음성 측정부(110)는 사용자로부터 기침음, 호흡음, 낭독음 및 발성음을 주기적으로 측정한다. 여기서, 이와 같은 음성 측정부(110)로는 스마트폰이 사용될 수 있다. 또한, 이상과 같은 음성 측정부(110)에 의해 측정되는 기침음은, 도 3의 (a)와 같이 스마트폰의 녹음 기능을 이용하여 환자의 기침 소리를 다수회(예를 들면, 3∼5회) 반복적으로 녹음함으로써 획득할 수 있다. 또한, 상기 호흡음은 (b)와 같이, 스마트폰의 녹음 기능을 이용하여 환자의 들숨 및 날숨을 다수회(예를 들면, 3∼5회) 반복적으로 녹음함으로써 획득할 수 있다. 또한, 상기 낭독음은 (c)와 같이, 스마트폰의 녹음 기능을 이용하여 제시된 문장(예를 들면, '나는 더 건강한 사회를 위해 음성을 녹음하는 것을 동의합니다.')을 평소 말투로 소리 내어 읽은 음을 녹음함으로써 획득할 수 있다. 또한, 상기 발성음은 (d)와 같이, 스마트폰의 녹음 기능을 이용하여 음의 높낮이가 있는 음(예를 들면, 글로벌 공통 음계인 '도레미파솔라시도' 같은 음의 높낮이가 있는 음)을 익숙한 멜로디에 따라 부르면서 나오는 발성을 통한 목소리를 녹음함으로써 획득할 수 있다. 또한, 이상과 같은 기침음, 호흡음, 낭독음, 발성음은 1시간∼N시간 주기로 스마트폰을 이용하여 측정할 수 있다.The voice measurement unit 110 periodically measures cough sounds, breathing sounds, reading sounds, and vocal sounds from the user. Here, a smartphone may be used as such a voice measurement unit 110 . In addition, the cough sound measured by the voice measurement unit 110 as described above is a patient's cough sound multiple times (eg, 3 to 5 times) can be obtained by repeatedly recording. In addition, as in (b), the breathing sound can be obtained by repeatedly recording the patient's inhalation and exhalation multiple times (eg, 3 to 5 times) using a recording function of a smartphone. In addition, as in (c), the reading sound is a sentence presented using the recording function of the smartphone (eg, 'I agree to record voices for a healthier society') in a normal tone. It can be obtained by recording the sound you read. In addition, as in (d), the voiced sound is a pitched sound (for example, a pitched sound such as 'Do Re Mi Fa Sol La Si Do', a global common scale) using the recording function of a smartphone to a familiar melody. It can be obtained by recording the voice through vocalization while singing along. In addition, cough sounds, breath sounds, reading sounds, and vocal sounds as described above can be measured using a smartphone at intervals of 1 hour to N hours.
데이터 수집부(120)는 상기 음성 측정부(110)에 의해 측정된 데이터에 대하여 호흡기 질환 중증도 점수로 평가하여 학습용 데이터를 수집한다. 여기서, 이러한 데이터 수집부(120)는, 도 4에 도시된 바와 같이, 상기 측정된 데이터에 대하여 호흡기 질환 중증도를 점수로 평가한다.The data collection unit 120 collects learning data by evaluating the data measured by the voice measurement unit 110 as a respiratory disease severity score. Here, the data collection unit 120, as shown in FIG. 4, evaluates the respiratory disease severity as a score for the measured data.
모델 학습부(130)는 상기 수집된 학습용 데이터를 바탕으로, 도 4에 도시된 바와 같이 임의의 K 시간의 시계열 측정 데이터를 호흡기 질환 예후 예측모델(160)의 입력값으로 하고, M 시간 후의 호흡기 질환 중증도 평가 점수를 출력값으로 하는 호흡기 질환 예후 예측모델(160)을 학습시킨다. 여기서, 이와 같은 모델 학습부 (130)는 상기 호흡기 질환 예후 예측모델(160)을 학습시킴에 있어서, 도 5에 도시된 바와 같이, 상기 기침음, 호흡음, 낭독음, 발성음 측정 시간과 동일한 시간대의 미세먼지(PM10, PM2.5), 온/습도, CO2, VOCs(휘발성 유기화합물)를 포함하는 개인 환경 데이터 및 공공 환경 데이터와, 혈압/심박수 데이터를 함께 학습시킬 수 있다. 또한, 이상과 같은 모델 학습부(140)는 상기 호흡기 질환 예후 예측모델(160)을 학습시킴에 있어서, 도 6에 도시된 바와 같이, 측정된 기침음, 호흡음, 낭독음, 발성음을 Mel-spectrogram을 이용하여 음성의 특징을 추출하고, 그것에 개인 생체, 환경 데이터와 공공 환경 데이터를 붙여 CNN(Convolutional Neural Network) 알고리즘을 이용하여 학습시킬 수 있다. 이때, 상기 개인 생체, 환경 데이터 및 공공 환경 데이터는 24시간∼48시간의 데이터를 모두 이용할 수 있다. 즉, 상기 개인 생체, 환경 데이터 및 공공 환경 데이터의 경우, 증상에 영향을 미치는 시점이 현재 시간 또는 N시간 전의 상황이 현재에 영향을 미칠 수 있기 때문에, 24시간∼48시간의 데이터를 모두 이용하는 것이 바람직하다.Based on the collected learning data, the model learning unit 130 sets the time-series measurement data of any K time as an input value of the respiratory disease prognosis prediction model 160 as shown in FIG. 4, and the respiratory disease after M time A respiratory disease prognosis prediction model 160 having a disease severity evaluation score as an output value is trained. Here, when the model learning unit 130 learns the respiratory disease prognosis prediction model 160, as shown in FIG. Personal environment data and public environment data including fine dust (PM10, PM2.5), temperature/humidity, CO 2 , VOCs (volatile organic compounds) of the time zone, and blood pressure/heart rate data can be learned together. In addition, in learning the respiratory disease prognosis prediction model 160, the model learning unit 140 as described above, as shown in FIG. 6, Mel Voice features can be extracted using -spectrogram, and personal biometric, environmental data, and public environment data can be attached to it and trained using CNN (Convolutional Neural Network) algorithm. At this time, as the personal biometric data, environment data, and public environment data, all data of 24 to 48 hours can be used. That is, in the case of the personal biometric data, environmental data, and public environmental data, since the time point influencing symptoms can affect the current time or the situation N hours ago can affect the present, it is recommended to use all data from 24 to 48 hours. desirable.
이때, 또한 기침음, 호흡음, 낭독음, 발성음 데이터와 상기 개인 생체, 환경 데이터와 공공 환경 데이터의 각각의 데이터에 대하여 x개의 채널로 구성하여 M_(x×m×n) 멀티채널 데이터를 만들고, CNN의 출력은 호흡기 질환 중증도 평가점수 회귀모델로 구성할 수 있다.At this time, M_(x×m×n) multi-channel data is formed by configuring x number of channels for cough sound, breath sound, reading sound, and vocal sound data and each data of the personal biometric, environmental data, and public environment data created, and the output of the CNN can be configured as a respiratory disease severity evaluation score regression model.
호흡기 질환 예후 예측부(140)는 도 4에 도시된 바와 같이, 상기 학습된 호흡기 질환 예후 예측모델(160)에 상기 K 시간의 시계열 측정 데이터를 입력하고, M 시간 후에 상기 호흡기 질환 예후 예측모델(160)이 출력하는 호흡기 질환 예후 예측값을 바탕으로 호흡기 질환 예후를 예측한다. 예를 들면, 호흡기 질환 예후 예측부(140)는 호흡기 질환 예후 예측모델(160)이 출력하는 호흡기 질환 예후 예측값을 미리 설정된 기준값(기준 범위)과 비교하여 예측값이 기준값(기준 범위)(예를 들면, 기준값의 0∼1점)에 속하면 "정상 상태", 예측값이 기준값(기준 범위)에 대해 제1 임계값(예를 들면, 기준값의 2∼4점)에 속하면 "주의 상태", 제2 임계값(예를 들면, 기준값의 5점)에 속하면 "경계 상태"로 호흡기 질환 예후를 예측한다. 이와 같은 호흡기 질환 예후 예측부(140)는 마이크로프로세서나 마이크로컨트롤러 등으로 구성될 수 있다.As shown in FIG. 4, the respiratory disease prognosis prediction unit 140 inputs the time-series measurement data of the K time to the learned respiratory disease prognosis prediction model 160, and after M time, the respiratory disease prognosis prediction model ( 160) predicts the respiratory disease prognosis based on the respiratory disease prognosis prediction value output. For example, the respiratory disease prognosis prediction unit 140 compares the respiratory disease prognosis prediction value output by the respiratory disease prognosis prediction model 160 with a preset reference value (reference range) so that the predicted value is a reference value (reference range) (eg , 0 to 1 point of the reference value), "steady state", if the predicted value belongs to the first threshold value (for example, 2 to 4 points of the reference value) for the reference value (reference range), "attention state", If it falls within 2 thresholds (eg, 5 points of the reference value), the respiratory disease prognosis is predicted as "alert state". Such a respiratory disease prognosis prediction unit 140 may be configured with a microprocessor or microcontroller.
제어부(150)는 상기 데이터 수집부(120), 모델 학습부(130), 호흡기 질환 예후 예측부(140)의 상태를 체크 및 동작을 제어하고, 상기 데이터 수집부(120)에 의한 호흡기 질환 중증도 평가 및 특정 클래스로의 분류와, 상기 모델 학습부(130)에 의한 호흡기 질환 예후 예측모델(160)의 학습과, 상기 호흡기 질환 예후 예측부 (140)에 의한 호흡기 질환 예후 예측을 위한 각각의 제어 명령을 송출한다. 이와 같은 제어부(150)는 마이크로프로세서나 마이크로컨트롤러 등으로 구성될 수 있다.The control unit 150 checks the status of the data collection unit 120, the model learning unit 130, and the respiratory disease prognosis prediction unit 140 and controls their operations, and determines the severity of respiratory disease by the data collection unit 120. Each control for evaluation and classification into a specific class, learning of the respiratory disease prognosis prediction model 160 by the model learning unit 130, and respiratory disease prognosis prediction by the respiratory disease prognosis prediction unit 140 send out command Such control unit 150 may be composed of a microprocessor or microcontroller.
여기서, 이상과 같은 데이터 수집부(120), 모델 학습부(130), 호흡기 질환 예후 예측부(140) 및 제어부(150)는 통합되어 하나의 컴퓨터 시스템으로 구성될 수 있다.Here, the data collection unit 120, the model learning unit 130, the respiratory disease prognosis prediction unit 140, and the control unit 150 as described above may be integrated into one computer system.
그러면, 이하에서는 이상과 같은 구성을 가지는 본 발명에 따른 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 시스템에 기반한 호흡기 질환 예후 예측 방법에 대해 설명해 보기로 한다.Hereinafter, a respiratory disease prognosis prediction method based on the respiratory disease prognosis prediction system through time series cough sound, breathing sound, reading sound, and speech sound measurement according to the present invention having the above configuration will be described.
도 2는 본 발명의 실시예에 따른 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 방법의 실행 과정을 나타낸 흐름도이다.2 is a flowchart illustrating an execution process of a method for predicting the prognosis of a respiratory disease through measurement of time-series coughing sound, breathing sound, reading sound, and vocalization sound according to an embodiment of the present invention.
도 2를 참조하면, 본 발명에 따른 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 방법은, 전술한 바와 같은 음성 측정부(110), 데이터 수집부(120), 모델 학습부(130), 호흡기 질환 예후 예측부(140) 및 제어부(150)를 포함하는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 시스템(100)에 기반한 호흡기 질환 예후 예측 방법으로서, 먼저 음성 측정부(110)에 의해(즉, 사용자 단말기(예컨대, 스마트폰)를 이용하여), 사용자로부터 기침음, 호흡음, 낭독음 및 발성음을 주기적으로 측정한다(단계 S201). 여기서, 상기 기침음은, 도 3의 (a)와 같이 스마트폰의 녹음 기능을 이용하여 환자의 기침 소리를 다수회(예를 들면, 3∼5회) 반복적으로 녹음함으로써 획득할 수 있다. 또한, 상기 호흡음은 (b)와 같이, 스마트폰의 녹음 기능을 이용하여 환자의 들숨 및 날숨을 다수회(예를 들면, 3∼5회) 반복적으로 녹음함으로써 획득할 수 있다. 또한, 상기 낭독음은 (c)와 같이, 스마트폰의 녹음 기능을 이용하여 제시된 문장(예를 들면, '나는 더 건강한 사회를 위해 음성을 녹음하는 것을 동의합니다.')을 평소 말투로 소리 내어 읽은 음을 녹음함으로써 획득할 수 있다. 또한, 상기 발성음은 (d)와 같이, 스마트폰의 녹음 기능을 이용하여 음의 높낮이가 있는 음(예를 들면, 글로벌 공통 음계인 '도레미파솔라시도' 같은 음의 높낮이가 있는 음)을 익숙한 멜로디에 따라 부르면서 나오는 발성을 통한 목소리를 녹음함으로써 획득할 수 있다. 또한, 이상과 같은 기침음, 호흡음, 낭독음, 발성음은 1시간∼N시간 주기로 스마트폰을 이용하여 측정할 수 있다.Referring to FIG. 2 , the method for predicting the prognosis of respiratory diseases through the measurement of time-series cough sound, breathing sound, reading sound, and vocalization sound according to the present invention includes the voice measurement unit 110, the data collection unit 120, Respiratory disease based on respiratory disease prognosis prediction system 100 through measurement of time-series cough sound, breathing sound, reading sound, and vocalization sound including model learning unit 130, respiratory disease prognosis prediction unit 140, and control unit 150 As a prognosis prediction method, first, the cough sound, breathing sound, reading sound, and vocalization sound are periodically measured from the user by the voice measuring unit 110 (ie, using a user terminal (eg, a smartphone)) (step S201). Here, the cough sound can be obtained by repeatedly recording the patient's cough sound multiple times (eg, 3 to 5 times) using a recording function of a smartphone, as shown in FIG. 3 (a). In addition, as in (b), the breathing sound can be obtained by repeatedly recording the patient's inhalation and exhalation multiple times (eg, 3 to 5 times) using a recording function of a smartphone. In addition, as in (c), the reading sound is a sentence presented using the recording function of the smartphone (eg, 'I agree to record voices for a healthier society') in a normal tone. It can be obtained by recording the sound you read. In addition, as in (d), the voiced sound is a pitched sound (for example, a pitched sound such as 'Do Re Mi Fa Sol La Si Do', a global common scale) using the recording function of a smartphone to a familiar melody. It can be obtained by recording the voice through vocalization while singing along. In addition, cough sounds, breath sounds, reading sounds, and vocal sounds as described above can be measured using a smartphone at intervals of 1 hour to N hours.
이렇게 하여 음성 측정부(110)에 의해 기침음, 호흡음, 낭독음 및 발성음이 측정되면, 데이터 수집부(120)는 상기 측정된 데이터에 대하여 호흡기 질환 중증도점수로 평가하여 학습용 데이터를 수집한다(단계 S202). 여기서, 이러한 데이터 수집부(120)는, 도 4에 도시된 바와 같이, 상기 측정된 데이터에 대하여 호흡기 질환 중증도 점수를 평가할 수 있다.In this way, when the cough sound, breathing sound, reading sound, and vocalization sound are measured by the voice measurement unit 110, the data collection unit 120 evaluates the measured data as a respiratory disease severity score and collects learning data. (Step S202). Here, such a data collection unit 120, as shown in Figure 4, can evaluate the respiratory disease severity score with respect to the measured data.
이상에 의해 데이터 수집이 완료되면, 모델 학습부(130)는 상기 수집된 학습용 데이터를 바탕으로 임의의 K 시간의 시계열 측정 데이터를 호흡기 질환 예후 예측모델(160)의 입력값으로 하고, M 시간 후의 호흡기 질환 중증도 평가점수를 출력값으로 하는 호흡기 질환 예후 예측모델(160)을 학습시킨다(단계 S203). 여기서, 이와 같은 호흡기 질환 예후 예측모델(160)을 학습시킴에 있어서, 도 5에 도시된 바와 같이, 상기 기침음, 호흡음, 낭독음, 발성음 측정 시간과 동일한 시간대의 미세먼지(PM10, PM2.5), 온/습도, CO2, VOCs(휘발성 유기화합물)를 포함하는 개인 환경 데이터 및 공공 환경 데이터와, 혈압/심박수 데이터를 함께 학습시킬 수 있다. 또한, 상기 모델 학습부(130)가 상기 호흡기 질환 예후 예측모델(160)을 학습시킴에 있어서, 도 6에 도시된 바와 같이, 측정된 기침음, 호흡음, 낭독음, 발성음을 Mel-spectrogram을 이용하여 음성의 특징을 추출하고, 그것에 개인 생체, 환경 데이터와 공공 환경 데이터를 붙여 CNN(Convolutional Neural Network) 알고리즘을 이용하여 학습시킬 수 있다. 이때, 상기 개인 생체, 환경 데이터 및 공공 환경 데이터는 24시간∼48시간의 데이터를 모두 이용할 수 있다. 이는 전술한 바와 같이, 상기 개인 생체, 환경 데이터 및 공공 환경 데이터의 경우, 증상에 영향을 미치는 시점이 현재 시간 또는 N시간 전의 상황이 현재에 영향을 미칠 수 있기 때문에, 24시간∼48시간의 데이터를 모두 이용하는 것이 바람직하다.When the data collection is completed as described above, the model learning unit 130 sets the time series measurement data of arbitrary K hours as an input value of the respiratory disease prognosis prediction model 160 based on the collected learning data, and M hours later A respiratory disease prognosis prediction model 160 having a respiratory disease severity evaluation score as an output value is trained (step S203). Here, in learning such a respiratory disease prognosis prediction model 160, as shown in FIG. 5, fine dust (PM10, PM2) in the same time zone as the cough sound, breathing sound, reading sound, and pronunciation sound measurement time .5), temperature/humidity, CO 2 , personal environment data and public environment data including VOCs (volatile organic compounds), and blood pressure/heart rate data can be learned together. In addition, when the model learning unit 130 learns the respiratory disease prognosis prediction model 160, as shown in FIG. 6, the measured cough sound, breathing sound, reading sound, and vocalization sound are Mel-spectrogram It is possible to extract features of voice using , and attach personal biometric data, environmental data, and public environment data to it, and learn them using a Convolutional Neural Network (CNN) algorithm. At this time, as the personal biometric data, environment data, and public environment data, all data of 24 to 48 hours can be used. As described above, in the case of the personal biometric data, environmental data, and public environment data, since the time point influencing symptoms can affect the current time or the situation N hours ago can affect the present, data of 24 to 48 hours It is preferable to use all of them.
이때, 또한 기침음, 호흡음, 낭독음, 발성음 데이터와 상기 개인 생체, 환경 데이터와 공공 환경 데이터의 각각의 데이터에 대하여 x개의 채널로 구성하여 M_(x×m×n) 멀티채널 데이터를 만들고, CNN의 출력은 호흡기 질환 중증도 평가 점수를 출력하는 회귀 모델로 구성할 수 있다. At this time, M_(x×m×n) multi-channel data is formed by configuring x number of channels for cough sound, breath sound, reading sound, and vocal sound data and each data of the personal biometric, environmental data, and public environment data created, and the output of the CNN can be configured as a regression model that outputs respiratory disease severity evaluation scores.
이후, 호흡기 질환 예후 예측부(140)는 도 4에 도시된 바와 같이, 상기 학습된 호흡기 질환 예후 예측모델(160)에 상기 K 시간의 시계열 측정 데이터를 입력하고, M 시간 후에 상기 호흡기 질환 예후 예측모델(160)이 출력하는 호흡기 질환 예후 예측값을 바탕으로 호흡기 질환 예후를 예측한다(단계 S204). 예를 들면, 호흡기 질환 예후 예측부(140)는 호흡기 질환 예후 예측모델(160)이 출력하는 호흡기 질환 예후 예측값을 미리 설정된 기준값(기준 범위)과 비교하여 예측값이 기준값(기준 범위)(예를 들면, 기준값의 0∼1점)에 속하면 "정상 상태", 예측값이 기준값(기준 범위)에 대해 제1 임계값(예를 들면, 기준값의 2∼4점)에 속하면 "주의 상태", 제2 임계값(예를 들면, 기준값의 5점)에 속하면 "경계 상태"로 호흡기 질환 예후를 예측한다.Thereafter, as shown in FIG. 4 , the respiratory disease prognosis prediction unit 140 inputs the time-series measurement data of time K into the learned respiratory disease prognosis prediction model 160, and predicts the respiratory disease prognosis after M time. A respiratory disease prognosis is predicted based on the respiratory disease prognosis prediction value output by the model 160 (step S204). For example, the respiratory disease prognosis prediction unit 140 compares the respiratory disease prognosis prediction value output by the respiratory disease prognosis prediction model 160 with a preset reference value (reference range) so that the predicted value is a reference value (reference range) (eg , 0 to 1 point of the reference value), "steady state", if the predicted value belongs to the first threshold value (for example, 2 to 4 points of the reference value) for the reference value (reference range), "attention state", If it falls within 2 thresholds (eg, 5 points of the reference value), the respiratory disease prognosis is predicted as "alert state".
이상의 설명과 같이, 본 발명에 따른 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 시스템 및 방법은 기침음, 호흡음, 낭독음, 발성음의 주기적인 측정을 통해 음성데이터를 시계열로 분석하고, 분석된 정보를 기반으로 호흡기 질환의 예후를 예측함으로써, 사용자가 사전에 적절히 대응할 수 있도록 하는 장점이 있다.As described above, the system and method for predicting the prognosis of respiratory diseases through the measurement of time-series cough sound, breathing sound, reading sound, and vocalization sound according to the present invention provide voice through periodic measurement of cough sound, breathing sound, reading sound, and vocalization sound. By analyzing data in time series and predicting the prognosis of respiratory diseases based on the analyzed information, there is an advantage in enabling users to respond appropriately in advance.
또한, 개인의 심박음 등의 생체 데이터와, 개인 환경 데이터 및 공공 환경 데이터를 함께 이용하여 주변 환경의 변화에 따른 호흡기 질환의 예후를 예측함으로써, 호흡기 질환의 예후 예측의 정확도를 한층 더 높일 수 있는 장점이 있다. In addition, by predicting the prognosis of respiratory diseases according to changes in the surrounding environment by using biometric data such as individual heartbeats, personal environment data and public environment data together, the accuracy of predicting the prognosis of respiratory diseases can be further improved There are advantages.

Claims (22)

  1. 사용자로부터 기침음, 호흡음, 낭독음 및 발성음을 주기적으로 측정하는 음성 측정부와;a voice measuring unit that periodically measures cough sounds, breath sounds, reading sounds, and vocal sounds from the user;
    상기 음성 측정부에 의해 측정된 데이터에 대하여 호흡기 질환 중증도 점수를 평가한 학습용 데이터를 수집하는 데이터 수집부와;a data collection unit for collecting learning data obtained by evaluating a respiratory disease severity score with respect to the data measured by the voice measurement unit;
    상기 수집된 학습용 데이터를 바탕으로 임의의 K 시간의 시계열 측정 데이터를 호흡기 질환 예후 예측모델의 입력값으로 하고, M 시간 후의 호흡기 질환 중증도 평가점수를 출력값으로 하는 호흡기 질환 예후 예측모델을 학습시키는 모델 학습부와; Based on the collected training data, model learning that trains a respiratory disease prognosis prediction model that takes the time series measurement data at any K time as an input value of the respiratory disease prognosis prediction model and uses the respiratory disease severity evaluation score after M hours as an output value. wealth;
    상기 학습된 호흡기 질환 예후 예측모델에 상기 K 시간의 시계열 측정 데이터를 입력하고, M 시간 후에 상기 호흡기 질환 예후 예측모델이 출력하는 호흡기 질환 예후 예측값을 바탕으로 호흡기 질환 예후를 예측하는 호흡기 질환 예후 예측부; 및 A respiratory disease prognosis prediction unit that inputs the time-series measurement data at time K to the learned respiratory disease prognosis prediction model and predicts a respiratory disease prognosis based on a respiratory disease prognosis prediction value output by the respiratory disease prognosis prediction model after M time ; and
    상기 데이터 수집부, 모델 학습부, 호흡기 질환 예후 예측부의 상태를 체크하고, 상기 데이터 수집부에 의한 호흡기 질환 중증도 평가 및 특정 클래스로의 분류와, 상기 모델 학습부에 의한 호흡기 질환 예후 예측모델의 학습과, 상기 호흡기 질환 예후 예측부에 의한 호흡기 질환 예후 예측을 위한 각각의 제어 명령을 송출하는 제어부를 포함하는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 시스템.The status of the data collection unit, the model learning unit, and the respiratory disease prognosis prediction unit are checked, the respiratory disease severity is evaluated by the data collection unit and classified into a specific class, and the respiratory disease prognosis prediction model is learned by the model learning unit. And, a respiratory disease prognosis prediction system through time-series cough sound, breathing sound, reading sound, and pronunciation sound measurement including a control unit that transmits each control command for predicting the respiratory disease prognosis by the respiratory disease prognosis prediction unit.
  2. 제1항에 있어서,According to claim 1,
    상기 음성 측정부에 의해 측정되는 기침음은 스마트폰의 녹음 기능을 이용하여 환자의 기침 소리를 다수회 반복적으로 녹음함으로써 획득하는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 시스템.The cough sound measured by the voice measurement unit is obtained by repeatedly recording the patient's cough sound multiple times using the recording function of the smartphone. Respiratory disease prognosis through measurement of time-series cough sound, breathing sound, reading sound, and vocalization sound prediction system.
  3. 제1항에 있어서,According to claim 1,
    상기 호흡음은 스마트폰의 녹음 기능을 이용하여 환자의 들숨 및 날숨을 다수회 반복적으로 녹음함으로써 획득하는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 시스템.The breathing sound is obtained by repeatedly recording the patient's inhalation and exhalation multiple times using the recording function of the smartphone. Respiratory disease prognosis prediction system through measurement of time-series cough sound, breathing sound, reading sound, and vocalization sound.
  4. 제1항에 있어서,According to claim 1,
    상기 낭독음은 스마트폰의 녹음 기능을 이용하여 제시된 문장을 평소 말투로 소리 내어 읽은 음을 녹음함으로써 획득하는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 시스템.The reading sound is a respiratory disease prognosis prediction system through the measurement of time-series cough sound, breathing sound, reading sound, and pronunciation sound obtained by recording the sound read aloud by using the recording function of the smartphone.
  5. 제1항에 있어서,According to claim 1,
    상기 발성음은 스마트폰의 녹음 기능을 이용하여 음의 높낮이가 있는 음을 익숙한 멜로디에 따라 부르면서 나오는 발성을 통한 목소리를 녹음함으로써 획득하는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 시스템.The vocalization sound is obtained by recording the voice through the vocalization produced while singing the pitched sound according to the familiar melody using the recording function of the smartphone. Respiratory disease prognosis prediction system.
  6. 제1항에 있어서,According to claim 1,
    상기 기침음, 호흡음, 낭독음, 발성음은 1시간∼N시간 주기로 스마트폰을 이용하여 측정하는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 시스템.The cough sound, breath sound, reading sound, and pronunciation sound are measured using a smartphone in a period of 1 hour to N hours. Respiratory disease prognosis prediction system through time-series cough sound, breathing sound, reading sound, and pronunciation sound measurement.
  7. 제1항에 있어서,According to claim 1,
    상기 데이터 수집부는 상기 측정된 데이터에 대하여 호흡기 질환 중증도를 평가하고, 이를 정상(0), 주의(1), 경계(2) 클래스로 분류하는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 시스템.The data collection unit evaluates the severity of respiratory diseases with respect to the measured data, and classifies them into normal (0), caution (1), and border (2) classes. Respiratory disease prognosis prediction system through
  8. 제1항에 있어서,According to claim 1,
    상기 모델 학습부는 상기 호흡기 질환 예후 예측모델을 학습시킴에 있어서, 상기 기침음, 호흡음, 낭독음, 발성음 측정 시간과 동일한 시간대의 미세먼지(PM10, PM2.5), 온/습도, CO2, VOCs(휘발성 유기화합물)를 포함하는 개인 환경 데이터 및 공공 환경 데이터와, 혈압/심박수 데이터를 함께 학습시키는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 시스템.The model learning unit learns the respiratory disease prognosis prediction model, fine dust (PM10, PM2.5), temperature / humidity, CO 2 in the same time zone as the cough sound, breathing sound, reading sound, and pronunciation sound measurement time A respiratory disease prognosis prediction system through time-series coughing, breathing, reading, and vocalization measurements that learn personal and public environment data, including VOCs (volatile organic compounds), and blood pressure/heart rate data together.
  9. 제1항에 있어서,According to claim 1,
    상기 모델 학습부는 상기 호흡기 질환 예후 예측모델을 학습시킴에 있어서, 측정된 기침음, 호흡음, 낭독음, 발성음을 Mel-spectrogram을 이용하여 음성의 특징을 추출하고, 그것에 개인 생체, 환경 데이터와 공공 환경 데이터를 붙여 CNN(Convolutional Neural Network) 알고리즘을 이용하여 학습시키는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 시스템.In learning the respiratory disease prognosis prediction model, the model learning unit extracts voice characteristics of measured cough sounds, breathing sounds, reading sounds, and vocalization sounds using a Mel-spectrogram, and personal biometric and environmental data and A respiratory disease prognosis prediction system through time-series coughing, breathing, reading, and vocalization measurements that are learned using CNN (Convolutional Neural Network) algorithms by attaching public environment data.
  10. 제9항에 있어서,According to claim 9,
    상기 개인 생체, 환경 데이터 및 공공 환경 데이터는 24시간∼48시간의 데이터를 모두 이용하는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 시스템.The personal biometric data, environmental data and public environmental data are respiratory disease prognosis prediction system through time-series cough sound, breathing sound, reading sound, and vocal sound measurement using all 24-48 hour data.
  11. 제9항에 있어서,According to claim 9,
    상기 기침음, 호흡음, 낭독음, 발성음 데이터와 상기 개인 생체, 환경 데이터와 공공 환경 데이터의 각각의 데이터에 대하여 x개의 채널로 구성하여 M_(x×m×n) 멀티채널 데이터를 만들고, CNN의 출력은 호흡기 질환 중증도 평가 점수의 회귀 모델로 구성하는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 시스템.Create M_(x×m×n) multi-channel data by configuring x number of channels for each of the cough sound, breath sound, reading sound, and vocalization sound data and the personal biometric data, environmental data, and public environment data; CNN's output is a respiratory disease prognosis prediction system through time-series cough sound, breathing sound, reading sound, and vocalization measurement consisting of a regression model of respiratory disease severity evaluation score.
  12. 음성 측정부, 데이터 수집부, 모델 학습부, 호흡기 질환 예후 예측부를 포함하는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 시스템에 기반한 호흡기 질환 예후 예측 방법으로서,A respiratory disease prognosis prediction method based on a respiratory disease prognosis prediction system through time-series cough sound, breathing sound, reading sound, and speech sound measurement including a voice measurement unit, a data collection unit, a model learning unit, and a respiratory disease prognosis prediction unit,
    a) 상기 음성 측정부에 의해 사용자로부터 기침음, 호흡음, 낭독음 및 발성음을 주기적으로 측정하는 단계와;a) periodically measuring cough sound, breathing sound, reading sound, and vocalization sound from the user by the voice measuring unit;
    b) 상기 데이터 수집부가 상기 측정된 데이터에 대하여 호흡기 질환 중증도점수를 평가한 학습용 데이터를 수집하는 단계와;b) collecting, by the data collection unit, learning data for evaluating respiratory disease severity scores with respect to the measured data;
    c) 상기 모델 학습부가 상기 수집된 학습용 데이터를 바탕으로 임의의 K 시간의 시계열 측정 데이터를 호흡기 질환 예후 예측모델의 입력값으로 하고, M 시간 후의 호흡기 질환 중증도 평가점수를 출력값으로 하는 호흡기 질환 예후 예측모델을 학습시키는 단계; 및c) Respiratory disease prognosis prediction in which the model learning unit takes time-series measurement data at any K time as an input value of a respiratory disease prognosis prediction model based on the collected learning data, and the respiratory disease severity evaluation score after M time as an output value training the model; and
    d) 상기 호흡기 질환 예후 예측부가 상기 학습된 호흡기 질환 예후 예측모델에 상기 K 시간의 시계열 측정 데이터를 입력하고, M 시간 후에 상기 호흡기 질환 예후 예측모델이 출력하는 호흡기 질환 예후 예측값을 바탕으로 호흡기 질환 예후를 예측하는 단계를 포함하는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 방법.d) The respiratory disease prognosis prediction unit inputs the time-series measurement data of the K time to the learned respiratory disease prognosis prediction model, and the respiratory disease prognosis is based on the respiratory disease prognosis prediction value output by the respiratory disease prognosis prediction model after M time Respiratory disease prognosis prediction method through time-series cough sound, breath sound, reading sound, and vocal sound measurement comprising the step of predicting.
  13. 제12항에 있어서,According to claim 12,
    상기 단계 a)에서 상기 기침음은 스마트폰의 녹음 기능을 이용하여 환자의 기침 소리를 다수회 반복적으로 녹음함으로써 획득하는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 방법.In step a), the cough sound is acquired by repeatedly recording the patient's cough sound multiple times using the recording function of the smartphone. Respiratory disease prognosis prediction method by measuring time-series cough sound, breathing sound, reading sound, and vocalization sound .
  14. 제12항에 있어서,According to claim 12,
    상기 호흡음은 스마트폰의 녹음 기능을 이용하여 환자의 들숨 및 날숨을 다수회 반복적으로 녹음함으로써 획득하는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 방법.The breathing sound is obtained by repeatedly recording the patient's inhalation and exhalation multiple times using the recording function of the smartphone. Respiratory disease prognosis prediction method through measurement of time-series cough sound, breathing sound, reading sound, and vocalization sound.
  15. 제12항에 있어서,According to claim 12,
    상기 낭독음은 스마트폰의 녹음 기능을 이용하여 제시된 문장을 평소 말투로 소리 내어 읽은 음을 녹음함으로써 획득하는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 방법.The reading sound is a respiratory disease prognosis prediction method through the measurement of time-series cough sound, breathing sound, reading sound, and pronunciation sound obtained by recording the sound read aloud in the usual tone of the presented sentence using the recording function of the smartphone.
  16. 제12항에 있어서,According to claim 12,
    상기 발성음은 스마트폰의 녹음 기능을 이용하여 음의 높낮이가 있는 음을 익숙한 멜로디에 따라 부르면서 나오는 발성을 통한 목소리를 녹음함으로써 획득하는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 방법.The vocalization sound is obtained by recording the voice through the vocalization produced while singing the pitched sound according to the familiar melody using the recording function of the smartphone. A method for predicting the prognosis of respiratory diseases.
  17. 제12항에 있어서,According to claim 12,
    상기 기침음, 호흡음, 낭독음, 발성음은 1시간∼N시간 주기로 스마트폰을 이용하여 측정하는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 방법.The cough sound, breathing sound, reading sound, and pronunciation sound are measured using a smartphone at a period of 1 hour to N hours. Method for predicting respiratory disease prognosis through measurement of time-series cough sound, breathing sound, reading sound, and pronunciation sound.
  18. 제12항에 있어서,According to claim 12,
    상기 단계 b)에서 상기 측정된 데이터에 대하여 호흡기 질환 중증도를 평가하고, 이를 정상(0), 주의(1), 경계(2) 클래스로 분류하는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 방법.Time-series cough sound, breathing sound, reading sound, and vocalization sound measurement that evaluates the severity of respiratory disease with respect to the measured data in step b) and classifies it into normal (0), caution (1), and alert (2) classes A method for predicting the prognosis of respiratory diseases through
  19. 제12항에 있어서,According to claim 12,
    상기 단계 c)에서 상기 호흡기 질환 예후 예측모델을 학습시킴에 있어서, 상기 기침음, 호흡음, 낭독음, 발성음 측정 시간과 동일한 시간대의 미세먼지(PM10, PM2.5), 온/습도, CO2, VOCs(휘발성 유기화합물)를 포함하는 개인 환경 데이터 및 공공 환경 데이터와, 혈압/심박수 데이터를 함께 학습시키는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 방법.In learning the respiratory disease prognosis prediction model in step c), fine dust (PM10, PM2.5), temperature / humidity, CO 2 , A method for predicting the prognosis of respiratory diseases through time-series coughing, breathing, reading, and vocalization measurements that learn personal and public environment data, including VOCs (volatile organic compounds), and blood pressure/heart rate data together.
  20. 제12항에 있어서,According to claim 12,
    상기 단계 c)에서 상기 호흡기 질환 예후 예측모델을 학습시킴에 있어서, 측정된 기침음, 호흡음, 낭독음, 발성음을 Mel-spectrogram을 이용하여 음성의 특징을 추출하고, 그것에 개인 생체, 환경 데이터와 공공 환경 데이터를 붙여 CNN (Convolutional Neural Network) 알고리즘을 이용하여 학습시키는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 방법.In learning the respiratory disease prognosis prediction model in step c), the measured cough sound, breathing sound, reading sound, and vocalization sound are extracted using the Mel-spectrogram, and personal biometric and environmental data are extracted therefrom. A method for predicting the prognosis of respiratory diseases through the measurement of cough sounds, breathing sounds, reading sounds, and vocalization sounds in a time series learned using CNN (Convolutional Neural Network) algorithm by attaching data and public environment data.
  21. 제20항에 있어서,According to claim 20,
    상기 개인 생체, 환경 데이터 및 공공 환경 데이터는 24시간∼48시간의 데이터를 모두 이용하는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 방법.The method of predicting the prognosis of respiratory diseases through the measurement of time-series cough sound, breathing sound, reading sound, and vocalization sound using all the personal biometric data, environmental data and public environmental data of 24 to 48 hours.
  22. 제20항에 있어서,According to claim 20,
    상기 기침음, 호흡음, 낭독음, 발성음 데이터와 상기 개인 생체, 환경 데이터와 공공 환경 데이터의 각각의 데이터에 대하여 x개의 채널로 구성하여 M_(x×m×n) 멀티채널 데이터를 만들고, CNN의 출력은 호흡기 질환 중증도 평가점수의 회귀 모델로 구성하는 시계열 기침음, 호흡음, 낭독음, 발성음 측정을 통한 호흡기 질환 예후 예측 방법.Create M_(x×m×n) multi-channel data by configuring x number of channels for each of the cough sound, breath sound, reading sound, and vocalization sound data and the personal biometric data, environmental data, and public environment data; A method for predicting respiratory disease prognosis through measurement of cough sound, breathing sound, reading sound, and vocalization sound in a time series consisting of regression models of respiratory disease severity evaluation scores.
PCT/KR2022/014058 2021-10-06 2022-09-20 System and method for predicting respiratory disease prognosis through time-series measurements of cough sounds, respiratory sounds, recitation sounds and vocal sounds WO2023058946A1 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
KR10-2021-0132110 2021-10-06
KR20210132110 2021-10-06
KR10-2022-0037496 2022-03-25
KR1020220037496A KR102624637B1 (en) 2021-10-06 2022-03-25 Respiratory disease prognosis prediction system and method through time-series cough sound, breathing sound, reading sound or vocal sound measurement

Publications (1)

Publication Number Publication Date
WO2023058946A1 true WO2023058946A1 (en) 2023-04-13

Family

ID=85804476

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2022/014058 WO2023058946A1 (en) 2021-10-06 2022-09-20 System and method for predicting respiratory disease prognosis through time-series measurements of cough sounds, respiratory sounds, recitation sounds and vocal sounds

Country Status (1)

Country Link
WO (1) WO2023058946A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117059283A (en) * 2023-08-15 2023-11-14 宁波市鄞州区疾病预防控制中心 Speech database classification and processing system based on pulmonary tuberculosis early warning

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20180099434A (en) * 2017-02-28 2018-09-05 삼성전자주식회사 Method and apparatus for monitoring patient
KR20190115330A (en) * 2018-04-02 2019-10-11 주식회사 씨씨앤아이리서치 An application for predicting an acute exacerbation of chronic respiratory disease
KR20210032919A (en) * 2019-09-17 2021-03-25 다인기술 주식회사 Method, system and non-transitory computer-readable recording medium for analyzing breath-related sounds

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20180099434A (en) * 2017-02-28 2018-09-05 삼성전자주식회사 Method and apparatus for monitoring patient
KR20190115330A (en) * 2018-04-02 2019-10-11 주식회사 씨씨앤아이리서치 An application for predicting an acute exacerbation of chronic respiratory disease
KR20210032919A (en) * 2019-09-17 2021-03-25 다인기술 주식회사 Method, system and non-transitory computer-readable recording medium for analyzing breath-related sounds

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
KRANTHI KUMAR LELLA; ALPHONSE PJA: "Automatic COVID-19 disease diagnosis using 1D convolutional neural network and augmentation with human respiratory sound based on parameters: cough, breath, and voice", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 14 December 2021 (2021-12-14), 201 Olin Library Cornell University Ithaca, NY 14853, XP091119757, DOI: 10.3934/publichealth.2021019 *
XIA TONG: "Listening to Your Health: Automatic and Reliable Respiratory Disease Detection through Sounds", CPGS FIRST YEAR REPORT, UNIVERSITY OF CAMBRIDGE, 1 June 2021 (2021-06-01), University of Cambridge, XP093055673, Retrieved from the Internet <URL:https://xtxiatong.github.io/files/FirstYearReport.pdf> [retrieved on 20230620] *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117059283A (en) * 2023-08-15 2023-11-14 宁波市鄞州区疾病预防控制中心 Speech database classification and processing system based on pulmonary tuberculosis early warning

Similar Documents

Publication Publication Date Title
CN106725532B (en) Depression automatic evaluation system and method based on phonetic feature and machine learning
CN112166475A (en) Respiratory system based sound management of respiratory conditions
WO2022019402A1 (en) Computer program and method for training artificial neural network model on basis of time series bio-signal
WO2021015381A1 (en) Pulmonary function estimation
WO2022014943A1 (en) Electrocardiogram generation device based on generative adversarial network algorithm, and method thereof
WO2021054742A1 (en) Method, system, and non-transitory computer-readable recording media for analyzing breathing-related sound
WO2019031794A1 (en) Method for generating prediction result for predicting occurrence of fatal symptoms of subject in advance and device using same
WO2022014942A1 (en) Heart disease diagnosis device using deep-learning-based electrocardiogram, and method therefor
WO2013077558A1 (en) Robot-based autism diagnosis device using electroencephalogram and method therefor
WO2019225910A1 (en) Method and system for predicting and analyzing stroke severity using nihss
CN108305680A (en) Intelligent parkinsonism aided diagnosis method based on multi-element biologic feature and device
Aly et al. A novel deep learning model to detect COVID-19 based on wavelet features extracted from Mel-scale spectrogram of patients’ cough and breathing sounds
Madruga et al. Multicondition training for noise-robust detection of benign vocal fold lesions from recorded speech
JP2019051129A (en) Deglutition function analysis system and program
Nasab et al. Coronavirus diagnosis using cough sounds: Artificial intelligence approaches
WO2023058946A1 (en) System and method for predicting respiratory disease prognosis through time-series measurements of cough sounds, respiratory sounds, recitation sounds and vocal sounds
CN108766462A (en) A kind of phonic signal character learning method based on Meier frequency spectrum first derivative
WO2023101089A1 (en) Method and device for classifying electrocardiogram waveform by using machine learning
KR20230050208A (en) Respiratory disease prognosis prediction system and method through time-series cough sound, breathing sound, reading sound or vocal sound measurement
Askari Nasab et al. Coronavirus diagnosis using cough sounds: Artificial intelligence approaches
KR102624676B1 (en) Respiratory disease prediction method using cough sound, breath sound, and voice sound measurement data collected by smartphone
Lai et al. ResNet for recognition of Qi-deficiency constitution and balanced constitution based on voice
WO2023211128A1 (en) Ai-based disease diagnosis method and device using voice data
WO2024058585A1 (en) Method and analysis device for classifying severity of lung disease of subject by using voice data and clinical information
Hee et al. Deep Neural Network Based Respiratory Pathology Classification Using Cough Sounds

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22878767

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE