WO2022073285A1 - 基于知识蒸馏的肺音分类方法、系统、终端及存储介质 - Google Patents

基于知识蒸馏的肺音分类方法、系统、终端及存储介质 Download PDF

Info

Publication number
WO2022073285A1
WO2022073285A1 PCT/CN2020/130136 CN2020130136W WO2022073285A1 WO 2022073285 A1 WO2022073285 A1 WO 2022073285A1 CN 2020130136 W CN2020130136 W CN 2020130136W WO 2022073285 A1 WO2022073285 A1 WO 2022073285A1
Authority
WO
WIPO (PCT)
Prior art keywords
lung sound
trained
audio
lung
model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/130136
Other languages
English (en)
French (fr)
Inventor
李坚强
陈杰
明仲
梁中明
王成
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen University
Original Assignee
Shenzhen University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen University filed Critical Shenzhen University
Publication of WO2022073285A1 publication Critical patent/WO2022073285A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
    • G10L25/24—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters the extracted parameters being the cepstrum
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/27—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique
    • G10L25/30—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique using neural networks
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G10L25/66—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for extracting parameters related to health condition

Definitions

  • the present invention relates to the technical field of audio classification, and in particular, to a method, system, terminal and storage medium for classifying lung sounds based on knowledge distillation.
  • the technical problem to be solved by the present invention is to provide a lung sound classification method, system, terminal and storage medium based on knowledge distillation in view of the above-mentioned defects of the prior art, aiming to solve the problem of using a neural network to classify lung sounds in the prior art.
  • classifying basically only a single type of classification can be achieved, which is used as a sub-classification of lung sounds, but there are still some limitations, and the analysis of lung sounds is not accurate enough, which affects the accuracy of classification.
  • the present invention provides a method for classifying lung sounds based on knowledge distillation, wherein the method includes:
  • the neural network model includes a plurality of teacher models and student models, and train a plurality of teacher models based on the lung sound data set to obtain a plurality of trained teacher models, and the trained plurality of teacher models are obtained.
  • the teacher model is a two-class model, used for classifying the lung sound audio, and each trained teacher model corresponds to the classification of two lung sound categories;
  • the student model is trained based on the multi-class soft label and lung sound dataset to obtain a trained student model, and the trained student model is used to classify the lung sound audio, and the trained student model is used to classify the lung sound audio.
  • the trained student model corresponds to the classification of various lung sound classes.
  • the lung sound audio is obtained, and based on the lung sound audio, the Mel spectrum feature of the lung sound audio is extracted to obtain a lung sound data set, including:
  • the mel spectral features are extracted from each of the audio segments respectively to obtain the lung sound data set.
  • the audio data enhancement processing includes: time transformation processing, pitch transformation processing, and ⁇ -law compression processing.
  • the method includes:
  • a zero-mean normalization process is performed on each of the audio segments.
  • the plurality of teacher models are trained based on the lung sound data set to obtain a plurality of trained teacher models, including:
  • a plurality of teacher models are trained to obtain the plurality of trained teacher models, and the real label is a lung sound category.
  • the student model is trained based on the multi-class soft label and the lung sound data set to obtain a trained student model, and the lung sound audio is performed by using the trained student model.
  • Categories including:
  • the student model is trained to obtain the trained student model, and the soft labels of the trained student model are close to the trained multiple teacher models soft label;
  • the lung sound audio is classified using the trained student model to obtain the classification of the lung sound audio.
  • both the teacher model and the student model are a 6-layer convolutional neural network model.
  • the present invention also provides a lung sound analysis system based on knowledge distillation, wherein the system includes:
  • an audio feature acquisition unit configured to acquire lung sound audio, and extract the Mel spectrum feature of the lung sound audio based on the lung sound audio to obtain a lung sound data set;
  • the teacher model training unit is used to obtain a neural network model, the neural network model includes a plurality of teacher models and student models, and the plurality of teacher models are trained based on the lung sound data set to obtain a plurality of trained teacher models,
  • the trained multiple teacher models are two-classification models for classifying the lung sound audio, and each trained teacher model corresponds to the classification of two lung sound categories;
  • a soft label fitting unit configured to calculate the soft labels of the lung sound data set according to the trained teacher model, and convert them to obtain multi-category soft labels
  • a student model training unit for training the student model based on the multi-classified soft label and the lung sound data set, obtaining a trained student model, and classifying the lung sound audio using the trained student model,
  • the trained student model is a multi-classification model, and the trained student model corresponds to the classification of multiple lung sound categories.
  • the present invention also provides an intelligent terminal, including a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors
  • the one or more programs include methods for performing any of the above aspects.
  • the present invention also provides a non-transitory computer-readable storage medium, wherein, when the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is enabled to execute any one of the above solutions. method described.
  • the present invention provides a method for classifying lung sounds based on knowledge distillation, first acquiring the audio of the lung sound, and extracting the Mel spectrum feature of the audio of the lung sound based on the audio of the lung sound, Obtain a lung sound data set; train a plurality of teacher models based on the lung sound data set to obtain a plurality of trained teacher models; respectively calculate the softness of the lung sound data set according to the trained plurality of teacher models. label and convert it to obtain a multi-class soft label; train the student model based on the multi-class soft label and the lung sound data set to obtain a trained student model, and use the trained student model to The lung sound audio is classified.
  • the present invention can guide student model learning by using the knowledge of multiple teacher models, so as to make full use of multi-party lung sound audio and accurately classify the lung sound audio.
  • FIG. 1 is a flowchart of an embodiment of a lung sound classification method based on knowledge distillation provided by an embodiment of the present invention.
  • FIG. 2 is a schematic block diagram of a lung sound classification system based on knowledge distillation provided by an embodiment of the present invention.
  • FIG. 3 is a functional schematic diagram of an intelligent terminal provided by an embodiment of the present invention.
  • this embodiment provides a lung sound classification method based on knowledge distillation.
  • the knowledge distillation in this embodiment can transfer the knowledge of one network to another network, and the two networks can be Homogeneous or heterogeneous.
  • the specific implementation is to train a teacher network first, and then use the output of the teacher network and the true labels of the data to train the student network.
  • Knowledge distillation can be used to transform a network from a large network into a small network and retain performance close to that of a large network; it can also transfer knowledge learned from multiple networks into a single network.
  • the knowledge of multiple teacher models can be used to guide the learning of the student models, so as to make full use of the multi-party lung sound audio and accurately classify the lung sound audio.
  • the method for classifying lung sounds based on knowledge distillation specifically includes the following steps:
  • Step S100 Acquire lung sound audio, and extract the Mel spectrum feature of the lung sound audio based on the lung sound audio to obtain a lung sound data set.
  • the audio of lung sounds in this embodiment may come from audio data collected by major hospitals, so that multi-party data can be used for analysis, so as to accurately classify the audio of lung sounds.
  • audio data enhancement processing is performed on the lung sound audio to obtain an enhanced lung sound audio.
  • this embodiment uses three data enhancement methods for data enhancement methods: time transformation, pitch transformation, and ⁇ -law compression.
  • Time transforms are designed to affect the sampling rate of audio by changing the parametric factor.
  • the pitch shift is to adjust the pitch of the audio in semitones.
  • ⁇ -law compression is a dynamic range compression algorithm. For the input signal s, -1 ⁇ s ⁇ 1, the calculation of ⁇ -law compression can be expressed as the following formula:
  • sgn(s) represents the sign function
  • abs(s) represents the absolute value of the signal s
  • ⁇ is the compression parameter
  • the value is the power of 2 minus 1.
  • the present embodiment obtains the mel spectrum feature from the enhanced lung sound audio, and the function of extracting the mel spectrum feature is to extract the distinguishable components in the audio signal, and then remove the interference factors of the lung sound audio, to improve the recognition accuracy.
  • the process of extracting the Mel spectrum feature in this embodiment mainly includes four steps. Specifically: First, pre-emphasis, framing and windowing are performed on the lung sound audio (some preprocessing to enhance speech signal performance (signal-to-noise ratio, processing accuracy, etc.)).
  • a corresponding spectrum is obtained through FFT (Fast Fourier Transformation, Fast Fourier Transform) (that is, spectrums distributed in different time windows on the time axis are obtained).
  • FFT Fast Fourier Transformation, Fast Fourier Transform
  • the above spectrum is passed through a Mel (Mel) filter bank to obtain a Mel spectrum (through the Mel spectrum, the linear natural spectrum is converted into a Mel spectrum reflecting the characteristics of human hearing).
  • Mel Mel
  • the extraction of the mel spectral feature in this embodiment can be implemented based on an open source python toolkit librosa.
  • this embodiment may slice the enhanced lung sound audio to obtain multiple audio segments. Then, the Mel spectral features are extracted from each of the audio segments to obtain the lung sound data set. Specifically, since the originally collected lung sound audio is a long time sequence, there will be obvious changes between 3 and 10 seconds. Therefore, in this embodiment, the enhancement processing can be performed according to a preset time window. The resulting lung sound audio is segmented into fixed-length audio segments, and then each audio segment is characterized by Mel spectrum to form the data set, and the model is trained through the lung sound data set. In an implementation manner, this embodiment also performs zero-mean normalization processing on each of the audio segments after the lung sound data set is obtained, which can speed up the optimization process of the model.
  • the normalization method is the more commonly used normalization method in audio and image processing tasks. For an audio segment s, zero-mean normalization is simply subtracting its mean After that, divide by the standard deviation ⁇ of s, as shown in the following formula:
  • Step S200 Obtain a neural network model, the neural network model includes a plurality of teacher models and student models, train a plurality of teacher models based on the lung sound data set, and obtain a plurality of trained teacher models, the trained teacher models are obtained.
  • the plurality of teacher models are binary classification models for classifying the lung sound audio, and each trained teacher model corresponds to the classification of two lung sound categories.
  • this embodiment first acquires a neural network model, which includes a plurality of teacher models and student models, and then acquires the predicted labels in the lung sound data set. Then, based on the predicted label and the real label, the teacher model is trained to obtain a plurality of trained teacher models, and the real label is a lung sound category.
  • each trained teacher model is a two-class model. Assuming that M types of lung sounds are shared, M teacher models need to be trained correspondingly, so that each trained teacher model can correspond to two types of lung sounds. Classification of lung sound categories.
  • the lung sound data set in this example is The predicted labels corresponding to the data are defined as
  • the teacher model is a 6-layer CNN (Convolutional Neural Networks, convolutional neural network) model
  • the first layer is a convolutional layer with 24 filters
  • the size of the convolution kernel is (5,5 ) with a stride of (4,2).
  • the second layer is also a convolutional layer, containing 48 filters, and the size and stride of the convolution kernel are the same as those of the first layer.
  • the third layer is a max pooling layer with a pool size of (2,2).
  • the number of filters and kernel size of the 4th convolutional layer are the same as those of the 2nd layer.
  • the fifth fully connected layer has 64 hidden units, and the last layer is the output layer.
  • each teacher model corresponds to a lung sound category, and fits the lung sound audio data collected from different hospitals.
  • the training goal is to minimize the difference between the predicted label and the true label of the teacher model.
  • the cross-entropy loss of , for the teacher model T i its loss function is as follows:
  • Step S300 Calculate the soft labels of the lung sound data set according to the multiple trained teacher models, and convert them to obtain multi-category soft labels.
  • the teacher model calculates the soft labels of the data set X S to be learned by the student model, and then lets the student model fit these soft labels.
  • the knowledge of the teacher model can be taught to the student model, that is, knowledge distillation can be realized.
  • For one data point in the data set X S When calculating its soft label by the teacher model Ti , it is shown in the following formula:
  • each teacher model T i is a binary classification model
  • the output of its logits layer has only 2 categories, and for the other M-1 categories, this embodiment adopts the method of taking a value of 0, so as to combine multiple Convert the soft label of a binary classification model to a multi-class soft label So as to be unified with the category dimension of the student model, With the corresponding category label. is calculated as follows:
  • the function is any element in the set ⁇ 1,...,M ⁇ , and the probability of each element being selected is equal; otherwise, the function return directly In this way, the trained student model can classify the lung sound audio, so as to obtain the category corresponding to the lung sound audio.
  • Step S400 Train a student model based on the multi-classified soft label and the lung sound data set to obtain a trained student model, and use the trained student model to classify the lung sound audio.
  • the model is a multi-class model, and the trained student model corresponds to the classification of various lung sound classes.
  • the multi-category soft label is obtained; based on the multi-category soft label and the lung sound data set, the student model is trained to obtain the trained student model.
  • the soft label is close to the soft label of the trained teacher model; the lung sound audio is classified by using the trained student model to obtain the classification of the lung sound audio.
  • the label corresponding to the data set X S to be learned by the student model is defined as Y S .
  • the student model is a simplified model based on the teacher model.
  • the student model has a total of 6 layers, the first layer is a convolutional layer with 6 filters, the convolution kernel size is (5, 5), and the stride is (4, 2).
  • the second layer is also a convolutional layer, containing 18 filters, and the size and stride of the convolution kernel are the same as those of the first layer.
  • the third layer is a max pooling layer with a pool size of (2,2).
  • the fourth convolutional layer has 24 filters, and the kernel size is the same as the second layer.
  • the 5th fully connected layer has 32 hidden units and then connects the output layer.
  • the soft label calculation of the student model is similar to that of the teacher model, except that the student model needs to consider M+1 categories, as shown in the following formula:
  • g j (.) represents the output of the logits layer when the student model corresponds to the category j ⁇ 0,...,M ⁇ , and 0 represents the normal category.
  • the learning objectives of the student model in this embodiment are also different from the teacher model.
  • the student model needs to make predictions as close to the real label as possible
  • It has richer information, which includes the class-to-class associations in the samples, which are learned by the complex teacher model. Due to its simple structure, the student model has a weaker learning ability than the teacher model. However, if the student model fits the associated information learned by the teacher model during the training process, it can play an auxiliary role in its own learning.
  • X S represents the number of elements in X S
  • f s represents the prediction function of S.
  • ⁇ [0,1] is a constant used by the student model to choose whether to focus on real labels or soft labels.
  • the accuracy of the model in the prior art is 98%
  • the parameter quantity is 0.28M
  • the time consumption for predicting 8000 pieces of data is 3.12 seconds
  • the accuracy of the student model in this embodiment is 95%
  • the parameter The amount is 0.06M, and it takes 2.48 seconds to predict the same data.
  • the accuracy of the model of this solution is 0.03 lower than that of the prior art model
  • the number of parameters is reduced by 79%
  • the time to complete the same task is reduced by 20%. It can be seen that the present invention can guide student model learning through knowledge distillation of multiple teacher models, so as to make full use of multi-party lung sound audio and accurately classify the lung sound audio.
  • the present invention also provides a lung sound analysis system based on knowledge distillation.
  • the system includes: an audio feature acquisition unit 10 , a teacher model training unit 20 , and a soft label fitting unit 30 And the student model training unit 40 .
  • the audio feature acquisition unit 10 is configured to acquire lung sound audio, and extract the mel spectrum feature of the lung sound audio based on the lung sound audio to obtain a lung sound data set.
  • the teacher model training unit 20 is used to obtain a neural network model, the neural network model includes a plurality of teacher models and student models.
  • a teacher model wherein the trained teacher models are binary classification models for classifying the lung sound audio, and each trained teacher model corresponds to the classification of two types of lung sounds.
  • the soft label fitting unit 30 is configured to separately calculate the soft labels of the lung sound data set according to the multiple trained teacher models, and convert them to obtain multi-category soft labels.
  • the student model training unit 40 is used to train a student model based on the multi-classified soft label and the lung sound data set, obtain a trained student model, and use the trained student model to classify the lung sound audio , the trained student model is a multi-classification model, and the trained student model corresponds to the classification of multiple lung sound categories.
  • the present invention also provides an intelligent terminal, the principle block diagram of which may be shown in FIG. 3 .
  • the intelligent terminal includes a processor, a memory, a network interface, a display screen, and a temperature sensor connected through a system bus.
  • the processor of the intelligent terminal is used to provide computing and control capabilities.
  • the memory of the intelligent terminal includes a non-volatile storage medium and an internal memory.
  • the nonvolatile storage medium stores an operating system and a computer program.
  • the internal memory provides an environment for the execution of the operating system and computer programs in the non-volatile storage medium.
  • the network interface of the intelligent terminal is used for communicating with an external intelligent terminal through a network connection.
  • the computer program when executed by a processor, implements a knowledge distillation-based lung sound analysis method.
  • the display screen of the smart terminal may be a liquid crystal display screen or an electronic ink display screen, and the temperature sensor of the smart terminal is pre-set inside the smart terminal to detect the operating temperature of the internal equipment.
  • FIG. 3 is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the intelligent terminal to which the solution of the present invention is applied. More or fewer components than shown in the figures may be included, or some components may be combined, or have a different arrangement of components.
  • an intelligent terminal includes a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors
  • One or more programs contain instructions to:
  • the neural network model includes a plurality of teacher models and student models, and train a plurality of teacher models based on the lung sound data set to obtain a plurality of trained teacher models, and the trained plurality of teacher models are obtained.
  • the teacher model is a two-class model, used for classifying the lung sound audio, and each trained teacher model corresponds to the classification of two lung sound categories;
  • the student model is trained based on the multi-class soft label and lung sound dataset to obtain a trained student model, and the trained student model is used to classify the lung sound audio, and the trained student model is used to classify the lung sound audio.
  • the trained student model corresponds to the classification of various lung sound classes.
  • Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory may include random access memory (RAM) or external cache memory.
  • RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Road (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
  • the present invention discloses a lung sound classification method, system, terminal and storage medium based on knowledge distillation, the method includes: acquiring lung sound audio, and extracting the melogram of the lung sound audio based on the lung sound audio based on the lung sound data set, train a plurality of teacher models to obtain a plurality of trained teacher models; respectively calculate the lung sound according to the trained plurality of teacher models
  • the soft label of the data set, and after converting it, a multi-class soft label is obtained; the student model is trained based on the multi-class soft label and the lung sound data set, and a plurality of trained student models are obtained.
  • the trained student model classifies the lung sound audio.
  • the present invention can guide student model learning by using the knowledge of multiple teacher models, so as to make full use of multi-party lung sound audio and accurately classify the lung sound audio.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Signal Processing (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Computational Linguistics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Epidemiology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

一种基于知识蒸馏的肺音分类方法、系统、终端及存储介质，方法包括：获取肺音音频，并基于肺音音频提取肺音音频的梅尔频谱特征，得到肺音数据集（S100）；基于肺音数据集对多个教师模型进行训练，得到已训练的多个教师模型（S200）；根据已训练的多个教师模型分别计算肺音数据集的软标签，并经转换后，得到多分类软标签（S300）；基于多分类软标签和肺音数据集训练学生模型，得到已训练的学生模型，并利用已训练的学生模型对肺音音频进行分类（S400）。该方法可以基于知识蒸馏，通过利用多个教师模型的知识，指导学生模型学习，从而充分利用多方肺音音频，并准确对肺音音频进行分类。

Description

基于知识蒸馏的肺音分类方法、系统、终端及存储介质 技术领域
本发明涉及音频分类技术领域，尤其涉及一种基于知识蒸馏的肺音分类方法、系统、终端及存储介质。
背景技术
传统对于肺音的分析是利用传统的听诊器采用人工听诊的方式来实现，这可能由于经验不足和人耳对肺音的低频段敏感性低等因素，导致判断结果不够准确。与主观听诊相比，对肺音的计算机分析能够进行可重复的肺音量化近年来，已有很多学者专注于研究肺音分析和检测中的机器学习技术。机器学习技术应用于肺音分析上，使得肺音识别的准确率得到了提升。
由于肺部声音信号不稳定，现有技术中使用神经网络来对肺音进行分类时，基本都只能实现单一类型的分类，用作肺音的细类区分，仍存在一定的局限性。并且，由于听取肺音是可能出现各种各样不确定音和噪音，使得对肺音的分析不够准确，影响分类的准确性。
因此，现有技术还有待改进和提高。
发明内容
本发明要解决的技术问题在于，针对现有技术的上述缺陷，提供一种基于知识蒸馏的肺音分类方法、系统、终端及存储介质，旨在解决现有技术中用神经网络来对肺音进行分类时，基本都只能实现单一类型的分类，用作肺音的细类区分，仍存在一定的局限性，并且对肺音的分析不够准确，影响分类的准确性。
为了解决上述技术问题，本发明所采用的技术方案如下：
第一方面，本发明提供一种基于知识蒸馏的肺音分类方法，其中，所述方法包括：
获取肺音音频，并基于所述肺音音频提取所述肺音音频的梅尔频谱特征，得到肺音数据集；
获取神经网络模型，所述神经网络模型包括多个教师模型与学生模型，基于所述肺音数据集对多个教师模型进行训练，得到已训练的多个教师模型，所述已训练的多个教师模型为二分类模型，用于对所述肺音音频进行分类，且每一个已训练的教师模型对应两种肺音类别的分类；
根据所述已训练的多个教师模型分别计算所述肺音数据集的软标签，并将其转换后，得到多分类软标签；
基于所述多分类软标签和肺音数据集训练所述学生模型，得到已训练的学生模型，并利用所述已训练的学生模型对所述肺音音频进行分类，所述已训练的学生模型为多分类模型，所述已训练的学生模型对应于多种肺音类别的分类。
在一种实现方式中，所述获取肺音音频，并基于所述肺音音频提取所述肺音音频的梅尔频谱特征，得到肺音数据集，包括：
采集所述肺音音频，并对所述肺音音频进行音频数据增强处理，得到增强后的肺音音频；
对所述增强后的肺音音频进行切片，得到多个音频片段；
分别从每一个所述音频片段中提取所述梅尔频谱特征，得到所述肺音数据集。
在一种实现方式中，所述音频数据增强处理包括：时间变换处理、音高变换处理和μ律压缩处理。
在一种实现方式中，所述获取肺音音频，并基于所述肺音音频提取所述肺音音频的梅尔频谱特征，得到肺音数据集之后，包括：
对每一个所述音频片段进行零均值归一化处理。
在一种实现方式中，所述基于所述肺音数据集对多个教师模型进行训练，得到已训练的多个教师模型，包括：
获取所述肺音数据集中的预测标签；
基于所述预测标签与真实标签，对多个教师模型进行训练，得到所述已训练多个的教师模型，所述真实标签为肺音类别。
在一种实现方式中，所述基于所述多分类软标签和肺音数据集训练所述学生模型，得到已训练的学生模型，并利用所述已训练的学生模型对所述肺音音频进行分类，包括：
获取所述多分类软标签；
基于所述多分类软标签与肺音数据集，对所述学生模型进行训练，得到所述已训练的学生模型，所述已训练的学生模型的软标签接近所述已训练的多个教师模型的软标签；
利用所述已训练的学生模型对所述肺音音频进行分类，得到所述肺音音频的类别。
在一种实现方式中，所述教师模型与所述学生模型均为一个6层的卷积神经网络模型。
第二方面，本发明还提供一种基于知识蒸馏的肺音分析系统，其中，所述系统包括：
音频特征获取单元，用于获取肺音音频，并基于所述肺音音频提取所述肺音音频的梅尔频谱特征，得到肺音数据集；
教师模型训练单元，用于获取神经网络模型，所述神经网络模型包括多个教师模型与学生模型，基于所述肺音数据集对多个教师模型进行训练，得到已训练的多个教师模型，所述已训练的多个教师模型为二分类模型，用于对所述肺音音频进行分类，且每一个已训练的教师模型对应两种肺音类别的分类；
软标签拟合单元，用于根据所述已训练的教师模型分别计算所述肺音数据集的软标签，并将其转换后，得到多分类软标签；
学生模型训练单元，用于基于所述多分类软标签和肺音数据集训练所述学生模型，得到已训练的学生模型，并利用所述已训练的学生模型对所述肺音音频进行分类，所述已训练的学生模型为多分类模型，所述已训练的学生模型对应于多种肺音类别的分类。
第三方面，本发明还提供一种智能终端，其中，包括有存储器，以及一个或者一个以上的程序，其中一个或者一个以上程序存储于存储器中，且经配置以由一个或者一个以上处理器执行所述一个或者一个以上程序包含用于执行上述方案中任意一项所述的方法。
第四方面，本发明还提供一种非临时性计算机可读存储介质，其中，当所述存储介质中的指令由电子设备的处理器执行时，使得电子设备能够执行上述方案中任意一项所述的方法。
有益效果：与现有技术相比，本发明提供了一种基于知识蒸馏的肺音分类方法，首先获取肺音音频，并基于所述肺音音频提取所述肺音音频的梅尔频谱特征，得到肺音数据集；基于所述肺音数据集对多个教师模型进行训练，得到已训练的多个教师模型；根据所述已训练的多个教师模型分别计算所述肺音数据集的软标签，并将其转换后，得到多分类软标签；基于所述多分类软标签和肺音数据集训练所述学生模型，得到已训练的学生模型，并利用所述已训练的学生模型对所述肺音音频进行分类。本发明可以基于知识蒸馏，通过利用多个教师模型的知识，指导学生模型学习，从而充分利用多方肺音音频，并准确对所述肺音音频进行分类。
附图说明
图1为本发明实施例提供的基于知识蒸馏的肺音分类方法的实施例流程图。
图2是本发明实施例提供的基于知识蒸馏的肺音分类系统的原理框图。
图3是本发明实施例提供的智能终端的功能原理图。
具体实施方式
为使本发明的目的、技术方案及效果更加清楚、明确，以下参照附图并举实施例对本发明进一步详细说明。应当理解，此处所描述的具体实施例仅用以解释本发明，并不用于限定 本发明。
传统对于肺音的分析是利用传统的听诊器采用人工听诊的方式来实现，这可能由于经验不足和人耳对肺音的低频段敏感性低等因素，导致判断结果不够准确。与主观听诊相比，对肺音的计算机分析能够进行可重复的肺音量化近年来，已有很多学者专注于研究肺音分析和检测中的机器学习技术。机器学习技术应用于肺音分析上，使得肺音识别的准确率得到了提升。
由于肺部声音信号不稳定，现有技术中也有提出一种使用小波变换分析肺部声音信号以及使用人工神经网络(ANN)进行分类的方法,但是这种方法需要人工标记肺音的周期。异常肺音本身是一种非周期信号，其可能不会在整个记录中出现。此外，目前还开发出一种自动肺音分析仪，该工具在不需要额外的流量信号或标记呼吸周期的情况下，就能识别肺部声音特征,但是这种方法主要是区分正常肺音和异常肺音，用作肺音的细类区分，仍存在一定的局限性。听诊时可能出现的各种各样的不确定音和噪声使肺音检测不够准确。目前，研究人员提出了基于隐马尔科夫模型提出“异常呼吸置信度”检测的两步分类方法，但是该工作也只是关注单一类型的肺音。可见，现有技术中使用神经网络来对肺音进行分类时，基本都只能实现单一类型的分类，用作肺音的细类区分，仍存在一定的局限性。并且，由于听取肺音是可能出现各种各样不确定音和噪音，使得对肺音的分析不够准确，影响分类的准确性。
为解决现有技术中的问题，本实施例提供一种基于知识蒸馏的肺音分类方法，本实施例中的知识蒸馏，是可以将一个网络的知识转移到另一个网络，两个网络可以是同构或者异构。具体实施是，是先训练一个教师网络，然后使用这个教师网络的输出和数据的真实标签去训练学生网络。知识蒸馏可以用来将网络从大网络转化成一个小网络，并保留接近于大网络的性能；也可以将多个网络的学到的知识转移到一个网络中。因此，本实施例可基于知识蒸馏的原理，利用多个教师模型的知识，指导学生模型学习，从而充分利用多方肺音音频，并准确对所述肺音音频进行分类。具体如图1中所示。所述基于知识蒸馏的肺音分类方法具体包括如下步骤：
步骤S100、获取肺音音频，并基于所述肺音音频提取所述肺音音频的梅尔频谱特征，得到肺音数据集。
本实施例中的肺音音频可来自各大医院所采集到的音频数据，这样就可以利用多方数据来进行分析，以便准确地对肺音音频进行分类。在本实施例中，当获取到所述肺音音频后，对所述肺音音频进行音频数据增强处理，得到增强后的肺音音频。具体地，本实施例使用三种数据增强方式来数据增强方法：时间变换、音高变换和μ律压缩。时间变换旨在通过改变参 数因子而影响音频的采样速率。而音高变换就是以半音为单位，对音频的音调进行调整。μ律压缩是一种动态范围压缩算法，对于输入的信号s，-1≤s≤1，计算μ律压缩可以表述为以下公式：
Figure PCTCN2020130136-appb-000001
其中，sgn(s)表示符号函数，abs(s)表示取信号s的绝对值，μ为压缩参数，一般取值为2的幂减1。通过这三种数据增强方法来对数据进行增强，使得采集的肺音音频可以明显的增强，使得整个数据集中的样本数据量增加。
接着，本实施例从所述增强后的肺音音频中获取梅尔频谱特征，提取梅尔频谱特征的作用在于把音频信号中具有辨识性的成分提取出来，然后去除肺音音频的干扰因素，以提高识别的准确性。具体实施时，本实施例中提取梅尔频谱特征的流程主要是包括四个步骤。具体为：首先是先对肺音音频进行预加重、分帧和加窗(加强语音信号性能(信噪比，处理精度等)的一些预处理)。然后，对每一个短时分析窗，通过FFT(Fast Fourier Transformation，快速傅氏变换)得到对应的频谱(即获得分布在时间轴上不同时间窗内的频谱)。接着将上面的频谱通过Mel(梅尔)滤波器组得到Mel频谱(通过Mel频谱，将线形的自然频谱转换为体现人类听觉特性的Mel频谱)。当然，本实施例中对于梅尔频谱特征的提取可以基于一个开源的python工具包librosa来实现。
在一种实现方式中，本实施例可以对所述增强后的肺音音频进行切片，得到多个音频片段。然后分别从每一个所述音频片段中提取所述梅尔频谱特征，得到所述肺音数据集。具体地，由于原始采集的肺音音频是一段较长的时序序列，在3至10秒之间会有较为明显的变化，因此本实施例中可以按照预先设定好的时间窗口，将增强处理后的肺音音频分割为固定时长的音频片段，然后再对各个音频片段进行梅尔频谱特征，并构成所述数据集，通过所述肺音数据集进行模型的训练。在一种实现方式中，本实施例在得到所述肺音数据集后还对每一个所述音频片段进行零均值归一化处理，能够加快模型的优化过程，本实施例使用的零均值归一化的方法是音频和图像处理任务中比较常用的归一化方法。对于音频片段s,零均值归一化就是减去其均值
Figure PCTCN2020130136-appb-000002
之后，再除以s的标准差σ，如以下公式所示：
Figure PCTCN2020130136-appb-000003
步骤S200、获取神经网络模型，所述神经网络模型包括多个教师模型与学生模型，基于所述肺音数据集对多个教师模型进行训练，得到已训练的多个教师模型，所述已训练的多个 教师模型为二分类模型，用于对所述肺音音频进行分类，且每一个已训练的教师模型对应两种肺音类别的分类。
在本实施例中，本实施例首先获取神经网络模型，所述神经网络模型包括多个教师模型与学生模型，然后获取所述肺音数据集中的预测标签。接着基于所述预测标签与真实标签，对所述教师模型进行训练，得到已训练的多个教师模型，所述真实标签为肺音类别。在本实施例中，每一个已训练的教师模型为二分类模型，假设共用M种肺音类别，则就需要对应训练M个教师模型，以使得每一个已训练的教师模型可以分别对应两种肺音类别的分类。本实施例中的肺音数据集为
Figure PCTCN2020130136-appb-000004
数据对应的预测标签定义为
Figure PCTCN2020130136-appb-000005
本实施例中，所述教师模型是一个6层的CNN(Convolutional Neural Networks，卷积神经网络)模型，第1层是有24个滤波器的卷积层，卷积核大小为(5,5)，步幅为(4,2)。第2层同样是卷积层，包含48个滤波器，卷积核大小、步幅与第1层相同。第3层是一个最大池化层，其pool size为(2,2)。第4层卷积层的滤波器数量和卷积核大小与第2层相同。第5层全连接层有64个隐藏单元，最后一层为输出层。
在训练多个教师模型的过程中，每个教师模型分别对应一种肺音类别，拟合来自不同医院采集到的肺音音频数据，训练目标为最小化教师模型的预测标签和真实标签之间的交叉熵损失，对于教师模型T i，其损失函数如下公式所示：
Figure PCTCN2020130136-appb-000006
这里
Figure PCTCN2020130136-appb-000007
表示
Figure PCTCN2020130136-appb-000008
中元素数量，
Figure PCTCN2020130136-appb-000009
表示
Figure PCTCN2020130136-appb-000010
中的一个数据点，
Figure PCTCN2020130136-appb-000011
是与
Figure PCTCN2020130136-appb-000012
相对应的类别标签，
Figure PCTCN2020130136-appb-000013
表示教师模型T i的预测函数。
步骤S300、根据所述已训练的多个教师模型分别计算所述肺音数据集的软标签，并将其转换后，得到多分类软标签。
对于每一个训练好的教师模型而言，其已经学习了某一种肺音类别的数据特征。先由教师模型计算学生模型待学习的数据集X S的软标签，然后让学生模型拟合这些软标签，可以将教师模型的知识传授给学生模型，即实现知识蒸馏。在本实施例中，对于数据集X S中的一个数据点
Figure PCTCN2020130136-appb-000014
通过教师模型T i计算其软标签时,如以下公式所示：
Figure PCTCN2020130136-appb-000015
这里的
Figure PCTCN2020130136-appb-000016
表示教师模型T i对应音频类别j∈{0,1}时logits层的输出，t为温控超参数，t取值越大，软标签的分布越平滑。
由于每一个教师模型T i都是一个二分类的模型，其logits层的输出只有2个类别，而对于另外的M-1个类别，本实施例采用取0值的方式，以此将多个二分类模型的软标签转换为一个多分类的软标签
Figure PCTCN2020130136-appb-000017
从而与学生模型的类别维度统一，
Figure PCTCN2020130136-appb-000018
是与
Figure PCTCN2020130136-appb-000019
相对应的类别标签。
Figure PCTCN2020130136-appb-000020
的计算如以下公式所示：
Figure PCTCN2020130136-appb-000021
其中，j∈{0,...,M}，对于任意一个
Figure PCTCN2020130136-appb-000022
如果
Figure PCTCN2020130136-appb-000023
定义0为正常类别，那么函数
Figure PCTCN2020130136-appb-000024
的返回值为集合{1,...,M}中任意一个元素，且满足每个元素被选中的概率相等；否则，函数
Figure PCTCN2020130136-appb-000025
直接返回
Figure PCTCN2020130136-appb-000026
这样通过训练后的所述学 生模型就可以对肺音音频进行分类，从而得到肺音音频对应的类别。
步骤S400、基于所述多分类软标签和肺音数据集训练学生模型，得到已训练的学生模型，并利用所述已训练的学生模型对所述肺音音频进行分类，所述已训练的学生模型为多分类模型，所述已训练的学生模型对应于多种肺音类别的分类。
本实施例中获取所述多分类软标签；基于所述多分类软标签与肺音数据集，对所述学生模型进行训练，得到所述已训练的学生模型，所述已训练的学生模型的软标签接近所述已训练的教师模型的软标签；利用所述已训练的学生模型对所述肺音音频进行分类，得到所述肺音音频的类别。
在本实施例中，学生模型对于待学习的数据集X S对应的标签定义为Y S。所述学生模型是在教师模型的基础上进行简化的一个模型。该学生模型共有6层，第1层是有6个滤波器的卷积层，卷积核大小为(5,5)，步幅为(4,2)。第2层同样是卷积层，包含18个滤波器，卷积核大小、步幅与第1层相同。第3层是一个最大池化层，其pool size为(2,2)。第4卷积层有24个滤波器，卷积核大小与第2层相同。第5层全连接层有32个隐藏单元，然后连接输出层。
在本实施例中，学生模型的软标签计算与教师模型类似，区别在于学生模型需要考虑M+1个类别，如以下公式所示：
Figure PCTCN2020130136-appb-000027
这里的g j(.)表示学生模型对应类别j∈{0,...,M}时logits层的输出，0表示正常的类别。
除此之外，本实施例的学生模型的学习目标和教师模型也不同，学生模型一方面需要使得预测
Figure PCTCN2020130136-appb-000028
尽量接近真实标签
Figure PCTCN2020130136-appb-000029
另外一方面需要使得自身的软标签
Figure PCTCN2020130136-appb-000030
接近教师模型的软标签
Figure PCTCN2020130136-appb-000031
与真实标签相比，软标签
Figure PCTCN2020130136-appb-000032
具有更为丰富的信息，其包含样本中类与类之间的关联，这些关联由复杂的教师模型学习得到。学生 模型由于自身结构比较简单，对样本的学习能力比教师模型弱，但如果在训练过程中，让学生模型拟合教师模型学习到的关联信息，能够对自身的学习起到辅助作用。
但是，由于教师模型也是存在一定的出错概率，与单纯的向教师模型学习相比，添加真实标签就可以减少学生模型学习错误知识的可能性，学生模型的损失如以下公式所示：
Figure PCTCN2020130136-appb-000033
这里的|X S|表示X S中元素数量，f s表示S的预测函数。λ∈[0,1]是一个常数，用于学生模型选择侧重真实标签，还是侧重软标签。
在相同实验条件下，现有技术中的模型准确率为98％，参数量为0.28M，预测8000条数据的时耗为3.12秒，而本实施例中的学生模型准确率为95％，参数量为0.06M，预测同样的数据时耗为2.48秒。本方案的模型虽然在准确率上比现有技术中的模型低0.03，但是参数量比其减少了79％，完成相同任务比其节省了20％时间。可见，本发明可以通过多个教师模型的知识蒸馏，指导学生模型学习，从而充分利用多方肺音音频，并准确对所述肺音音频进行分类。
基于上述实施例，本发明还提供一种基于知识蒸馏的肺音分析系统，如图2中所示，所述系统包括：音频特征获取单元10、教师模型训练单元20、软标签拟合单元30以及学生模型训练单元40。
具体地，所述音频特征获取单元10，用于获取肺音音频，并基于所述肺音音频提取所述肺音音频的梅尔频谱特征，得到肺音数据集。
所述教师模型训练单元20，用于获取神经网络模型，所述神经网络模型包括多个教师模型与学生模型，基于所述肺音数据集对多个教师模型进行训练，得到已训练的多个教师模型，所述已训练的多个教师模型为二分类模型，用于对所述肺音音频进行分类，且每一个已训练的教师模型对应两种肺音类别的分类。
所述软标签拟合单元30，用于根据所述已训练的多个教师模型分别计算所述肺音数据集的软标签，并将其转换后，得到多分类软标签。
所述学生模型训练单元40，用于基于所述多分类软标签和肺音数据集训练学生模型，得到已训练的学生模型，并利用所述已训练的学生模型对所述肺音音频进行分类，所述已训练的学生模型为多分类模型，所述已训练的学生模型对应于多种肺音类别的分类。
基于上述实施例，本发明还提供了一种智能终端，其原理框图可以如图3所示。该智能终端包括通过系统总线连接的处理器、存储器、网络接口、显示屏、温度传感器。其中，该智能终端的处理器用于提供计算和控制能力。该智能终端的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统和计算机程序。该内存储器为非易失性存储介质中的操作系统和计算机程序的运行提供环境。该智能终端的网络接口用于与外部的智能终端通过网络连接通信。该计算机程序被处理器执行时以实现一种基于知识蒸馏的肺音分析方法。该智能终端的显示屏可以是液晶显示屏或者电子墨水显示屏，该智能终端的温度传感器是预先在智能终端内部设置，用于检测内部设备的运行温度。
本领域技术人员可以理解，图3中示出的原理框图，仅仅是与本发明方案相关的部分结构的框图，并不构成对本发明方案所应用于其上的智能终端的限定，具体的智能终端可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
在一个实施例中，提供了一种智能终端，包括有存储器，以及一个或者一个以上的程序，其中一个或者一个以上程序存储于存储器中，且经配置以由一个或者一个以上处理器执行所述一个或者一个以上程序包含用于进行以下操作的指令：
获取肺音音频，并基于所述肺音音频提取所述肺音音频的梅尔频谱特征，得到肺音数据集；
获取神经网络模型，所述神经网络模型包括多个教师模型与学生模型，基于所述肺音数据集对多个教师模型进行训练，得到已训练的多个教师模型，所述已训练的多个教师模型为二分类模型，用于对所述肺音音频进行分类，且每一个已训练的教师模型对应两种肺音类别的分类；
根据所述已训练的多个教师模型分别计算所述肺音数据集的软标签，并将其转换后，得到多分类软标签；
基于所述多分类软标签和肺音数据集训练所述学生模型，得到已训练的学生模型，并利用所述已训练的学生模型对所述肺音音频进行分类，所述已训练的学生模型为多分类模型，所述已训练的学生模型对应于多种肺音类别的分类。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一非易失性计算机可读取存储介质中，该计算机程序在执行时，可包括如上述各方法的实施例的流程。其中，本发明所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和/或易失性存储器。非易失性存储器可包括只读存储器(ROM)、可编程ROM(PROM)、 电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)或者外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDRSDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)等。
综上，本发明公开了一种基于知识蒸馏的肺音分类方法、系统、终端及存储介质，所述方法包括：获取肺音音频，并基于所述肺音音频提取所述肺音音频的梅尔频谱特征，得到肺音数据集；基于所述肺音数据集对多个教师模型进行训练，得到已训练的多个教师模型；根据所述已训练的多个教师模型分别计算所述肺音数据集的软标签，并将其转换后，得到多分类软标签；基于所述多分类软标签和肺音数据集训练所述学生模型，得到已训练的多个学生模型，并利用所述已训练的学生模型对所述肺音音频进行分类。本发明可以基于知识蒸馏，通过利用多个教师模型的知识，指导学生模型学习，从而充分利用多方肺音音频，并准确对所述肺音音频进行分类。
最后应说明的是：以上实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。

Claims (10)

  1. 一种基于知识蒸馏的肺音分类方法，其特征在于，所述方法包括：
    获取肺音音频，并基于所述肺音音频提取所述肺音音频的梅尔频谱特征，得到肺音数据集；
    获取神经网络模型，所述神经网络模型包括多个教师模型与学生模型，基于所述肺音数据集对多个教师模型进行训练，得到已训练的多个教师模型，所述已训练的多个教师模型为二分类模型，用于对所述肺音音频进行分类，且每一个已训练的教师模型对应两种肺音类别的分类；
    根据所述已训练的多个教师模型分别计算所述肺音数据集的软标签，并将其转换后，得到多分类软标签；
    基于所述多分类软标签和肺音数据集训练所述学生模型，得到已训练的学生模型，并利用所述已训练的学生模型对所述肺音音频进行分类，所述已训练的学生模型为多分类模型，所述已训练的学生模型对应于多种肺音类别的分类。
  2. 根据权利要求1所述的基于知识蒸馏的肺音分类方法，其特征在于，所述获取肺音音频，并基于所述肺音音频提取所述肺音音频的梅尔频谱特征，得到肺音数据集，包括：
    采集所述肺音音频，并对所述肺音音频进行音频数据增强处理，得到增强后的肺音音频；
    对所述增强后的肺音音频进行切片，得到多个音频片段；
    分别从每一个所述音频片段中提取所述梅尔频谱特征，得到所述肺音数据集。
  3. 根据权利要求2所述的基于知识蒸馏的肺音分类方法，其特征在于，所述音频数据增强处理包括：时间变换处理、音高变换处理和μ律压缩处理。
  4. 根据权利要求2所述的基于知识蒸馏的肺音分类方法，其特征在于，所述获取肺音音频，并基于所述肺音音频提取所述肺音音频的梅尔频谱特征，得到肺音数据集之后，包括：
    对每一个所述音频片段进行零均值归一化处理。
  5. 根据权利要求1所述的基于知识蒸馏的肺音分类方法，其特征在于，所述基于所述肺音数据集对多个教师模型进行训练，得到已训练的多个教师模型，包括：
    获取所述肺音数据集中的预测标签，所述预测标签为所述肺音数据集中的梅尔频谱特征与预测音频类别的对应关系；
    基于所述预测标签与真实标签，对多个教师模型进行训练，得到所述已训练的多个教师模型，所述真实标签为肺音类别。
  6. 根据权利要求1所述的基于知识蒸馏的肺音分类方法，其特征在于，所述基于所述多分类软标签和肺音数据集训练所述学生模型，得到已训练的学生模型，并利用所述已训练的 学生模型对所述肺音音频进行分类，包括：
    获取所述多分类软标签；
    基于所述多分类软标签与肺音数据集，对所述学生模型进行训练，得到所述已训练的学生模型，所述已训练的学生模型的软标签接近所述已训练的多个教师模型的软标签；
    利用所述已训练的学生模型对所述肺音音频进行分类，得到所述肺音音频的类别。
  7. 根据权利要求1所述的基于知识蒸馏的肺音分类方法，其特征在于，所述教师模型与所述学生模型均为一个6层的卷积神经网络模型。
  8. 一种基于知识蒸馏的肺音分析系统，其特征在于，所述系统包括：
    音频特征获取单元，用于获取肺音音频，并基于所述肺音音频提取所述肺音音频的梅尔频谱特征，得到肺音数据集；
    教师模型训练单元，用于获取神经网络模型，所述神经网络模型包括多个教师模型与学生模型，基于所述肺音数据集对多个教师模型进行训练，得到已训练的多个教师模型，所述已训练的多个教师模型为二分类模型，用于对所述肺音音频进行分类，且每一个已训练的教师模型对应两种肺音类别的分类；
    软标签拟合单元，用于根据所述已训练的多个教师模型分别计算所述肺音数据集的软标签，并将其转换后，得到多分类软标签；
    学生模型训练单元，用于基于所述多分类软标签和肺音数据集训练所述学生模型，得到已训练的学生模型，并利用所述已训练的学生模型对所述肺音音频进行分类，所述学生模型为多分类模型，所述已训练的学生模型对应于多种肺音类别的分类。
  9. 一种智能终端，其特征在于，包括有存储器，以及一个或者一个以上的程序，其中一个或者一个以上程序存储于存储器中，且经配置以由一个或者一个以上处理器执行所述一个或者一个以上程序包含用于执行如权利要求1-7中任意一项所述的方法。
  10. 一种非临时性计算机可读存储介质，其特征在于，当所述存储介质中的指令由电子设备的处理器执行时，使得电子设备能够执行如权利要求1-7中任意一项所述的方法。
PCT/CN2020/130136 2020-10-09 2020-11-19 基于知识蒸馏的肺音分类方法、系统、终端及存储介质 Ceased WO2022073285A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011070732.X 2020-10-09
CN202011070732.XA CN111933185A (zh) 2020-10-09 2020-10-09 基于知识蒸馏的肺音分类方法、系统、终端及存储介质

Publications (1)

Publication Number Publication Date
WO2022073285A1 true WO2022073285A1 (zh) 2022-04-14

Family

ID=73334332

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/130136 Ceased WO2022073285A1 (zh) 2020-10-09 2020-11-19 基于知识蒸馏的肺音分类方法、系统、终端及存储介质

Country Status (2)

Country Link
CN (1) CN111933185A (zh)
WO (1) WO2022073285A1 (zh)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115019317A (zh) * 2022-06-13 2022-09-06 首都师范大学 一种文本识别方法及系统
CN115064155A (zh) * 2022-06-09 2022-09-16 福州大学 一种基于知识蒸馏的端到端语音识别增量学习方法及系统
CN116110022A (zh) * 2022-12-10 2023-05-12 河南工业大学 基于响应知识蒸馏的轻量化交通标志检测方法及系统
CN116189055A (zh) * 2023-02-27 2023-05-30 美的集团(上海)有限公司 视频识别模型训练方法、视频识别方法、设备和存储介质
CN116312628A (zh) * 2023-02-09 2023-06-23 安徽大学 基于自我知识蒸馏的虚假音频检测方法及其系统
CN116563731A (zh) * 2023-03-31 2023-08-08 浙江武义电气安装工程有限公司 一种基于知识蒸馏的无人机影像云边协同识别方法
CN116597867A (zh) * 2023-05-04 2023-08-15 广东白云学院 一种肺音数据相似性匹配的方法及系统
CN116825130A (zh) * 2023-08-24 2023-09-29 硕橙（厦门）科技有限公司 一种深度学习模型蒸馏方法、装置、设备及介质
CN118522309A (zh) * 2024-07-22 2024-08-20 浙江交科环境科技有限公司 利用卷积神经网络进行公路沿线噪声源识别的方法及装置
CN119475153A (zh) * 2024-10-22 2025-02-18 华南理工大学 基于深度学习和知识蒸馏的车联网异常检测轻量化方法
CN119600331A (zh) * 2024-11-12 2025-03-11 四川大学 基于软标签的面部痤疮分类方法、系统、设备及介质
CN120183382A (zh) * 2025-04-14 2025-06-20 海南经贸职业技术学院 基于知识蒸馏的海南方言语音识别优化系统
CN121410481A (zh) * 2025-12-30 2026-01-27 山东科技大学 基于知识蒸馏的1d-cnn的在线局部放电识别方法

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111933185A (zh) * 2020-10-09 2020-11-13 深圳大学 基于知识蒸馏的肺音分类方法、系统、终端及存储介质
CN114566175A (zh) * 2020-11-27 2022-05-31 北京搜狗科技发展有限公司 一种语音增强及模型训练方法、装置和电子设备
US12321846B2 (en) * 2020-12-09 2025-06-03 International Business Machines Corporation Knowledge distillation using deep clustering
CN112687290B (zh) * 2020-12-30 2022-09-20 同济大学 一种经过压缩的咳嗽自动检测方法及嵌入式设备
CN114299993B (zh) * 2021-12-24 2024-12-03 中北大学 一种基于数据蒸馏的多枪声混合智能分类识别系统及方法
CN114266897B (zh) * 2021-12-24 2025-06-20 深圳数联天下智能科技有限公司 痘痘类别的预测方法、装置、电子设备及存储介质
WO2023245381A1 (zh) * 2022-06-20 2023-12-28 北京小米移动软件有限公司 识别音频数据类别的方法、装置及存储介质
CN115329956A (zh) * 2022-07-20 2022-11-11 北京百度网讯科技有限公司 一种模型训练方法、装置、电子设备和存储介质
CN116129182A (zh) * 2023-01-10 2023-05-16 南京大学 一种基于知识蒸馏和近邻分类的多维度医疗图像分类方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110098591A1 (en) * 2008-05-29 2011-04-28 Technion Research And Development Foundation Ltd. Carbon nanotube structures in sensor apparatuses for analyzing biomarkers in breath samples
CN106022258A (zh) * 2016-05-18 2016-10-12 成都济森科技有限公司 数字听诊器与滤除心音提取肺音的方法
CN107818366A (zh) * 2017-10-25 2018-03-20 成都力创昆仑网络科技有限公司 一种基于卷积神经网络的肺音分类方法、系统及用途
CN110705624A (zh) * 2019-09-26 2020-01-17 广东工业大学 一种基于多信噪比模型的心肺音分离方法及系统
CN110970042A (zh) * 2019-12-13 2020-04-07 苏州美糯爱医疗科技有限公司 一种电子听诊器的肺部啰音人工智能实时分类方法、系统、装置及可读存储介质
CN111933185A (zh) * 2020-10-09 2020-11-13 深圳大学 基于知识蒸馏的肺音分类方法、系统、终端及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110098591A1 (en) * 2008-05-29 2011-04-28 Technion Research And Development Foundation Ltd. Carbon nanotube structures in sensor apparatuses for analyzing biomarkers in breath samples
CN106022258A (zh) * 2016-05-18 2016-10-12 成都济森科技有限公司 数字听诊器与滤除心音提取肺音的方法
CN107818366A (zh) * 2017-10-25 2018-03-20 成都力创昆仑网络科技有限公司 一种基于卷积神经网络的肺音分类方法、系统及用途
CN110705624A (zh) * 2019-09-26 2020-01-17 广东工业大学 一种基于多信噪比模型的心肺音分离方法及系统
CN110970042A (zh) * 2019-12-13 2020-04-07 苏州美糯爱医疗科技有限公司 一种电子听诊器的肺部啰音人工智能实时分类方法、系统、装置及可读存储介质
CN111933185A (zh) * 2020-10-09 2020-11-13 深圳大学 基于知识蒸馏的肺音分类方法、系统、终端及存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
LI JIANQIANG, WANG CHENG;HUANG ZHI-CHAO;CHEN JIE: "Lung disease analysis using ensemble knowledge distillation", KONGZHI LILUN YU YINGYONG - CONTROL THEORY & APPLICATIONS, HUANAN LIGONG DAXUE,, CN, vol. 38, no. 1, 27 September 2020 (2020-09-27), CN , pages 130 - 136, XP055920115, ISSN: 1000-8152, DOI: 10.7641/CTA.2020.00235 *

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115064155A (zh) * 2022-06-09 2022-09-16 福州大学 一种基于知识蒸馏的端到端语音识别增量学习方法及系统
CN115019317A (zh) * 2022-06-13 2022-09-06 首都师范大学 一种文本识别方法及系统
CN116110022A (zh) * 2022-12-10 2023-05-12 河南工业大学 基于响应知识蒸馏的轻量化交通标志检测方法及系统
CN116110022B (zh) * 2022-12-10 2023-09-05 河南工业大学 基于响应知识蒸馏的轻量化交通标志检测方法及系统
CN116312628A (zh) * 2023-02-09 2023-06-23 安徽大学 基于自我知识蒸馏的虚假音频检测方法及其系统
CN116189055A (zh) * 2023-02-27 2023-05-30 美的集团(上海)有限公司 视频识别模型训练方法、视频识别方法、设备和存储介质
CN116563731A (zh) * 2023-03-31 2023-08-08 浙江武义电气安装工程有限公司 一种基于知识蒸馏的无人机影像云边协同识别方法
CN116597867A (zh) * 2023-05-04 2023-08-15 广东白云学院 一种肺音数据相似性匹配的方法及系统
CN116825130A (zh) * 2023-08-24 2023-09-29 硕橙（厦门）科技有限公司 一种深度学习模型蒸馏方法、装置、设备及介质
CN116825130B (zh) * 2023-08-24 2023-11-21 硕橙（厦门）科技有限公司 一种深度学习模型蒸馏方法、装置、设备及介质
CN118522309A (zh) * 2024-07-22 2024-08-20 浙江交科环境科技有限公司 利用卷积神经网络进行公路沿线噪声源识别的方法及装置
CN119475153A (zh) * 2024-10-22 2025-02-18 华南理工大学 基于深度学习和知识蒸馏的车联网异常检测轻量化方法
CN119600331A (zh) * 2024-11-12 2025-03-11 四川大学 基于软标签的面部痤疮分类方法、系统、设备及介质
CN119600331B (zh) * 2024-11-12 2025-07-01 四川大学 基于软标签的面部痤疮分类方法、系统、设备及介质
CN120183382A (zh) * 2025-04-14 2025-06-20 海南经贸职业技术学院 基于知识蒸馏的海南方言语音识别优化系统
CN121410481A (zh) * 2025-12-30 2026-01-27 山东科技大学 基于知识蒸馏的1d-cnn的在线局部放电识别方法

Also Published As

Publication number Publication date
CN111933185A (zh) 2020-11-13

Similar Documents

Publication Publication Date Title
WO2022073285A1 (zh) 基于知识蒸馏的肺音分类方法、系统、终端及存储介质
CN109243491B (zh) 在频谱上对语音进行情绪识别的方法、系统及存储介质
CN108877775B (zh) 语音数据处理方法、装置、计算机设备及存储介质
CN119360278A (zh) 一种基于多模态分析的危险行为识别与预警方法
WO2019232851A1 (zh) 语音区分模型训练方法、装置、计算机设备及存储介质
CN119908724A (zh) 基于多智能体协同的多模态情感分析方法及装置
Yan et al. Optimizing MFCC parameters for the automatic detection of respiratory diseases
CN117762372A (zh) 一种多模态人机交互系统
WO2019232848A1 (zh) 语音区分方法、装置、计算机设备及存储介质
CN116842460A (zh) 基于注意力机制与残差神经网络的咳嗽关联疾病识别方法和系统
CN119131018A (zh) 基于自监督学习和多任务学习的病理图像分析方法及设备
Revathi et al. Robust respiratory disease classification using breathing sounds (RRDCBS) multiple features and models
CN118351888A (zh) 一种根据家禽叫声判断呼吸道疾病的方法及系统
CN120670911A (zh) 基于多域特征融合与轻量建模的水下声呐目标识别系统
Merzougui et al. Diagnosing spasmodic dysphonia with the power of AI
CN119889367A (zh) 基于深度学习的睡眠呼吸障碍识别系统
CN118762718A (zh) 一种能动态跟踪识别个人音色长期渐进性变化的方法与系统
Paramitha et al. Performance comparison of deep learning algorithm for speech emotion recognition
Samanta Classification analysis using CNN and LSTM on wheezing sounds
CN116486840A (zh) 一种基于自然语音的抑郁症自动评估系统
CN114974267A (zh) 鸟语分类模型训练方法及鸟语识别方法
CN119837542B (zh) 一种脑电信号预处理和特征提取方法及系统
CN119274591B (zh) 基于人工智能的语音情感分析方法及系统
Kanase et al. A Robust Approach of Estimating Voice Disorder Due to Thyroid Disease
Therese et al. A linear visual assessment tendency based clustering with power normalized cepstral coefficients for audio signal recognition system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20956600

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 11.07.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 20956600

Country of ref document: EP

Kind code of ref document: A1