KR20240107023A - Method and device for learning of auscultation sound analysis model - Google Patents

Abstract

A method and device for learning an auscultation sound analysis model are provided. The method of learning the auscultation sound analysis model is to pre-process the auscultation sound data to obtain learning data, pre-train the auscultation sound analysis model using the learning data, and analyze the pre-learned auscultation sound using the learning data and labeling data. It includes the step of supervised learning of the model.

Description

Method and device for learning auscultation sound analysis model {METHOD AND DEVICE FOR LEARNING OF AUSCULTATION SOUND ANALYSIS MODEL}

The present invention relates to a method and device for learning an auscultation sound analysis model.

Auscultation sound-based treatment is a simple method compared to other medical tests, but it is a very important treatment technique in that it can detect key signs related to diseases, and is widely adopted in various medical testing fields such as respiratory examination.

Recently, as digital auscultation technology has developed, research has been actively attempted to analyze digital auscultation sound data based on artificial intelligence (e.g., neural network models) and diagnose patient symptoms accordingly.

In order to analyze auscultation sounds based on artificial intelligence, a neural network model must be trained to extract meaningful information from the collected digital auscultation sound data, and the information extracted from the digital auscultation sound data must be delivered to the user using the learned neural network model.

However, due to limitations such as patient privacy protection, the number of collected digital auscultation sound data is small, and among the collected digital auscultation sound data, the data with normal auscultation sounds have a higher rate than the data with auscultation sounds for actual symptoms. Due to this problem, there is a problem that the learning performance of the neural network model using the collected digital auscultation sound data is deteriorated.

In this way, transfer learning for the neural network model was proposed to prevent the learning performance of the neural network model from deteriorating in an environment with sparse data, that is, a small number of data for training the neural network model. Transfer learning of a neural network model can solve the problem of overfitting of a neural network model to sparse data by using the knowledge expressed as the weights of the neural network model learned with separate data as the basis for learning on sparse data.

However, since digital auscultation sound data has different characteristics from general image or voice data, a problem of deterioration in learning performance occurs when transfer learning is applied to a neural network model for digital auscultation sound data.

For example, auscultation sound data has a gap in data distribution compared to general image or voice data due to the characteristic that classification must be performed according to symptoms that occur in a short period of time. This gap in data distribution causes a problem of reduced efficiency during transfer learning of neural network models.

In addition, auscultation sound data may have different characteristics even for the same symptom depending on the environment in which it was collected, such as factors such as the patient's age, gender, weight, and breathing pattern. Due to these characteristics of auscultation sound data, the neural network model learns the characteristics of each patient individually during transfer learning, resulting in a problem that reduces the accuracy of symptom classification for auscultation sound data.

Korean Patent No. 10-0805181 (2008.02.21.)

The purpose of the present invention is to provide a method and device for learning an auscultation sound analysis model that can enable learning to focus on symptom characteristics rather than patient characteristics even in an environment with a small number of auscultation sound data.

A method of learning an auscultation sound analysis model according to an embodiment of the present invention includes the steps of preprocessing auscultation sound data to obtain learning data; Pre-training the auscultation sound analysis model by providing the learning data to the auscultation sound analysis model; And providing actual symptom information and actual patient information using the learning data and labeling data to a pre-learned auscultation sound analysis model to supervised learning the pre-learned auscultation sound analysis model.

The step of pre-training the auscultation sound analysis model includes: replicating a symptom classification unit of the auscultation sound analysis model; Obtaining converted learning data by converting the learning data; Inputting the learning data into the symptom classification unit to obtain a first feature vector output from the symptom classification unit; Inputting the converted learning data into a duplicated symptom classifier to obtain a second feature vector output from the duplicated symptom classifier; determining a prior learning loss value for the symptom classifier based on the first feature vector and the second feature vector; and adjusting parameter weights of the symptom classifier so that the determined pre-learning loss value is minimized.

The step of pre-training the auscultation sound analysis model further includes generating a masking feature vector by masking at least one of the plurality of feature values of the first feature vector.

Determining the pre-learning loss value includes determining the pre-learning loss value based on the masking feature vector and the second feature vector.

Pre-training the auscultation sound analysis model includes obtaining an exponential moving average value for the parameter weight of the symptom classification unit; and adjusting parameter weights of the replicated symptom classifier based on the exponential moving average value.

Obtaining the transformed learning data includes obtaining the transformed learning data by transforming at least one of a time component or a frequency component of the learning data.

The step of supervised learning of the pre-trained auscultation sound analysis model includes providing the learning data and the actual symptom information to a symptom classification unit of the pre-learned auscultation sound analysis model to classify symptoms for the learning data from the symptom classification unit. obtaining a result; determining a supervised learning loss value based on the actual symptom data and the symptom classification result; Providing the intermediate output value of the symptom classification unit and the actual patient information to a patient information classification unit of the pre-learned auscultation sound analysis model to obtain a patient information classification result for the learning data from the patient information classification unit; determining an adversarial loss value based on the actual patient information and the patient information classification result; Determining a final loss value for the pre-trained auscultation sound analysis model based on the supervised learning loss value and the adversarial loss value; and adjusting parameter weights of the symptom classification unit so that the determined final loss value is minimized.

The step of acquiring the learning data includes converting the auscultation sound data into a time-frequency response signal in a two-dimensional vector form to obtain the learning data.

The auscultation sound analysis model learning device according to an embodiment of the present invention includes a memory storing a neural network model learning program; And executing the neural network model learning program, preprocessing the auscultation sound data to obtain learning data, providing the learning data to the auscultation sound analysis model to pre-train the auscultation sound analysis model, and pre-training the learning data to the auscultation sound analysis model. It includes a processor for supervised learning of the pre-learned auscultation sound analysis model by providing it to an auscultation sound analysis model.

The processor replicates the symptom classification unit of the auscultation sound analysis model, converts the learning data to obtain converted learning data, inputs the learning data into the symptom classification unit, and outputs a first feature from the symptom classification unit. Obtain a vector, input the converted learning data into the duplicated symptom classifier, obtain a second feature vector output from the duplicated symptom classifier, and based on the first feature vector and the second feature vector, A pre-learning loss value for the symptom classifier is determined, and parameter weights of the symptom classifier are adjusted so that the determined pre-learning loss value is minimized.

The processor generates a masking feature vector by masking at least one of the plurality of feature values of the first feature vector, and determines the dictionary learning loss value based on the masking feature vector and the second feature vector.

The processor obtains an exponential moving average value for the parameter weight of the symptom classification unit, and adjusts the parameter weight of the replicated symptom classification unit based on the exponential moving average value.

The processor obtains the converted learning data by converting at least one of the time component or the frequency component of the learning data.

The processor provides the learning data and the actual symptom information to a symptom classification unit of the pre-learned auscultation sound analysis model to obtain a symptom classification result for the learning data from the symptom classification unit, and obtains a symptom classification result for the learning data from the symptom classification unit. A supervised learning loss value is determined based on the symptom classification results, and the intermediate output value of the symptom classification unit and the actual patient information are provided to the patient information classification unit of the pre-trained auscultation sound analysis model to obtain the learning data from the patient information classification unit. Obtain a patient information classification result, determine an adversarial loss value based on the actual patient information and the patient information classification result, and analyze the pre-learned auscultation sound based on the supervised learning loss value and the adversarial loss value. The final loss value for the model is determined, and the parameter weights of the symptom classifier are adjusted so that the determined final loss value is minimized.

The processor acquires the learning data by converting the auscultation sound data into a time-frequency response signal in a two-dimensional vector form.

The present invention can pre-train an auscultation sound analysis model using learning data generated from auscultation sound data, and then supervised learning the pre-trained auscultation sound analysis model again.

In addition, the present invention can update the parameters of the auscultation sound analysis model by determining an adversarial loss value so that the auscultation sound analysis model is not trained by focusing on the classification of patient information included in the learning data.

Accordingly, the present invention can narrow the data distribution gap caused by model learning in an environment with a small number of auscultation sound data, and enable learning about symptoms rather than patient information from auscultation sound data, so that the auscultation sound analysis model Learning efficiency and performance can be improved.

Figure 1 is a diagram showing a learning device for an auscultation sound analysis model according to an embodiment of the present invention.
FIG. 2 is a diagram conceptually showing the function of the neural network model learning program of FIG. 1.
FIG. 3 is a diagram conceptually showing the function of the model dictionary learning unit of FIG. 2.
FIG. 4 is a diagram conceptually showing the function of the model supervised learning unit of FIG. 2.
Figure 5 is a diagram showing a method for supervised learning of the auscultation sound analysis model of the present invention.
Figure 6 is a diagram showing a method of learning an auscultation sound analysis model according to an embodiment of the present invention.
Figure 7 is a diagram showing a method for pre-training the auscultation sound analysis model of Figure 6.
Figure 8 is a diagram showing a method of supervised learning of the auscultation sound analysis model of Figure 6.

The advantages and features of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. The present embodiments are merely provided to ensure that the disclosure of the present invention is complete and to be understood by those skilled in the art It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims.

In describing embodiments of the present invention, if a detailed description of a known function or configuration is judged to unnecessarily obscure the gist of the present invention, the detailed description will be omitted. The terms described below are defined in consideration of functions in the embodiments of the present invention, and may vary depending on the intention or custom of the user or operator. Therefore, the definition should be made based on the contents throughout this specification.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

Figure 1 is a diagram showing a learning device for an auscultation sound analysis model according to an embodiment of the present invention.

Referring to FIG. 1, the auscultation sound analysis model learning device 100 of this embodiment may include an input/output unit 110, a processor 120, and a memory 130.

The input/output unit 110 may receive stethoscope sound data from the outside. In addition, the input/output unit 110 may receive learning data generated from auscultation sound data through the processor 120, which will be described later, and output it to the auscultation sound analysis model 200.

When the processor 120 receives auscultation sound data through the input/output unit 110, the processor 120 may generate learning data corresponding to the auscultation sound data using the neural network model learning program 140 stored in the memory 130.

In addition, the processor 120 may output learning data to the auscultation sound analysis model 200 through the input/output unit 110 and train the auscultation sound analysis model 200 using the learning data.

Here, the auscultation sound analysis model 200 may include a symptom classification unit 210 and a patient information classification unit 220, each of which includes a plurality of convolution layers. This auscultation sound analysis model 200 can perform learning at least twice using learning data by the processor 120.

For example, the processor 120 may pre-train the auscultation sound analysis model 200 using learning data. In addition, the processor 120 may supervised learning the pre-trained auscultation sound analysis model 200 using the learning data. At this time, the processor 120 provides labeling data, such as actual patient symptom information or actual patient information, together with the learning data to the auscultation sound analysis model 200, and the auscultation sound analysis model 200 It can be supervised learning.

The memory 130 may store the neural network model learning program 140 and information necessary for its execution. The neural network model learning program 140 generates learning data for learning the auscultation sound analysis model 200 from the auscultation sound data provided through the input/output unit 110, and uses the generated learning data to create an auscultation sound analysis model (200). ) may be software that includes instructions that can learn.

Accordingly, when the processor 120 receives auscultation sound data through the input/output unit 110, the processor 120 executes the neural network model learning program 140 to generate learning data from the auscultation sound data, and outputs it to the auscultation sound analysis model 200. Thus, the auscultation sound analysis model 200 can be pre-trained and supervised learned.

FIG. 2 is a diagram conceptually illustrating the function of the neural network model learning program of FIG. 1, FIG. 3 is a diagram conceptually illustrating the function of the model dictionary learning unit of FIG. 2, and FIG. 4 is a diagram conceptually illustrating the function of the model supervised learning unit of FIG. 2. , and Figure 5 is a diagram showing a method for supervised learning of the auscultation sound analysis model of the present invention.

Referring to FIG. 2, the neural network model learning program 140 of this embodiment may include a data pre-processing unit 141, a model pre-training unit 143, and a model supervised learning unit 145.

The data pre-processing unit 141, model dictionary learning unit 143, and model guidance learning unit 145 shown in FIG. 2 are divided to easily explain the function of the neural network model learning program 140 of this embodiment, and are used according to the present invention. is not limited to this.

For example, the functions of the data pre-processing unit 141, the model dictionary learning unit 143, and the model supervised learning unit 145 may be merged or separated, and may be implemented as a series of instructions included in one program. .

The data pre-processing unit 141 may generate a plurality of learning data by pre-processing the auscultation sound data provided through the input/output unit 110.

For example, the stethoscope sound data received by the input/output unit 110 may be a one-dimensional vector waveform signal. Accordingly, the data preprocessor 141 performs preprocessing to convert the auscultation sound data into a frequency-time response signal in the form of a two-dimensional vector based on [Equation 1] below to generate learning data from the auscultation sound data. .

[Equation 1]

Here, n is the time index in the digital audio signal, m is the additional time index for the convolution operation, w is the Hann or Gaussian window function to be used in the convolution operation, and R is the processing interval (hop size) of the discrete-time Fourier transform, is the frequency value and x(n) is the audio signal intensity at the nth discrete time, may be a discrete-time Fourier transform value of data centered on the discrete time mR.

The model pre-learning unit 143 may provide the learning data generated in the data pre-processing unit 141 to the auscultation sound analysis model 200 to pre-train the auscultation sound analysis model 200. Here, the model pre-training unit 143 may pre-train the symptom classification unit 210 of the auscultation sound analysis model 200 using the learning data.

Referring to FIG. 3, the model dictionary learning unit 143 in this embodiment includes a neural network model replication unit 151, a learning data conversion unit 152, a feature vector masking unit 153, a dictionary learning loss calculation unit 154, and a model It may include an update unit, for example, a first update unit 155.

Here, for convenience of explanation, the model update unit of the model dictionary learning unit 143 will be referred to as the first update unit, and the model update unit of the model supervised learning unit 145, which will be described later, will be referred to as the second update unit 163. I decided to do it.

The neural network model replication unit 151 may replicate the symptom classification unit 210 of the auscultation sound analysis model 200.

As described above, the symptom classification unit 210 of the auscultation sound analysis model 200 may include a plurality of convolutional layers. Accordingly, the neural network model replication unit 151 may construct a replicated symptom classification unit by replicating each of the plurality of convolutional layers of the symptom classification unit 210.

The learning data conversion unit 152 may convert the learning data to generate one or more converted learning data.

Here, the learning data may be a frequency-time response signal in the form of a two-dimensional vector generated by the data preprocessor 141. The learning data conversion unit 152 may generate one or more converted learning data from the learning data by converting at least one of the frequency component or the time component of the above-described learning data.

For example, the learning data conversion unit 152 may generate first converted learning data by clipping some of the frequency components of the learning data. Additionally, the learning data conversion unit 152 may generate second converted learning data by clipping some of the time components of the learning data.

The learning data generated through the data pre-processing unit 141 is provided as input to the symptom classification unit 210 of the auscultation sound analysis model 200, and the converted learning data generated through the learning data conversion unit 152 is replicated. It can be provided as input to a symptom classification unit. Accordingly, the auscultation sound analysis model 200 of this embodiment may perform dictionary learning based on the learning data and converted learning data.

The feature vector masking unit 153 acquires the first feature vector output from the symptom classification unit 210 for the learning data, and masks at least one of the plurality of feature values of the obtained first feature vector to create a masked feature vector. can be created.

For example, the first feature vector may be an N-dimensional vector (N is a natural number equal to or greater than 2) with a plurality of feature values. The feature vector masking unit 153 may generate a feature value mask that can mask at least one feature value among the plurality of feature values of the above-described first feature vector, for example, change it to 0. Here, the feature value mask may be a vector that has the same number of feature values as the first feature vector, but some of the feature values are masked to 0.

Accordingly, the feature vector masking unit 153 may generate a masking feature vector for the first feature vector through element product between the first feature vector and the feature value mask, as shown in [Equation 2] below.

[Equation 2]

Here, F may be the first feature vector, and M may be a feature value mask.

In this way, the feature vector masking unit 153 creates a masked feature vector by masking some feature values of the first feature vector, thereby solving the problem of lack of diversity in generating converted data of the data conversion unit 152. This can be explained by perturbing the existing feature vector through random masking, thereby increasing the entropy of the feature vector.

The dictionary learning loss calculation unit 154 acquires the second feature vector output for the converted learning data from the duplicate symptom classification unit, and calculates the second feature vector based on the masking feature vector and the second feature vector generated through the feature vector masking unit 153. Thus, the loss value according to the pre-learning of the auscultation sound analysis model 200, for example, the pre-learning loss value of the symptom classification unit 210 can be determined.

Here, the pre-learning loss value can be determined as a distance-based loss value between vectors according to [Equation 3] below.

[Equation 3]

Here, F' is a masking vector, F'' is a second feature vector, i is a dimension index, and n may be the total number of dimensions of the feature vector.

The first update unit 155 may adjust the weight of one or more parameters of the symptom classification unit 210 of the auscultation sound analysis model 200 so that the pre-learning loss value is minimized based on the determined pre-learning loss value. .

For example, the first update unit 155 may calculate the weight according to [Equation 4] below.

[Equation 4]

here, is the previous parameter weight of the symptom classification unit, is the learning rate that specifies the degree of update of the parameter weights, is the weight is the gradient from the loss function for , and D may be information such as data given to calculate this loss function and gradient.

The first update unit 155 may adjust the parameter weights of the symptom classification unit 210 based on the calculated weights.

Additionally, the first update unit 155 may calculate an exponential moving average value for the adjusted parameter weights. Next, the first update unit 155 may adjust the parameter weights of the replicated symptom classification unit based on the exponential moving average value.

In this way, the model pre-training unit 143 of this embodiment generates converted learning data from the training data, uses these to pre-train the symptom classification unit 210 and its duplicate symptom classification unit, and classifies symptoms with the determined loss value. Each of the unit 210 and the duplicated symptom classification unit can be updated. Pre-learning for the symptom classifier 210 may be repeatedly performed until the pre-learning loss value becomes less than a set reference value.

Referring again to FIG. 2, the model supervised learning unit 145 may supervised learning the pre-trained auscultation sound analysis model 200 using learning data and corresponding labeling data.

Referring to FIG. 4, the supervised model learning unit 145 of this embodiment may include a supervised learning loss calculation unit 161, an adversarial loss calculation unit 162, and a second update unit 163.

Referring to Figures 4 and 5, the supervised learning loss calculation unit 161 is a symptom classification unit 210 based on the symptom classification result obtained from the symptom classification unit 210 of the pre-trained auscultation sound analysis model 200. The supervised learning loss value for can be determined.

For example, the model guidance learning unit 145 may provide learning data generated in the data pre-processing unit 141 as input to the symptom classification unit 210 of the pre-trained auscultation sound analysis model 200. Additionally, the model guidance learning unit 145 may provide actual symptom information as labeling data as input to the symptom classification unit 210. Accordingly, the symptom classification unit 210 may output a symptom classification result for the input learning data, that is, a symptom analysis result for the auscultation sound of the learning data.

The supervised learning loss calculation unit 161 may compare the symptom classification result output from the symptom classification unit 210 with the labeling data and determine a supervised learning loss value according to the comparison result. Here, the supervised learning loss calculation unit 161 may determine the supervised learning loss value according to the cross entropy of [Equation 5] below.

[Equation 5]

Here, c is the symptom index, Pc is the truth symptom function (1 at the corresponding index), can be a probability inference about symptoms.

The adversarial loss calculation unit 162 may determine the adversarial loss value of the patient information classification unit 220 based on the patient information classification result obtained from the patient information classification unit 220 of the pre-learned auscultation sound analysis model 200. .

For example, the model supervised learning unit 145 may provide an intermediate output value output from one of the plurality of convolutional layers of the symptom classification unit 210 as an input to the patient information classification unit 220. Additionally, the model supervised learning unit 145 can provide actual patient information as labeling data as input to the patient information classification unit 220. Accordingly, the patient information classification unit 220 may output a patient information classification result for the input intermediate output value, that is, a classification result for patient information such as patient number, age, and gender included in the learning data.

The adversarial loss calculation unit 162 may compare the patient information classification result output from the patient information classification unit 220 with the labeling data and determine a loss value according to the comparison result. Additionally, the adversarial loss calculation unit 162 may determine the adversarial loss value so that the loss value increases based on the determined loss value. Here, the adversarial loss calculation unit 162 can determine the adversarial loss value according to [Equation 6] below.

[Equation 6]

where pid is the patient index, is the truth patient information function, may be a classification probability inference for the patient.

The second update unit 163 determines the final loss value of the pre-trained auscultation sound analysis model 200 based on the determined supervised learning loss value and the adversarial loss value, and the final loss value is determined based on the determined final loss value. The weight of one or more parameters of the symptom classification unit 210 may be adjusted to minimize the weight.

For example, the second update unit 163 may determine the final loss value by adding the supervised learning loss value and the adversarial loss value as shown in [Equation 7] below.

[Equation 7]

Next, the second update unit 163 can calculate a weight that minimizes the predetermined final loss value, as shown in [Equation 8] below. is designed to be learned in the direction of growth as mentioned above, so the minimum at this time can be regarded as an act of finding a kind of saddle point.

[Equation 8]

here, is the previous parameter weight of the symptom classification unit, is the learning rate that specifies the degree of update of the parameter weights, is the weight is the gradient from the loss function for , and D may be information such as data given to calculate this loss function and gradient.

The second update unit 163 may adjust the weight for one or more parameters of the symptom classification unit 210 based on the calculated weight.

In this way, the model supervised learning unit 145 of this embodiment supervised learning the pre-trained auscultation sound analysis model 200 using the learning data and the corresponding labeling data, and analyzed the auscultation sound with the final loss value determined accordingly. Model 200 can be updated. Supervised learning for this auscultation sound analysis model 200 can be repeatedly performed until the final loss value is below the set reference value.

As described above, the auscultation sound analysis model learning device 100 of this embodiment pre-trains the auscultation sound analysis model 200 using learning data generated by pre-processing the auscultation sound data, and pre-learns the auscultation sound analysis model. (200) can be supervised using learning data.

In addition, the auscultation sound analysis model learning device 100 of the present embodiment focuses on the classification of patient information included in the learning data when learning the pre-learned auscultation sound analysis model 200. The auscultation sound analysis model 200 can be updated by determining the adversarial loss value to prevent learning.

Accordingly, the present invention can narrow the data distribution gap caused by model learning in an environment with a small number of auscultation sound data, and can enable learning about symptoms rather than patient information from auscultation sound data, so that an auscultation sound analysis model ( 200) can increase learning efficiency and performance.

Figure 6 is a diagram showing a method of learning the auscultation sound analysis model according to an embodiment of the present invention, Figure 7 is a diagram showing a method of pre-training the auscultation sound analysis model of Figure 6, and Figure 8 is a diagram showing the auscultation sound analysis model of Figure 6. This is a diagram showing a method for supervised learning of an analysis model.

Referring to FIG. 6, the auscultation sound analysis model learning device 100 of this embodiment may receive auscultation sound data through the input/output unit 110. Accordingly, the processor 120 may generate learning data from auscultation sound data by executing the neural network model learning program 140 stored in the memory 130 (S10).

Here, the auscultation sound data may be a waveform signal in the form of a one-dimensional vector. Accordingly, the data preprocessing unit 141 may obtain learning data from the auscultation sound data through preprocessing to convert the auscultation sound data provided through the input/output unit 110 into a frequency-time response signal in a two-dimensional vector form.

Next, the model pre-learning unit 143 may pre-train the auscultation sound analysis model 200 using the learning data (S20).

Referring to FIG. 7, the model dictionary learning unit 143 inputs the learning data into the symptom classification unit 210 of the auscultation sound analysis model 200 to obtain the first feature vector output from the symptom classification unit 210. You can do it (S110).

Next, the feature vector masking unit 153 may mask at least one of the plurality of feature values of the obtained first feature vector to a value of 0 to generate a masking feature vector including the masked feature value (S120) ).

Next, the neural network model replication unit 151 may replicate the symptom classification unit 210 of the auscultation sound analysis model 200 (S130).

At this time, the neural network model replication unit 151 may generate a duplicate symptom classification unit by replicating the symptom classification unit 210 so that all convolutional layers of the symptom classification unit 210 are identically replicated.

Subsequently, the learning data conversion unit 152 may convert the learning data to obtain one or more converted learning data (S140).

As described above, since the learning data is a frequency-time response signal, the learning data conversion unit 152 can generate one or more converted learning data by converting the frequency component or time component of the learning data.

Next, the model dictionary learning unit 143 may input the converted learning data into the replicated symptom classification unit and obtain a second feature vector output from the replicated symptom classification unit (S150).

Meanwhile, in FIG. 7, the step of generating a masking feature vector from the first feature vector obtained through the symptom classifier 210 and then acquiring the second feature vector through the duplicate symptom classifier is shown. However, in this case, The invention is not limited thereto.

For example, in the present invention, generating a masking feature vector from the first feature vector obtained through the symptom classification unit 210 using learning data (S110, S120), and classifying replicated symptoms using the converted learning data. The steps (S130, S140, and S150) of acquiring the second feature vector through the wealth may be performed in parallel.

Next, the dictionary learning loss calculation unit 154 may determine a dictionary learning loss value for the symptom classification unit 210 of the auscultation sound analysis model 200 based on the masking feature vector and the second feature vector (S160).

Here, the dictionary learning loss calculation unit 154 may determine a dictionary learning loss value according to the distance difference between the masking feature vector and the second feature vector.

Next, the first update unit 155 may adjust the weight of one or more parameters of the symptom classification unit 210 so that the pre-learning loss value is minimized based on the determined pre-learning loss value.

And, the first update unit 155 may update the symptom classification unit 210 by adjusting one or more parameter values of the symptom classification unit 210 based on the adjusted weight (S170).

In addition, the first update unit 155 calculates an exponential moving average value for the adjusted parameter values of the symptom classification unit 210, and adjusts the weight of one or more parameters of the replicated symptom classification unit based on the calculated exponential moving average value. , the duplicated symptom classification unit can be updated.

Referring again to FIG. 6, the model supervised learning unit 145 may supervised learning the pre-trained auscultation sound analysis model 200 using the learning data and the corresponding labeling data (S30).

Referring to FIG. 8, the model map learning unit 145 uses the learning data generated in the data pre-processing unit 141 and labeling data for the learning data to determine the symptoms of the pre-trained auscultation sound analysis model 200 using actual symptom information. It can be entered into the classification unit 210.

In addition, the model guidance learning unit 145 can input actual patient information using learning data and labeling data for the learning data into the patient information classification unit 220 of the pre-trained auscultation sound analysis model 200.

Accordingly, the model guidance learning unit 145 can obtain the symptom classification result output from the symptom classification unit 210 and the patient information classification result output from the patient information classification unit 220 (S210).

Next, the supervised learning loss calculation unit 161 may compare the symptom classification result output from the symptom classification unit 210 with the labeling data, and determine the supervised learning loss value according to the comparison result (S220).

Subsequently, the adversarial loss calculation unit 162 may compare the patient information classification result output from the patient information classification unit 220 with the labeling data and determine a loss value according to the comparison result. Next, the adversarial loss calculation unit 162 may determine the adversarial loss value so that the loss value increases based on the determined loss value (S230).

Next, the second update unit 163 may determine the final loss value by adding the supervised learning loss value and the adversarial loss value (S240).

Accordingly, the second update unit 163 calculates a weight so that the determined final loss value is minimal, and adjusts the weight for one or more parameters of the symptom classification unit 210 based on the calculated weight to adjust the weight of the symptom classification unit 210. (210) can be updated (S250).

In this way, the auscultation sound analysis model learning device 100 of this embodiment pre-trains the auscultation sound analysis model 200 using learning data generated from auscultation sound data, and uses the pre-learned auscultation sound analysis model 200. Supervised learning can be done using learning data.

In addition, the auscultation sound analysis model learning device 100 of the present embodiment focuses on the classification of patient information included in the learning data when learning the pre-learned auscultation sound analysis model 200. The auscultation sound analysis model 200 can be updated by determining the adversarial loss value to prevent learning.

Accordingly, the present invention can narrow the data distribution gap caused by model learning in an environment with a small number of auscultation sound data, and can enable learning about symptoms rather than patient information from auscultation sound data, so that an auscultation sound analysis model ( 200) can increase learning efficiency and performance.

Combinations of each block of the block diagram of the present invention and each step of the flowchart described above may be performed by computer program instructions. Since these computer program instructions can be mounted on the encoding processor of a general-purpose computer, special-purpose computer, or other programmable data processing equipment, the instructions performed through the encoding processor of the computer or other programmable data processing equipment are included in each block or block of the block diagram. Each step of the flowchart creates a means to perform the functions described. These computer program instructions may also be stored in computer-usable or computer-readable memory that can be directed to a computer or other programmable data processing equipment to implement a function in a particular way, so that the computer-usable or computer-readable memory The instructions stored in can also produce manufactured items containing instruction means that perform the functions described in each block of the block diagram or each step of the flowchart. Computer program instructions can also be mounted on a computer or other programmable data processing equipment, so that a series of operational steps are performed on the computer or other programmable data processing equipment to create a process that is executed by the computer, thereby generating a process that is executed by the computer or other programmable data processing equipment. Instructions that perform processing equipment may also provide steps for executing functions described in each block of the block diagram and each step of the flowchart.

Additionally, each block or each step may represent a module, segment, or portion of code that includes one or more executable instructions for executing specified logical function(s). Additionally, it should be noted that in some alternative embodiments it is possible for the functions mentioned in blocks or steps to occur out of order. For example, it is possible that two blocks or steps shown in succession may in fact be performed substantially simultaneously, or the blocks or steps may sometimes be performed in reverse order depending on the corresponding function.

The above description is merely an illustrative explanation of the technical idea of the present invention, and those skilled in the art will be able to make various modifications and variations without departing from the essential quality of the present invention. Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but are for illustrative purposes, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention shall be interpreted in accordance with the claims below, and all technical ideas within the scope equivalent thereto shall be construed as being included in the scope of rights of the present invention.

100: Auscultation sound analysis model learning device
110: input/output unit
120: processor
130: memory
141: Data preprocessing unit
143: Model pre-training unit
145: Model supervised learning unit
200: Auscultation sound analysis model
210: Symptom classification unit
220: Patient information classification unit

Claims (15)

Obtaining learning data by preprocessing auscultation sound data;
Pre-training the auscultation sound analysis model by providing the learning data to the auscultation sound analysis model; and
A method of learning an auscultation sound analysis model comprising the step of providing actual symptom information and actual patient information to a pre-learned auscultation sound analysis model using the learning data and labeling data to supervisely train the pre-learned auscultation sound analysis model.
According to paragraph 1,
The step of pre-training the auscultation sound analysis model is,
Replicating the symptom classification part of the auscultation sound analysis model;
Obtaining converted learning data by converting the learning data;
Inputting the learning data into the symptom classification unit to obtain a first feature vector output from the symptom classification unit;
Inputting the converted learning data into a duplicated symptom classifier to obtain a second feature vector output from the duplicated symptom classifier;
determining a prior learning loss value for the symptom classifier based on the first feature vector and the second feature vector; and
A method of learning an auscultation sound analysis model comprising the step of adjusting parameter weights of the symptom classification unit so that the determined pre-learning loss value is minimized.
According to paragraph 2,
The step of pre-training the auscultation sound analysis model is,
Further comprising generating a masking feature vector by masking at least one of the plurality of feature values of the first feature vector,
The step of determining the pre-learning loss value is,
A method of training an auscultatory sound analysis model comprising determining the pre-learning loss value based on the masking feature vector and the second feature vector.
According to paragraph 2,
The step of pre-training the auscultation sound analysis model is,
Obtaining an exponential moving average value for the parameter weight of the symptom classification unit; and
A method of learning an auscultation sound analysis model further comprising adjusting parameter weights of the replicated symptom classifier based on the exponential moving average value.
According to paragraph 2,
The step of acquiring the converted learning data is,
A method of learning an auscultatory sound analysis model comprising the step of obtaining the converted learning data by converting at least one of the time component or the frequency component of the learning data.
According to paragraph 1,
The step of supervised learning the pre-learned auscultation sound analysis model is,
Providing the learning data and the actual symptom information to a symptom classification unit of the pre-learned auscultation sound analysis model to obtain a symptom classification result for the learning data from the symptom classification unit;
determining a supervised learning loss value based on the actual symptom data and the symptom classification result;
Providing the intermediate output value of the symptom classification unit and the actual patient information to a patient information classification unit of the pre-learned auscultation sound analysis model to obtain a patient information classification result for the learning data from the patient information classification unit;
determining an adversarial loss value based on the actual patient information and the patient information classification result;
Determining a final loss value for the pre-trained auscultation sound analysis model based on the supervised learning loss value and the adversarial loss value; and
A method of learning an auscultation sound analysis model comprising the step of adjusting parameter weights of the symptom classification unit so that the determined final loss value is minimized.
According to paragraph 1,
The step of acquiring the learning data is,
A method of learning an auscultation sound analysis model comprising the step of converting the auscultation sound data into a time-frequency response signal in a two-dimensional vector form to obtain the learning data.
Memory where the neural network model learning program is stored; and
Execute the neural network model learning program, pre-process the auscultation sound data to obtain learning data, provide the learning data to the auscultation sound analysis model to pre-train the auscultation sound analysis model, and apply the learning data to the pre-learned auscultation sound analysis model. An auscultatory sound analysis model learning device comprising a processor for supervised learning of the pre-learned auscultation sound analysis model by providing it to a sound analysis model.
According to clause 8,
The processor,
Duplicating the symptom classification unit of the auscultation sound analysis model, converting the learning data to obtain converted learning data, inputting the learning data into the symptom classification unit to obtain a first feature vector output from the symptom classification unit, , input the converted learning data into the replicated symptom classification unit, obtain a second feature vector output from the replicated symptom classification unit, and input the converted learning data to the symptom classification unit based on the first feature vector and the second feature vector. An auscultation sound analysis model learning device that determines a pre-learning loss value for and adjusts parameter weights of the symptom classification unit so that the determined pre-learning loss value is minimized.
According to clause 9,
The processor,
Learning an auscultation sound analysis model that generates a masking feature vector by masking at least one of the plurality of feature values of the first feature vector and determines the pre-learning loss value based on the masking feature vector and the second feature vector. Device.
According to clause 9,
The processor,
An auscultation sound analysis model learning device for obtaining an exponential moving average value for the parameter weight of the symptom classification unit and adjusting the parameter weight of the replicated symptom classification unit based on the exponential moving average value.
According to clause 9,
The processor,
An auscultation sound analysis model learning device that obtains the converted learning data by converting at least one of the time component or the frequency component of the learning data.
According to clause 8,
The processor,
The learning data and the actual symptom information are provided to the symptom classification unit of the pre-learned auscultation sound analysis model to obtain a symptom classification result for the learning data from the symptom classification unit, and the actual symptom data and the symptom classification result are obtained. Based on this, a supervised learning loss value is determined, and the intermediate output value of the symptom classification unit and the actual patient information are provided to the patient information classification unit of the pre-learned auscultation sound analysis model, and patient information for the learning data is collected from the patient information classification unit. Obtain a classification result, determine an adversarial loss value based on the actual patient information and the patient information classification result, and finalize the pre-trained auscultation sound analysis model based on the supervised learning loss value and the adversarial loss value. An auscultatory sound analysis model learning device that determines a loss value and adjusts parameter weights of the symptom classification unit so that the determined final loss value is minimized.
According to paragraph 1,
The processor,
An auscultation sound analysis model learning device that obtains the learning data by converting the auscultation sound data into a time-frequency response signal in a two-dimensional vector form.
A computer program stored on a computer-readable recording medium,
The computer program is,
Obtaining learning data by preprocessing auscultation sound data;
Pre-training the auscultation sound analysis model by providing the learning data to the auscultation sound analysis model; and
A method of learning an auscultation sound analysis model comprising the step of providing actual symptom information and actual patient information to a pre-learned auscultation sound analysis model using the learning data and labeling data and supervised learning of the pre-learned auscultation sound analysis model. A computer program stored on a recording medium that contains instructions for execution by a processor.