KR20230034762A - Machine learning training method and apparatus using same - Google Patents

Abstract

The present invention relates to a method for training a machine learning model and an apparatus for the same. The purpose of the present invention is to provide an efficient learning/training method for improving performance of artificial neural network. The method includes: a step of obtaining each first weighted value set of a medical machine learning model based on medical data of each group, from the medical data of multiple groups; a step of estimating the performance of the medical machine learning model to which the each first weighed value set is applied, based on multiple feature estimation methods; a step of selecting the best first weighted value set with the highest performance estimated among the multiple first weighted value sets by each feature estimation method one by one; and a step of applying a second weighted value set to the medical machine learning model, wherein the multiple best first weighted value sets are connected to the second weighted value set.

Description

Machine learning method and apparatus {MACHINE LEARNING TRAINING METHOD AND APPARATUS USING SAME}

The present invention relates to a machine learning method and an apparatus using the same. More specifically, the present invention relates to machine learning (eg, deep learning) of an artificial neural network for use for medical purposes and an apparatus using the same.

In the field of artificial intelligence and big data medical devices, the importance of not only diagnosis and treatment of diseases, but also prevention and proactive management is expanding as the burden of medical expenses has soared worldwide due to the aging of the population and the increase in chronic diseases. Recently, due to the development of technologies such as artificial intelligence/big data, the health care paradigm is changing in the direction of not only predicting and preventing diseases in advance, but also precisely diagnosing and treating diseases early and effectively, and operating a continuous follow-up management system to prevent recurrence. there is. Accordingly, interest in medical technology that not only provides medical services regardless of time and space but also implements optimized personalized health management and customized treatment based on data analysis is increasing.

An object of the present invention is to provide an effective machine learning method.

More specifically, an object of the present invention is to provide an efficient learning/training method for improving the performance of an artificial neural network for use for medical purposes.

The characteristic configuration of the present invention for achieving the object of the present invention as described above and realizing the characteristic effects of the present invention described later is as follows.

According to one aspect of the present invention, in a method used by a computing device supporting machine learning, each of a medical machine learning model is trained based on a plurality of groups of medical data and trained based on each group of medical data. obtaining a first set of weights of ; evaluating performance of the medical machine learning model to which each of the first weight sets is applied based on a plurality of characteristic evaluation methods; selecting a best first weight set having the highest performance among the plurality of first weight sets for each characteristic evaluation method; and applying a second weight set provided based on the plurality of first best weight sets to the medical machine learning model.

According to another aspect of the present invention, there is provided a computer program stored in a medium, including instructions implemented to cause a computing device to perform the following method. Here, the following method includes: acquiring a first weight set for each medical machine learning model through training based on the medical data of each group, targeting a plurality of groups of medical data; evaluating performance of the medical machine learning model to which each of the first weight sets is applied based on a plurality of characteristic evaluation methods; selecting a best first weight set having the highest performance among the plurality of first weight sets for each characteristic evaluation method; and applying a second weight set provided based on the plurality of first best weight sets to the medical machine learning model.

According to another aspect of the present invention, in a computing device supporting machine learning, the communication unit for obtaining medical data; and a processor connected to the communication unit, wherein the processor acquires a first weight set for a medical machine learning model through training based on the medical data of each group, targeting a plurality of groups of medical data; The performance of the medical machine learning model to which each of the first weight sets is applied is evaluated based on a plurality of characteristic evaluation methods, and the first best performance evaluated the highest among the plurality of first weight sets for each characteristic evaluation method. A computing device configured to select weight sets one by one and apply a second weight set provided based on the plurality of first best weight sets to the medical machine learning model.

Preferably, the method may include analyzing medical data based on the medical machine learning model to which the second weight set is applied.

Preferably, the number of the plurality of best first weight sets may be smaller than the number of the first weight sets.

Preferably, the plurality of characteristic evaluation methods include (1) a first characteristic evaluation method for evaluating a characteristic based on a vital sign unit, and (2) a second characteristic evaluation method for evaluating a characteristic based on a patient unit. can

Preferably, the first characteristic evaluation method comprises evaluating event occurrence prediction within a predetermined time in the future as a unit based on the time point at which the vital sign is measured, and the second characteristic evaluation method includes evaluating event occurrence prediction based on the event occurrence time point. , it may include evaluating an event occurrence prediction within the event occurrence time from a predetermined time ago.

Preferably, the second set of weights may include:

- a set of average weights obtained from said plurality of first sets of best weights;

- a first best weight set most selected from among the plurality of first best weight sets;

- a weight set combined based on a selected weight of each first best weight set within said plurality of first best weight sets, or

- a weight set derived by learning the plurality of first best weight sets.

The present invention can provide an effective machine learning method.

More specifically, the present invention can provide an efficient learning/training method for improving the performance of an artificial neural network for use for medical purposes.

The following drawings attached to be used for description of the embodiments of the present invention are only some of the embodiments of the present invention, and those skilled in the art to which the present invention pertains based on these drawings without effort leading to a separate invention. Thus, other figures can be obtained.
1 illustrates a machine learning process.
Figure 2 illustrates a data set and unit of learning.
3 illustrates a stochastic weighted averaging method.
4 illustrates a machine learning process according to an example of the present invention.
5 illustrates a performance evaluation method.
6 illustrates a machine learning process according to an example of the present invention.
7 illustrates a computing device that may be applied to the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following detailed description of the present invention refers to the accompanying drawings, which illustrate specific embodiments in which the present invention may be practiced in order to make the objects, technical solutions and advantages of the present invention clear. These embodiments are described in sufficient detail to enable a person skilled in the art to practice the present invention.

As used throughout the description and claims herein, the terms "image" or "image data" refer to multidimensional data composed of discrete image elements (e.g., pixels for a two-dimensional image or voxels for a three-dimensional image). refers to

For example, "image" may mean a two-dimensional image corresponding to a slide of a predetermined tissue observed using a microscope, but "image" is not limited thereto, and computerized (cone-beam) It may be a medical image of a subject collected by computed tomography, magnetic resonance imaging (MRI), ultrasound, or any other medical imaging system known in the art. Images may also be provided in a non-medical context, for example, a remote sensing system, electron microscopy, and the like.

Throughout the description and claims herein, 'image' is a term that refers to a visible image or a digital representation of an image (eg, displayed on a screen).

In the drawings presented for convenience of explanation, slide image data is illustrated as an exemplary image format. However, those skilled in the art know that the image formats used in various embodiments of the present invention are X-ray images, MRI, CT, PET (positron emission tomography), PET-CT, SPECT, SPECT-CT, MR-PET, 3D ultrasound images, etc. It will be appreciated that it includes but is not limited to the illustratively enumerated forms.

The medical images described throughout the detailed description and claims of this specification may conform to the 'Digital Imaging and Communications in Medicine (DICOM)' standard. The DICOM standard is a term that collectively refers to various standards used for digital image expression and communication in medical devices, and the DICOM standard is announced by an association committee formed by the American College of Radiology (ACR) and the American Electrical Manufacturers Association (NEMA).

In addition, the medical images described throughout the detailed description and claims of this specification may be stored or transmitted through a 'Picture Archiving and Communication System (PACS)', which is a DICOM standard. It may be a system that stores, processes, and transmits medical images according to the user's needs. Medical images obtained using digital medical imaging equipment such as X-ray, CT, and MRI are stored in DICOM format and can be transmitted to terminals inside and outside the hospital through a network, to which observation results and medical records can be added. .

And throughout the detailed description and claims of this specification, 'learning' or 'learning' is a term that refers to performing machine learning through procedural computing, and is intended to refer to mental operations such as human educational activities. , and training is used in the generally accepted sense of machine learning. For example, 'deep learning' refers to machine learning using deep artificial neural networks. A deep neural network automatically learns the characteristics of each data by learning a large amount of data in a structure composed of multi-layer artificial neural networks, and through this, a machine that proceeds with learning in a way that minimizes the error of the objective/loss function, that is, classification accuracy. It is a learning model and can extract and classify various levels of features, from low-level features such as points, lines, and planes to complex and meaningful high-level features.

And throughout the description and claims herein, the word 'comprise' and variations thereof are not intended to exclude other technical features, additions, components or steps. In addition, 'one' or 'one' is used to mean more than one, and 'another' is limited to at least two or more.

Other objects, advantages and characteristics of the present invention will appear to those skilled in the art, in part from this specification and in part from practice of the invention. The examples and drawings below are provided as examples and are not intended to limit the invention. Accordingly, details disclosed herein with respect to a particular structure or function are not to be construed in a limiting sense, but are merely representative and provide guidance for those skilled in the art to variously practice the present invention with any detailed structures substantially suitable. It should be interpreted as basic data.

Moreover, the present invention covers all possible combinations of the embodiments presented herein. It should be understood that the various embodiments of the present invention are different from each other but are not necessarily mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in one embodiment in another embodiment without departing from the spirit and scope of the invention. Additionally, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the detailed description set forth below is not to be taken in a limiting sense, and the scope of the present invention, if properly described, is limited only by the appended claims, along with all equivalents as claimed by those claims. Like reference numbers in the drawings indicate the same or similar function throughout the various aspects.

In this specification, unless otherwise indicated or clearly contradicted by context, terms referred to in the singular encompass the plural unless the context requires otherwise. In addition, in describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present invention, the detailed description will be omitted.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily practice the present invention.

1 illustrates a machine learning process. In general, the machine learning process can proceed as follows: (1) explore and preprocess data, (2) construct a training data set, (3) construct a model by selecting an algorithm suitable for the problem to be solved, (4) ) train the model with the training data set, and (5) validate the model with the test data set.

Here, learning a model with a training data set includes calculating parameters (eg, weights) of each layer constituting an artificial neural network (simply, a neural network). Referring to FIG. 1 , predicted values of the neural network are compared with actual values (eg, labels), and model learning may proceed by updating the weights of the neural network in a direction that minimizes an error between the predicted values and the actual values. The error between the predicted value and the actual value is calculated using a loss function, and weights may be updated through a backpropagation process based on the loss function. The back-propagation algorithm used in neural networks is divided into a forward pass that goes through the process of calculating the weights of each layer from input to output, and a backward pass that goes back in reverse to the forward pass and goes through the calculation process again to correct the existing weights. . When the forward/backward pass process is completed for the entire data set, it can be said that the training of the model is finished, and the model can be verified using the weights of each layer calculated up to this point (hereinafter referred to as weight sets). there is.

Figure 2 illustrates a data set and unit of learning.

Referring to FIG. 2 , a learning unit of a data set may be divided into an epoch and a batch. Epoch refers to the number of times the entire training data set passes through the neural network (i.e., the number of learnings). For example, 1-epoch may mean that the forward/backward pass process is completed once for the entire training data set in FIG. 1 . On the other hand, as the amount of data used in machine learning becomes vast, several problems may arise if all data sets are used for one training session. For example, when training is performed at one time using hundreds of thousands of medical image data, considerable hardware performance (eg, memory, computing power) is required due to the data size, and learning may take too long. Accordingly, the learning units of the data set can be divided into batches. That is, since it is difficult to complete the learning at once, the computing power and time required for learning the neural network can be reduced by dividing the data set and learning. Weight update may be performed once in batch units. A batch can be understood as defining the number of data to be used for one weight update. The number of batches required to complete a 1-epoch is called an iteration. That is, iteration means the number of weight updates required to complete 1-epoch. For example, if the iteration is 5 and training is performed by extracting 20 batches from the training data, this can be understood as 4-epoch, that is, machine learning was performed using the entire training data 4 times.

3 illustrates machine learning based on the stochastic weighted averaging method.

Referring to FIG. 3 , in the process of learning one model, the neural network may be learned step by step. Here, the step corresponds to the arrangement described in FIG. 2, and hereinafter, the arrangement may be used interchangeably with the step. At each step, a randomized portion (eg, batch) of training data can be used to train the neural network. Parameter sets (eg, weight sets) may be calculated/updated for each step. The stochastic weight averaging method stores the weight sets learned in each step in the process of learning a model, and averages the weight sets learned in each step in the final step. In the case of big data, it is practically impossible to train a neural network using all the data because it generally includes at least 10 million data and in some cases more than 100 million data. Instead, it would be possible to obtain a weight set that shows performance/distribution similar to that of the weight set (see the asterisk in the loss function graph) as when learning using the entire data set by calculating the weight average using randomly extracted data for each step. can

On the other hand, as machine learning using artificial neural networks develops, technologies for early prediction of severe deterioration of patients' conditions, such as cardiac arrest and death, are being researched and developed. However, the definition of which method/unit to verify the performance of the developed predictive model is clinically correct varies from study to study. For example, you can evaluate the performance of a predictive model on a per-vital-sign or per-patient basis. However, when the present inventors develop a model by defining an evaluation method based on patient unit performance, the vital sign unit performance tends to be low, and conversely, when a model is developed using a vital sign unit performance evaluation method, patient unit performance tends to be low. confirmed that

In addition, in the case of machine learning models, prediction performance tends to be low when data from other distributions in addition to the training data distribution come in. Therefore, when learning a neural network by dividing training data into stages (eg, batches), it is preferable to perform machine learning based on the distribution of learning data that can improve the performance of a predictive model. In this regard, the stochastic weight averaging method of FIG. 3 stores the weights learned in each step in the process of learning a model and averages them in the final step. There are no clear criteria for this.

Therefore, the present invention proposes a method for developing robust models in various performance evaluation techniques and various data distributions. Specifically, the present invention proposes a method of providing final weights (eg, average weights) based on the stored weights after storing weights that yield high performance in various evaluation methods.

According to the present invention, models with high predictive performance in various institutions and patients and high performance in various clinical evaluation methods can be learned. In addition, prediction models with high performance in various distributions may be developed by weights provided based on weights that exhibit high performance in various evaluation methods. That is, in the prior art, as a model is selected/generated based on one evaluation method, a model that is selectively robust only for a specific evaluation method and data distribution can be developed. On the other hand, in the present invention, since models are selected/created based on various evaluation methods, a model robust to various evaluation methods and data distribution can be developed.

4 illustrates a machine learning process according to an example of the present invention. Referring to FIG. 4 , machine learning using an artificial neural network may be performed by dividing entire training data into stages. Here, an artificial neural network may be used to learn a medical model. The medical model may include various models used for medical purposes, such as a medical prediction model and a medical diagnosis model. For example, a medical model analyzes various medical data such as CT images, X-ray images, MR images, and electronic health record (EHR) analysis, and based on this, an early prediction/diagnosis model for the symptom of interest is created. can include Specifically, the medical model may include a medical prediction model for early prediction of severe deterioration of a patient's condition, such as cardiac arrest or death. In addition, the medical model may include a medical diagnosis model for diagnosing early dementia based on brain images.

In the present invention, a model for learning a neural network is learned in stages, and parameters (eg, a weight set) learned in stages in the process of learning a model are stored. Referring to the figure, n number of steps are performed, and as a result, n weight sets (hereinafter referred to as first weight sets) may be calculated and stored. Thereafter, by using an evaluation method for evaluating the performance of the model, a set of weights with high performance in each step may be determined for each evaluation method. The evaluation method includes various evaluation methods including at least one of a vital sign unit evaluation method and a patient unit evaluation method, and may preferably include at least both a vital sign unit evaluation method and a patient unit evaluation method. In the figure, n evaluation methods were used to evaluate model performance. At this time, for evaluation method 2, the highest performance was obtained when the weight set of step 1 was used. In addition, when the weight set of step n was used for evaluation method 3, the highest performance was obtained when the weight set of step 2 was used for evaluation method n. For other evaluation methods, the set of weights of the step exhibiting the highest performance can be determined in the same manner.

Thereafter, a new single weight set (hereinafter referred to as the second weight set) is obtained based on the weight set (hereinafter referred to as the first best weight set) that exhibits high performance in various evaluation methods, and each layer of the neural network uses the second weight set. By updating it, you can end neural network learning for the model. Here, the second weight set may be provided by various combinations or by applying various methods based on the first best weight sets. Although not limited to this, the new single weight set may include:

- an average weight set obtained from said plurality of first best weight sets;

- a first best weight set most selected from among the plurality of first best weight sets;

- a weight set combined based on a selected weight of each first best weight set within said plurality of first best weight sets, or

- a weight set derived by learning the plurality of first best weight sets.

According to the present invention, after storing weight sets (ie, the first best weight set(s)) that exhibit high performance in various evaluation methods, a new weight set (ie, the second best weight set(s)) is based on the stored weight sets. For example, by providing a set of average weights), a medical artificial neural network prediction model can be developed that not only achieves high performance in various performance evaluation techniques, but also performs well in various distributions.

5 illustrates an evaluation method that can be applied to the present invention. 5 illustrates a vital sign unit evaluation method and a patient unit evaluation method.

Referring to FIG. 5, the vital sign unit evaluation method predicts/diagnostic performance of the subject's symptom of interest based on the subject's vital signs, eg, measured values of body temperature, respiratory rate, pulse, blood pressure (diastolic, systolic), etc. means how to evaluate For example, model performance may be evaluated/verified by evaluating whether an event is predicted within n hours from the time point at which an active sign is measured. Measurement results can be classified into TP (True Positive), FP (False Positive), TN (True Negative), and FN (False Negative). Based on the measurement results, the model performance is evaluated by AUROC (Area Under the Receiver Operating Characteristic), AUPRC (Area Under the Precision-Recall Curve), sensitivity, specificity, TPR (True Positive Rate), FPR (False Positive Rate), PPV (Positive Predictive Value), NPV (Negative Predictive Value), f-measure, etc. More specifically, events (e.g., cardiac arrest) within the next 24 hours are based on vital signs (i.e., body temperature, respiratory rate, pulse, and blood pressure (diastolic, systolic)) collected from the subject's Electronic Medical Record (EMR). ) can be predicted.

On the other hand, in the patient unit evaluation method, model performance can be evaluated/verified by considering the event to be predicted if an alarm sounds even once within n hours before the event occurrence to within the event occurrence time based on the patient's event occurrence time. In the case of normal data, specificity can be evaluated at each time point at which vital signs are measured. Model performance can be evaluated/validated using AUROC or AUPRC.

In addition to the illustrated evaluation method, the following evaluation method may be defined. In addition, various definitions are possible by adjusting the event window time (n) in FIG. 5 .

- For the same patient, if the time difference between alarms is small, consider it as one alarm and evaluate AUROC/AUPRC (per patient and/or per vital sign)

- AUROC/AUPRC assessment (per patient and/or per vital signs) within the clinically valid false positive rate (FPR) range

6 illustrates a machine learning process according to an example of the present invention. 6 corresponds to the content described in FIG. 4 . Referring to FIG. 6 , after defining a model for learning a neural network, learning may be performed step by step on the one model. Assuming that seven steps are performed, parameters (eg, weight sets) may be calculated for each step (W ₁ to W ₇ ). Thereafter, a new weight set (eg, weight average) may be obtained by focusing on weight sets that exhibit high performance in various evaluation methods. In the figure, it is assumed that the weight set of steps 3/6/1/3 shows the highest performance for evaluation methods 1 to 4 (W ₃ , W ₆ , W ₁ , W ₃ ). Here, the evaluation method may refer to the description of FIGS. 4 and 5, and preferably may include both a vital sign unit evaluation method and a patient unit evaluation method. Although not limited thereto in the present invention, the number of evaluation methods may be smaller than the number of steps.

After that, according to the proposal of the present invention, a weight average is applied to the weight set that shows high performance in each evaluation method. In the case of FIG. 6 , a weight average (W _avg ) can be obtained by averaging the weight sets (W ₃ , W ₆ , W ₁ , and W ₃ ) of steps 3/6/1/3. Thereafter, by updating each layer of the neural network using a weighted average (W _avg ), neural network learning for the corresponding model may be terminated. Thereafter, the trained neural network may be used to provide early prediction/diagnosis information of a symptom/disease of interest by analyzing medical data (eg, vital signs, images, EMR) according to the applied medical model.

7 illustrates a computing device according to one example of the present invention. The computing device 200 according to an example of the present invention includes a communication unit 210 and a processor 220, and may directly or indirectly communicate with an external computing device (not shown) through the communication unit 210.

Specifically, computing device 200 may include typical computer hardware (e.g., computer processors, memory, storage, input and output devices, and other devices that may include components of conventional computing devices; electronic devices such as routers, switches, and the like). communication devices; electronic information storage systems such as network-attached storage (NAS) and storage area networks (SAN)) and computer software (i.e., instructions that cause a computing device to function in a particular way); s) may be used to achieve desired system performance.

The communication unit 210 of the computing device may transmit/receive requests and responses with other computing devices that are interlocked. As an example, such requests and responses may be made through the same transmission control protocol (TCP) session. It is not limited, and may be transmitted and received as, for example, a user datagram protocol (UDP) datagram. In addition, in a broad sense, the communication unit 210 may include a pointing device such as a keyboard or mouse for receiving commands or instructions, other external input devices, printers, displays, and other external output devices.

In addition, the processor 220 of the computing device may include a micro processing unit (MPU), a central processing unit (CPU), a graphics processing unit (GPU) or a tensor processing unit (TPU), a cache memory, and a data bus. ) may include hardware configurations such as In addition, it may further include a software configuration of an operating system and an application that performs a specific purpose.

The present invention illustrated with reference to FIGS. 4 to 6 may be configured based on hardware/software, and the processor 220 of the computing device may be configured to perform/control the operation of the present invention according to FIGS. 4 to 6 .

Based on the description of the above embodiments, a person skilled in the art can understand that the methods and/or processes of the present invention, and the steps thereof, can be realized with hardware, software, or any combination of hardware and software suitable for a particular application. point can be clearly understood. The hardware may include a general purpose computer and/or a dedicated computing device or a specific computing device or a particular aspect or component of a specific computing device. The processes may be realized by one or more microprocessors, microcontrollers, embedded microcontrollers, programmable digital signal processors or other programmable devices having internal and/or external memory. Additionally, or alternatively, the processes may be configured to process application specific integrated circuit (ASIC), programmable gate array, programmable array logic (PAL) or electronic signals. can be implemented with any other device or combination of devices. Furthermore, the subject matter of the technical solution of the present invention or parts contributing to the prior art may be implemented in the form of program instructions that can be executed through various computer components and recorded on a machine-observable recording medium. The machine-observable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the machine-observable recording medium may be specially designed and configured for the present invention, or may be known and usable to those skilled in the art in the field of computer software. Examples of machine-observable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROM, DVD, and Blu-ray, and magneto-optical media such as floptical disks. (magneto-optical media), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include stored and compiled or interpreted for execution on any one of the foregoing devices, as well as a heterogeneous combination of processors, processor architectures, or different combinations of hardware and software, or any other machine capable of executing program instructions. Machine code, This includes not only bytecode, but also high-level language code that can be executed by a computer using an interpreter or the like.

Accordingly, in one aspect according to the present specification, when the methods and combinations thereof described above are performed by one or more computing devices, the methods and combinations of methods may be implemented as executable code that performs each step. In another aspect, the method may be implemented as systems performing the steps, the methods may be distributed in several ways across devices or all functions may be integrated into one dedicated, stand-alone device or other hardware. In another aspect, the means for performing the steps associated with the processes described above may include any of the hardware and/or software described above. All such sequential combinations and combinations are intended to fall within the scope of this disclosure.

For example, the hardware device may be configured to act as one or more software modules to perform processing according to the present disclosure, and vice versa. The hardware device may include a processor such as an MPU, CPU, GPU, TPU coupled to a memory such as ROM/RAM for storing program instructions and configured to execute instructions stored in the memory, and external devices and signals It may include a communication unit capable of sending and receiving. In addition, the hardware device may include a keyboard, mouse, and other external input devices for receiving commands written by developers.

In the above, the present invention has been described by specific details such as specific components and limited embodiments and drawings, but these are provided to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments. , Various modifications and variations can be made from these descriptions by those of ordinary skill in the art to which the present invention belongs.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments and should not be determined, and not only the claims to be described later, but also all modifications equivalent or equivalent to these claims fall within the scope of the spirit of the present invention. will do it

Such equivalent or equivalent modifications will include, for example, logically equivalent methods that can produce the same results as those performed by the method according to the present specification, the spirit and scope of the present invention. should not be limited by the above examples, and should be understood in the broadest sense permitted by law.

Claims (8)

A method used by a computing device supporting machine learning, comprising:
acquiring first weight sets for medical machine learning models through training based on the medical data of each group, targeting a plurality of groups of medical data;
evaluating performance of the medical machine learning model to which each of the first weight sets is applied based on a plurality of characteristic evaluation methods;
selecting a best first weight set having the highest performance among the plurality of first weight sets for each characteristic evaluation method; and
and applying a second set of weights provided based on the plurality of first sets of best weights to the medical machine learning model. According to claim 1,
and analyzing medical data based on the medical machine learning model to which the second weight set is applied. According to claim 1,
The number of the plurality of best first weight sets is less than the number of the first weight sets. According to claim 1,
The method of claim 1 , wherein the plurality of characterization methods include (1) a first characterization method that evaluates a characteristic based on a vital sign unit, and (2) a second characterization method that evaluates a characteristic based on a patient unit. According to claim 4,
The first characteristic evaluation method includes evaluating event occurrence predictions within a predetermined time in the future as a unit based on the time point at which vital signs are measured,
The second characteristic evaluation method includes evaluating an event occurrence prediction within the event occurrence time point from a predetermined time prior to the event occurrence time point based on the event occurrence time point. According to claim 1,
wherein the second set of weights comprises:
- a set of average weights obtained from said plurality of first sets of best weights;
- a first best weight set most selected from among the plurality of first best weight sets;
- a weight set combined based on a selected weight of each first best weight set within said plurality of first best weight sets, or
- a weight set derived by learning the plurality of first best weight sets. A computer program stored on a medium comprising instructions implemented to cause a computing device to perform the method of any one of claims 1 to 6. A computing device supporting machine learning,
a communication unit for obtaining medical data; and
Includes a processor connected to the communication unit, wherein the processor
For the plurality of groups of medical data, obtain respective first weight sets for the medical machine learning model through learning based on the medical data of each group;
Evaluating the performance of the medical machine learning model to which each of the first weight sets is applied based on a plurality of characteristic evaluation methods;
Selecting a best first weight set having the highest performance among the plurality of first weight sets for each characteristic evaluation method, and
A computing device configured to apply a second set of weights provided based on the plurality of first sets of best weights to the medical machine learning model.

Images