KR20200092805A - Method and system for providing customized diagnosis system - Google Patents

Abstract

Disclosed are a method for providing a customized diagnostic system and a system thereof. The method of providing a diagnostic system to achieve an above technical task includes the steps of: transmitting M pieces of customized training data which are not annotated to a first diagnostic system by a reference diagnostic system, wherein M is an integer, 2 or higher; allowing the reference diagnostic system to receive the M pieces of customized training data which are annotated by a first diagnostic side from the first diagnostic system; and reflecting the received M pieces of annotated customized training data to the reference diagnostic system, wherein the M pieces of customized training data annotated by the first diagnostic side are reflected to a neural network for the reference diagnostic system under a condition of receiving a predetermined price.

Description

Method and system for providing customized diagnosis system

The present invention relates to a method and system for diagnosing a disease through learning. More specifically, the present invention relates to a method and system for implementing a customized diagnostic system reflecting a diagnostic tendency of a specific diagnoser using a diagnostic system learned based on supervised learning through a neural network.

One of the main tasks performed by the pathology department or the pathology department is to perform a diagnosis by reading a patient's biological image and determining a condition or symptom for a specific disease. This diagnosis is dependent on the experience and knowledge of a long-time skilled healthcare practitioner.

Recently, due to the development of machine learning, attempts have been made to automate tasks such as recognizing or classifying images by computer systems. In particular, attempts have been made to automate diagnostics performed by skilled medical personnel using a neural network (a deep learning method using a convolutional neurla network (CNN)), which is a kind of machine learning.

In particular, the diagnosis through deep learning using a neural network (e.g., CNN) is not simply automating the experience and knowledge of a skilled medical practitioner, but deriving the desired solution by finding characteristic elements through learning by themselves. On the contrary, in some cases, the characteristics of disease factors that an experienced medical practitioner did not know may be found in images.

In general, diagnosis of a disease through a neural network using biometric data (e.g., biometric image) is performed by an expert trained in biometric data annotating the condition of a specific disease (e.g., whether or not cancer has been expressed), and such annotated The neural network is trained by using a large number of data as training data. That is, learning through annotated learning data is mainly used by annotating learning data for learning, and this learning method is called supervised learning.

However, in supervised learning, the performance or propensity of the learned system is dominated by the annotator's judgment propensity or tendency to perform annotations. That is, the diagnostic result of the learned diagnostic system is dependent on the annotation tendency or tendency of the annotator.

However, it is a reality that biometric data that is difficult to be clearly classified exists in the biometric data to be learned so that different annotations are performed depending on the annotator when performing the actual annotation. That is, even if an experienced medical practitioner performs an annotation, opinions may differ in classifying whether the disease is expressed or the degree of disease progression according to the biological data.

Nevertheless, in the case of a diagnosis system for diseases learned based on supervised learning, there is a vulnerability in that a judgment dependent on the tendency of the annotator is output as a diagnosis result.

In addition, in order to construct a diagnostic system using a neural network, a large number of training data and an experienced behavior by an experienced healthcare professional for annotating such a plurality of training data is required, which requires a lot of time and money.

Therefore, there is a practical difficulty for a small medical institution to construct a diagnostic system using the learned neural network. In addition, even if a neural network learned by another institution is purchased or rented, as described above, since the neural network is dependent on annotation, it is difficult to construct a diagnostic system reflecting its own diagnostic tendency.

Therefore, a technical idea that can solve these problems is required.

Korean Registered Patent 10-20170057399 "Disease diagnosis system through modular reinforced learning"

The technical problem to be achieved by the present invention is to provide a method and a system for providing a diagnostic system having a customized tendency to a diagnostician (diagnostic organ).

In addition, it is to provide a method and system for performing diagnosis using the customized diagnostic system.

In addition, a plurality of diagnostic systems based on supervised learning learned by different annotation subjects are provided, and final diagnostic results, that is, between diagnostic systems, are based on the diagnostic results performed by the plurality of diagnostic systems on predetermined biological data. It is to provide a method and system that can compensate for weaknesses based on supervised learning by trying to derive agreed diagnosis results.

It is also to provide a method and a system that can improve the performance of individual diagnostic systems by enabling re-learning of individual diagnostic systems using diagnostic results through the consensus of these diagnostic systems.

A method of providing a diagnostic system for achieving the above technical problem is that the reference diagnostic system transmits M (M is an integer of 2 or more) unannotated customizing learning data to the first diagnostic system, and the reference diagnostic system provides Receiving from the first diagnostic system the M customized training data annotated by the first diagnostic side, and reflecting the received received M customized training data to the reference diagnostic system; , Characterized in that the M pieces of learning data for customization, which are annotated by the first diagnosis side, are reflected in the neural network for the reference diagnostic system under the condition that a predetermined price is paid.

The method for providing a customized diagnostic system comprises the steps of receiving, by the reference diagnostic system, the diagnostic data reflecting the annotated M customizing learning data, receiving diagnostic data from the first diagnostic system, generating diagnostic results, and transmitting the diagnostic results to the first diagnostic system. It can contain.

The reference diagnostic system may be characterized in that the larger the M pieces or the larger the cost is, depending on the weight of the M pieces compared to the number of learning data learned by the reference diagnostic system.

The reference diagnostic system may be a system that includes a set of individual diagnostic systems including a plurality of individual diagnostic systems, and performs diagnosis through consensus based on individual diagnosis results output by the plurality of individual diagnostic systems.

Reflecting at least some of the plurality of individual diagnostic systems using the annotated M customizing learning data is a step of reflecting the received received M customizing learning data to the reference diagnostic system. It can contain.

The M customizing learning data may include data in which at least two of the individual diagnostic systems included in the individual diagnostic system set output different diagnostic results.

A method for providing a diagnostic system for solving the above technical problem is a reference diagnostic system-the reference diagnostic system includes a set of individual diagnostic systems including a plurality of individual diagnostic systems-M (M is an integer of 2 or more) not annotated Transmitting customizing learning data to a first diagnostic system, receiving, by the reference diagnostic system, the M customizing learning data annotated by the first diagnostic side from the first diagnostic system; Reflecting the annotated M customizing learning data to any one first diagnostic system included in the reference diagnostic system set, and a predetermined diagnostic request for a diagnostic request including diagnostic data to be diagnosed by the reference diagnostic system. Step of receiving from the system, the type of the diagnostic result corresponding to the diagnostic request is determined as the first individual diagnostic result of the first individual diagnostic system according to the received diagnostic request, and the first individual diagnostic system inputs the input data And transmitting a diagnostic result including the first individual diagnostic result received and output to the diagnostic request system.

The above method can be implemented by a computer program installed in a data processing apparatus and stored in a computer-readable recording medium.

The system for solving the above technical problem includes a processor and a storage device in which a program executed by the processor is stored, the storage device includes at least one neural network, and the program is M (M is an integer of 2 or more). Transmits two non-annotated customization learning data to a first diagnostic system, receives the M customized learning data annotated by the first diagnostic system side from the first diagnostic system, and receives the received annotated learning data It is characterized in that the M customizing learning data is reflected in the reference diagnostic system, and the M customizing learning data annotated by the first diagnosis side is reflected in the neural network as a condition of receiving a predetermined price. .

A system for solving the above technical problem includes a processor and a storage device in which programs executed by the processor are stored, and the storage device includes a plurality of neural networks corresponding to a plurality of individual diagnostic systems, and the program M (M is an integer greater than or equal to 2) transmits the unannotated customizing learning data to the first diagnostic system, and the M customizing learning data annotated by the first diagnostic system from the first diagnostic system. Upon receiving, the received M customized learning data for customization is reflected in any one first neural network included in the neural network for the plurality of individual diagnostic systems, and a diagnostic request including diagnostic data to be diagnosed is received. Upon reception from a predetermined diagnostic request system, the type of the diagnostic result corresponding to the diagnostic request is determined as the first individual diagnostic result of the first individual diagnostic system according to the received diagnostic request, and the first individual diagnostic system inputs the The diagnostic results including the first individual diagnostic results received and outputted data are transmitted to the diagnostic request system.

According to the technical idea of the present invention, there is an effect that it is possible to easily construct a customized diagnostic system reflecting the propensity of a specific diagnoser using a reference diagnostic system.

In addition, by providing such a customized diagnostic system, there is an effect that can generate new revenue.

In addition, there is an effect of supplementing the dependency of annotation of diagnosis results that may occur when diagnosing a disease through a supervised learning-based diagnosis system learned using learning data annotated by an annotation subject.

In addition, according to the technical idea of the present invention, individual diagnostic systems may be re-learned by using the diagnosis result through consensus, and in this case, the performance of individual diagnostic systems may also be improved.

BRIEF DESCRIPTION OF THE DRAWINGS In order to better understand the drawings cited in the detailed description of the present invention, a brief description of each drawing is provided.
1 is a view for explaining a schematic concept for implementing a method for providing a customized diagnostic system according to an embodiment of the present invention.
2 is a diagram for illustratively explaining an example of implementation of a reference diagnostic system according to an embodiment of the present invention.
3 is a view for explaining the concept of customizing a reference diagnostic system according to an embodiment of the present invention.
4 is a diagram illustrating a schematic system configuration when a reference diagnostic system according to an embodiment of the present invention performs a supervised learning-based consensus diagnosis method.
5 is a view for explaining the concept of using a customized diagnostic system according to an embodiment of the present invention.
6 is a view for explaining the schematic configuration of each of the diagnostic systems according to an embodiment of the present invention.
7 is a view for explaining the concept of deriving a consensus based on the individual diagnostic results, the reference diagnostic system according to an embodiment of the present invention.
8 is a view for explaining the concept of divided annotation of learning data according to an embodiment of the present invention.
9 is a diagram for explaining the concept of re-learning an individual diagnostic system using a consensus diagnosis result according to an embodiment of the present invention.

The present invention can be applied to a variety of transformations and may have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all conversions, equivalents, and substitutes included in the spirit and scope of the present invention. In the description of the present invention, when it is determined that a detailed description of known technologies related to the present invention may obscure the subject matter of the present invention, the detailed description is omitted.

Terms such as first and second may be used to describe various components, but the components are not limited by the terms. The terms are used only for the purpose of distinguishing one component from other components.

Terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise.

In this specification, terms such as “include” or “have” are intended to indicate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, or one or more other. It should be understood that features or numbers, steps, operations, components, parts, or combinations thereof are not excluded in advance.

In addition, in the present specification, when one component'transmits' data to another component, the component may directly transmit the data to the other component, or through at least one other component It means that the data may be transmitted to the other components. Conversely, when one component'directly transmits' data to another component, it means that the data is transmitted from the component to the other component without passing through the other component.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. The same reference numerals in each drawing denote the same members.

1 is a view for explaining a schematic concept for implementing a method for providing a customized diagnostic system according to an embodiment of the present invention.

Referring to FIG. 1, in order to implement a method for providing a customized diagnostic system according to an embodiment of the present invention, a reference diagnostic system 1 may be implemented.

The reference diagnostic system 1 may provide a customized diagnostic system according to the technical idea of the present invention.

According to an example, the reference diagnostic system 1 may be changed to a customized diagnostic system. Alternatively, the reference diagnostic system 1 includes a plurality of neural networks, and some of the plurality of neural networks may be implemented as the customized diagnostic system.

Accordingly, the reference diagnostic system 1 may include one or more neural networks.

The reference diagnostic system 1 communicates with a system (e.g., a first diagnostic system, 200) of a predetermined diagnostician (defined as meaning including a diagnostic institution) that wants to build a customized diagnostic system. The technical idea of can be implemented.

The first diagnosis system 200 refers to a system used by the first diagnosis device, and may not necessarily be specified by any one physical device.

The reference diagnostic system 1 may be a diagnostic system learned using predetermined learning data. When the reference diagnostic system 1 learned in advance as described above is used, as described later, the first diagnostic system 200 can perform annotation on only a relatively small amount of learning data.

Of course, according to an embodiment, the reference diagnostic system 1 may perform learning at a time when additional learning data annotated from the first diagnostic system 200 is received in a state in which only learning data is prepared.

According to an example, the reference diagnostic system 1 may be a system previously learned through a plurality of learning data. The reference diagnostic system 1 may be a system that is trained to output a judgment result for a disease, that is, a diagnosis result when receiving learning data, for example, a biological image. The learning data used for learning of the reference diagnostic system 1 may be data annotated by someone other than the first diagnosis.

In addition, the reference diagnostic system 1 may retain some unannotated learning data.

Then, M pieces of learning data (M is an integer of 2 or more) among the non-annotated learning data may be transmitted to the first diagnostic system 200 (S100).

The M learning data may be learning data used to reflect the diagnostic propensity of the first diagnostic system 200, that is, the first diagnostic system to the reference diagnostic system 1. Accordingly, M learning data transmitted to the first diagnostic system 200 may be defined as custom learning data.

The number of learning data for customization, that is, M may be determined in advance. In addition, the M may be determined in association with the number of learning data used for learning of the reference diagnostic system 1. That is, if the learning data for customization that is too small than the number of training data used for learning of the reference diagnostic system 1 is used, the diagnostic tendency of the first diagnosis may not be well expressed in the reference diagnostic system 1 to be.

For example, if the reference diagnostic system 1 is a diagnostic system trained with 10000 learning data, M may be determined to be about 200. The determination of the number of Ms may be determined by an experiment, but the number of the first diagnosis may be reflected, but not too many.

When the M customizing learning data is transmitted to the first diagnostic system 200, the M customizing learning data may be annotated by the first diagnostic. Then, the annotated learning data for customization may be transmitted from the first diagnostic system 200 to the reference diagnostic system 1 (S110).

According to an embodiment, the M learning data for customization may not be arbitrarily selected, but may be data selectively selected through a predetermined process. For example, learning data having an ambiguous characteristic in which diagnosis results may be slightly different for each diagnosis may be selected in advance as customizing learning data. For example, the learning data may be a bio-image, and depending on the bio-image, a specific diagnoser may determine that the disease has been expressed, and another diagnoser may determine that the disease has not been expressed. Alternatively, when the diagnostic system is trained to determine the degree of disease progression, for a same biological image, a specific diagnoser may determine that the disease is in the first state, and another diagnoser may determine that the disease is in the second state. As such, in reality, there may be a case in which a specific biological image from a pathological point of view may be ambiguous so that the degree of disease development or the degree of disease progression may be determined differently depending on the diagnosis.

When data including features having ambiguity is selected and used as learning data for customizing, even if a smaller number of customizing learning data is used, the tendency of a specific diagnoser can be better reflected. This is because, if anyone sees learning data with features that are clear enough to produce the same diagnostic results, there may be little room for the diagnostic propensity to be reflected, and when these data are used as customizable learning data, the pre-trained reference diagnostic system (1 ).

According to an example, as will be described later, the reference diagnostic system 1 may be implemented to derive a final diagnosis result by agreement of a plurality of individual diagnostic systems, in which case at least two of the plurality of individual diagnostic systems Input data outputting different diagnostic results may be selected as custom learning data. That is, when the reference diagnostic system 1 is implemented as a consensus diagnostic system, a plurality of individual diagnostic systems may be included. In this case, since each individual diagnostic system can perform a function of a diagnostician, diagnosis of the individual diagnostic system naturally There is an effect in that data including features having ambiguity can be selected using the result.

Then, the reference diagnostic system 1 may reflect the annotation customized learning data to the reference diagnostic system 1 (S120).

The fact that the annotated learning data for customization is reflected in the reference diagnostic system 1 performs a series of data processing so that the annotated learning data for customization affects the diagnosis of the reference diagnostic system 1. Can mean

For example, the reference diagnostic system 1 may reflect the customized learning data to the reference diagnostic system 1 by performing re-learning or transfer learning using the received learning data for customization.

As described above, the learning data for customization that is annotated by the first diagnoser is reflected in the reference diagnostic system 1, and the reference diagnostic system 1 reflects the diagnostic propensity of the first diagnose, that is, customized by the first diagnoser. It can function as a diagnostic system.

On the other hand, the subject side of the reference diagnostic system 1 may perform charging in exchange for implementing a customized diagnostic system on the first diagnostic unit (S130). For the charging, the reference diagnostic system 1 may communicate with the first diagnostic system 200 to perform a predetermined charging process. An average expert in the technical field of the present invention can easily deduce that the charging process exists in various ways. In addition, it is needless to say that the charging may proceed through a separate process by a predetermined contract, not an automated charging process by the reference diagnostic system 1.

In addition, the average charge of the technical field of the present invention can be easily inferred that the charging may be performed before the customizing learning data is reflected in the reference diagnostic system 1 or after it is reflected. .

Meanwhile, the billing amount may be selected to be dependent on the quality or number of learning data for customization.

For example, when the learning data for customizing is randomly selected from among a plurality of learning data, a lower billing amount may be set, and when selected as learning data including features having ambiguity as described above, a higher billing amount is higher. It may be priced.

Or, the proportion of the number of learning data for customization to the number of learning data other than the learning data for customization in the diagnostic system (eg, the reference diagnostic system 1 or the individual diagnostic system to be described later) in which the learning data for customization will be reflected. Depending on the billing amount may be selected. In general, the higher the specific gravity, the more dependent on the propensity of the first diagnosis, so a higher amount can be selected as the billing amount. Alternatively, the higher the number of learning data for customization, the higher the billing amount can be set.

By constructing a customized diagnostic system in the first diagnostic unit, which reflects the diagnostic tendency of the first diagnostic unit, an individual diagnostic institution secures a large number of training data and annotates the acquired training data individually to build a diagnostic system. It is effective to construct a diagnostic system specialized for an individual diagnostic institution even if there is no specialized knowledge to construct a diagnostic system by utilizing such learning data.

In addition, from the perspective of the provider of the reference diagnostic system 1, there is an effect that can generate additional revenue by easily constructing a diagnostic system that reflects the tendency of individual diagnoses.

Although the reference diagnostic system 1 is illustrated as one physical device in FIG. 1, the reference diagnostic system 1 may be a system in which a plurality of physical devices are organically combined according to an embodiment.

In addition, the reference diagnostic system 1 may be provided with a plurality of individual neural networks, and each neural network may be a neural network trained to individually receive input data and output diagnostic results. Of course, depending on the embodiment, the reference diagnostic system 1 may be provided with any one neural network, and in this case, learning data for customization may be reflected in the one neural network.

Depending on the embodiment of the reference diagnostic system 1, embodiments in which the learning data for customization is reflected on the reference diagnostic system 1 may be varied.

An example of this will be described with reference to FIG. 2.

2 is a diagram for illustratively explaining an example of implementation of a reference diagnostic system according to an embodiment of the present invention.

Referring to FIG. 2, the reference diagnostic system 1 may include a plurality of individual diagnostic systems (eg, 10 to 30) each including a plurality of neural networks. Depending on the embodiment, the stadend diagnostic system 40 may be further included. In addition, the reference diagnostic system 1 may include a predetermined consensus diagnostic system 100.

Each of the individual diagnostic systems (eg, diagnostic system 1 to diagnostic system N, 10 to 30) may be a system trained to output individual diagnostic results (eg, individual diagnostic results 1 to individual diagnostic results N). Then, as will be described later, the consensus diagnosis system 100 may be a system implemented to output a consensus diagnosis result based on these individual diagnosis results (eg, individual diagnosis results 1 to individual diagnosis results N).

Each of the diagnostic systems (eg, diagnostic systems 1 to 10, 30 to 30) is learned based on supervised learning, and when predetermined biometric data (eg, a biometric image) is input, the neural network learning the biometric data is generated. It may be a system that classifies into a predetermined classification (eg, a classification according to whether or not the disease has developed or the degree of disease progression) and outputs the result as a diagnosis result.

The diagnostic systems (e.g., diagnostic system 1 to diagnostic system N, 10 to 30) receive the same biometric data, respectively, and diagnosis results through each learned neural network, that is, the individual diagnostic results (e.g., individual Diagnostic results 1 to individual diagnostic results N) can be output.

At this time, the diagnostic results of each of the diagnostic systems (eg, diagnostic systems 1 to 10, 30 to 30) may be dependent on annotation according to the annotator. That is, each of the diagnostic systems (eg, diagnostic system 1 to diagnostic system N, 10 to 30) may output an individual diagnosis result to have a dependency on the annotated learning data.

In order to compensate for this vulnerability, each of the diagnostic systems according to the technical concept of the present invention (eg, the diagnostic systems 1 to the diagnostic systems N, 10 to 30) may be a system trained with a learning data set annotated by different annotation subjects. have. Depending on the embodiment, each of the diagnostic systems (eg, diagnostic system 1 to diagnostic system N, 10 to 30) may be diagnostic systems that are trained using the same annotated learning data. In addition, each of the diagnostic systems (eg, diagnostic systems 1 to 10, 30 to 30) may be customized later by different diagnosticians.

For example, diagnostic system 1 (10) may be a system learned with a first learning data set annotated by a first annotator, and diagnostic system 2 (20) may be a second learning data set annotated by a second annotator. It may be a system learned by, and the diagnostic system N 30 may be a system trained with an N-learning data set annotated by the N-annotator.

In this case, each of the diagnostic systems (eg, the diagnostic systems 1 to the diagnostic systems N, 10 to 30) may have characteristics that depend on the tendency or tendency of the annotator of the learning data used for learning.

Then, the diagnostic systems (for example, the diagnostic systems 1 to the diagnostic systems N, 10 to 30) that receive the same diagnostic data (eg, a biological image) ideally output the same diagnostic results, but as described above, The judgment tendency of the data influences the diagnosis results, and in some cases, different diagnosis results may be output. That is, the diagnostic systems (for example, the diagnostic systems 1 to the diagnostic systems N, 10 to 30) may output different diagnostic results for the same diagnostic data according to the tendency of the annotator.

However, these vulnerabilities can be compensated through consensus diagnosis using individual diagnosis results of a plurality of diagnosis systems (eg, diagnosis system 1 to diagnosis system N, 10 to 30) according to the technical idea of the present invention.

For example, the reference diagnostic system 1 may output a consensus diagnosis result based on individual diagnosis results output from each of the diagnosis systems (eg, diagnosis systems 1 to 10, 30 to 30). The consensus diagnosis result may refer to the final diagnosis result generated based on all or part of the individual diagnosis results.

According to an embodiment, the reference diagnostic system 1 may further include a standard diagnostic system 40 as will be described later. In addition, the consensus diagnosis result may be determined based on the diagnosis result of the standard diagnosis system 40, that is, the standard individual diagnosis result.

The standard diagnostic system 40 may mean a diagnostic system trained with gold standard learning data. Gold standar learning data may mean standard or ideal annotated learning data. The gold standard learning data may not simply be annotated by one subject, but may mean data that has been annotated by a plurality of subjects or has been annotated by criteria agreed by multiple subjects. . In general, it may be desirable to perform learning using the gold standard training data, but in practice, it may be expensive to generate the gold standard training data, but rather, one learned using the gold standard training data. Rather than constructing a diagnostic system, constructing a consensus diagnosis result by having a plurality of diagnostic systems learned by individual subjects may exhibit higher diagnostic performance. In the case of gold standard learning data, when there are different annotation opinions on annotations on specific biometric data, only a small amount of opinions are disregarded and only the agreed annotations are reflected in the learning, but as in the technical idea of the present invention, This is because a small number of opinions are reflected in learning in at least an individual diagnostic system when diagnostic systems individually learned through annotation by a subject (eg, diagnostic systems 1 to 9, 10 to 30) are constructed.

Moreover, when the standard diagnostic system 40 is included among a plurality of diagnostic systems for consensus diagnosis, higher diagnostic performance may be exhibited.

3 is a view for explaining the concept of customizing a reference diagnostic system according to an embodiment of the present invention.

Referring to FIG. 3, when a plurality of diagnostic systems (eg, 10 to 40) are included in the reference diagnostic system 1 as described above, the reference diagnostic system 1 may include the first diagnostic system 200. After receiving M annotated customizing learning data from (S110), any one of the diagnostic systems (e.g., diagnostic system 1 to diagnostic system N, 10 to 30) diagnostic system (e.g., diagnostic system 1, 10) ) Can be limited to reflect the annotated learning data for customization (S120). Of course, re-learning or transfer learning may be used to reflect the annotated learning data for customization.

That is, when the reference diagnostic system 1 is provided with a plurality of diagnostic systems (e.g., diagnostic system 1 to diagnostic system N, 10 to 30), the plurality of diagnostic systems (e.g., diagnostic system 1 to diagnostic system) N, 10 to 30) can be customized to the first diagnostic.

Then, the customized diagnostic system 1 (10) is customized to the first diagnostic unit, and other diagnostic systems (eg, 20 to 40) may remain without reflecting the diagnostic propensity of the first diagnostic unit. In this case, when the reference diagnostic system 1 receives predetermined data, that is, diagnostic data to be diagnosed, from the first diagnostic system 200, the diagnostic result customized to the first diagnostic, that is, the diagnostic system 1 (10) In addition to the output of the individual diagnosis results, it is possible to provide diagnosis results of other diagnostic systems or consensus diagnosis results together.

According to another embodiment, the annotated customized learning data received by the reference diagnostic system 1 from the first diagnostic system 200 is diagnostic systems (eg, diagnostic systems 1 to 9, 10 to 30). It may be reflected in all. Of course, in this case, all of the diagnostic systems (for example, the diagnostic systems 1 to the diagnostic systems N, 10 to 30) may be changed to output a diagnostic result reflecting the diagnostic propensity of the first diagnosis.

Of course, at this time, the standard diagnostic system 40 may not reflect the annotated customized learning data received from the first diagnostic system 200.

4 is a diagram illustrating a schematic system configuration when a reference diagnostic system according to an embodiment of the present invention performs a supervised learning-based consensus diagnosis method.

Referring to FIG. 4, as described above, the reference diagnostic system 1 may include a plurality of diagnostic systems (eg, diagnostic systems 1 to diagnostic systems N, 10 to 30) and a consensus diagnostic system 100. In addition, it may further include a standard diagnostic system (30).

In addition, in order to output the consensus diagnosis system 100, the consensus diagnosis system 100 may simply derive the consensus diagnosis result according to a majority vote when the individual diagnosis results have different diagnosis results. Alternatively, the consensus diagnosis system 100 assigns a weight to each of the diagnosis systems (eg, diagnosis systems 1 to 10, and 30 to 30), and applies the consensus diagnosis result by applying the assigned weight to the corresponding individual diagnosis result. You can also print

The weight may be assigned according to the diagnosis history of each of the diagnosis systems (eg, diagnosis system 1 to diagnosis system N, 10 to 30). For example, a weighting process may be driven by the consensus diagnosis system 100 so that a weight is reduced by a predetermined criterion in a diagnosis system that outputs a diagnosis result different from the consensus diagnosis result. In addition, the weighting process may only consider whether it is different from the consensus diagnosis result, but may further consider the number of types of diagnosis results among all the individual diagnosis results.

For example, when 10 diagnostic systems are implemented to implement consensus diagnosis, only one of the 10 diagnostic systems determines that the disease has not been expressed with respect to specific biological data, and all other diagnostic systems determine that the disease has been expressed. Can. In this case, the weight of the specific diagnostic system may be reduced by a predetermined first reference value (eg, 0.1).

However, four of the ten diagnostic systems may determine that the disease has not been expressed with respect to the specific biological data, and all six diagnostic systems may determine that the disease has been expressed. In this case, even if the consensus diagnosis result is the manifestation of a disease, the preliminary value of the four diagnostic systems may be reduced only by a predetermined second reference value (eg, 0.05) smaller than the first reference value (eg, 0.1).

In other words, if the consensus diagnosis result is assumed to be the correct answer, the weight of the diagnostic system may be simply adjusted according to the number of times the wrong answer was output. Depending on the degree of adjustment, or in some cases, it is determined that it is not an incorrect answer, so the weight may not be adjusted.

In addition, various embodiments of the method for deriving the consensus diagnosis result and the methods of weighting each of the diagnostic systems (eg, the diagnostic systems 1 to the diagnostic systems N, 10 to 30) may be possible in various embodiments. The average expert can easily deduce.

According to an embodiment, the consensus diagnosis system 100 may further receive a standard individual diagnosis result from the standard diagnosis system 40, which will be described later. In addition, the consensus diagnosis result may be output based on the received standard individual diagnosis result.

In addition, the consensus diagnostic system 100 assigns a higher weight to the standard diagnostic system 40 than each of the individual diagnostic systems (eg, the diagnostic systems 1 to 10, and the diagnostic systems N to 10 to 30), thereby allowing the diagnostic systems ( For example, a consensus result may be derived by reflecting the weights assigned to each of the diagnostic systems 1 to diagnostic systems N, 10 to 30).

Meanwhile, FIG. 4 illustrates a case where the consensus diagnostic system 100 is implemented as a separate physical device from the diagnostic systems (eg, the diagnostic system 1 to the diagnostic system N, 10 to 30) or the standard diagnostic system 40 However, the consensus diagnostic system 100 may be implemented in a device that is physically the same as at least one of the diagnostic systems (eg, diagnostic system 1 to diagnostic system N, 10 to 30) or the standard diagnostic system 40. Will be easily deduced by the average expert in the technical field of the present invention.

Of course, the consensus diagnosis system 100 can be implemented with various data processing systems (for example, a computer, a server, a smart phone, or a dedicated device) as long as it can perform the functions defined in this specification.

5 is a view for explaining the concept of using a customized diagnostic system according to an embodiment of the present invention.

Referring to FIG. 5, the reference diagnostic system 1 may be a system in which M pieces of annotated customizing learning data received from the first diagnostic system 200 are reflected. And according to an example, the reference diagnostic system 1 includes a plurality of diagnostic systems (eg, diagnostic systems 1 to 10, 30 to 30) and/or a standard diagnostic system 40 as described above. In addition, it may be a system implemented to perform consensus diagnosis according to their individual diagnosis results.

In this case, the annotated customized learning data received from the first diagnostic system 200 is any one of a plurality of diagnostic systems (eg, diagnostic system 1 to diagnostic system N, 10 to 30) (eg, diagnostic system 1). (10)).

Thereafter, the reference diagnostic system 1 may receive diagnostic data from the diagnostic request system. The diagnostic request system transmits diagnostic data (eg, biometric data, such as a biological image) to the reference diagnostic system 1, and that is, a diagnosis result corresponding to the diagnostic data is received from the reference diagnostic system 1 , It may be a system that makes a diagnostic request with the reference diagnostic system (1). The diagnosis request system may be the first diagnosis system 200 or may be a system of another diagnosis organization.

The reference diagnostic system 1 may receive a diagnostic request including the diagnostic data, and determine the type of diagnostic result based on the diagnostic request. The type of the diagnosis result may be, for example, which diagnostic system is requesting the diagnosis result. For example, according to a diagnosis request, only individual diagnosis results of a specific individual diagnosis system may be transmitted, or consensus diagnosis results may be transmitted together.

For example, the reference diagnostic system 1 may determine whether or not the diagnostic request requests a diagnostic result through a customized diagnostic system to the first diagnostic unit. The diagnosis request through the diagnostic system customized to the first diagnosis may be a diagnosis request performed by the first diagnosis, but is not limited thereto, and other diagnosis may also trust the diagnosis propensity of the first diagnosis, and in this case, other diagnosis Again, you may want a diagnostic result customized to the first diagnostic.

For example, the reference diagnostic system 1 may specify the diagnosis result type using the attributes (eg, IP, MAC address, etc.) of the system that outputs the diagnostic request. Alternatively, when a service page for using the reference diagnostic system 1 is provided, the type of diagnosis result to be transmitted may be determined in response to a diagnosis request through login information. Alternatively, the diagnosis request may explicitly include information on the type of diagnosis result (eg, identification information of the first diagnosis, identification information of the diagnosis system 1, etc.).

For example, when a diagnostic request including predetermined diagnostic data is received from the first diagnostic system 200, the reference diagnostic system 1 may specify the type of diagnostic result corresponding to the diagnostic request.

Then, the reference diagnostic system 1 may transmit only individual diagnostic results of the diagnostic system 1 10 customized to the first diagnostic system to the first diagnostic system 200. Of course, depending on the embodiment, the diagnosis result of the first diagnosis may be assisted by transmitting the consensus diagnosis result together with the individual diagnosis results of the diagnostic system 1 (10). In particular, when the individual diagnosis result of the diagnosis system 1 (10) is different from the consensus diagnosis result, it is effective to inform the first diagnosis that more careful diagnosis is necessary.

6 is a view for explaining the schematic configuration of each of the diagnostic systems according to an embodiment of the present invention.

First, referring to FIG. 6A, the reference diagnostic system 1 may include any one neural network, that is, a neural network for a reference diagnostic system, and as described above, a plurality of diagnostic systems (eg, diagnostic systems 1 to diagnostic system) N, 10 to 30) may also include neural networks for individual diagnostic systems. That is, there may be a plurality of neural networks for a reference diagnostic system, which are neural networks included in the reference diagnostic system 1.

The individual diagnostic systems (eg, diagnostic systems (eg, diagnostic system 1 to diagnostic system N, 10 to 30)) may include a configuration as shown in FIG. 6A. Since the standard diagnostic system 40 may also be implemented in the same configuration as that of the individual diagnostic systems (eg, diagnostic systems 1 to 10, 30 to 30), individual diagnostic systems (eg, diagnostic systems) Only 1 to diagnostic systems N, 10 to 30) will be described. In addition, the individual diagnostic systems (eg, diagnostic systems 1 to diagnostic systems N, 10 to 30) may also include the same or similar configurations, and therefore, the diagnostic systems (eg, diagnostic systems 1 to diagnostic system N, 10 to 10) 30) Only one of the configurations will be described.

The first diagnostic system (diagnostic system 1) 10 may include a processor 11 and a storage device 12. The first diagnostic system 10 refers to a data processing device having a computing power for realizing the technical idea of the present invention, and generally, as a personal computer, a portable terminal, etc., as well as a data processing device accessible to a client through a network. The average expert in the technical field of the present invention can easily infer that it can be implemented with any device capable of performing a specific service.

The processor 11 may refer to a computing device capable of driving the program 12-1 for realizing the technical idea of the present invention, and the processor 11 may include the program 12-1 and the present invention. Diagnosis can be performed using a neural network (12-2) defined by the technical idea of. The neural network may be a convolutional neural network, and upon receiving biometric data (eg, an image), a diagnostic result may be output through the learned neural network.

The program 12-1 may mean software defined to train the neural network 12-2 through supervised learning or to perform diagnosis using the learned neural network 12-2. .

The storage device 12 may mean data storage means capable of storing the program 12-1 and the neural network 12-2, and may be implemented as a plurality of storage means according to an implementation example. Also, the storage device 12 may mean not only a main storage device included in the first diagnosis system 10, but also a temporary storage device or memory that may be included in the processor 11.

Hereinafter, in this specification, that the diagnostic system (eg, 10) performs a predetermined function, a processor (eg, 11) provided in the diagnostic system (eg, 10) uses the program (eg, 12-1). Of course, it may mean that a predetermined function is performed.

In the present specification, that the first diagnostic system 10 performs diagnosis may mean a series of processes for receiving biometric data and outputting output data defined in this specification, for example, a diagnosis result.

The first diagnostic system 10 may receive biometric data for each predetermined unit unit. The unit unit may be, for example, a pixel unit, a patch unit, or a slide unit.

The diagnosis result of the first diagnosis system 10 may be simply whether the disease is expressed or a corresponding value (for example, probability, etc.) depending on the type of the disease, or when the disease is expressed, a state indicating the degree of disease condition It may be information.

For example, as will be described later, when the technical idea of the present invention is used for the diagnosis of prostate cancer, the state information may include a Gleason Pattern or a Gleason Score that is an indicator of the progression of prostate cancer. have. For example, the Gleason pattern has a value of 2 to 5, and the larger the number, the more severe the prostate cancer is expressed. Therefore, the status information is information or normal corresponding to the probability that the biological tissue corresponding to the unit unit to be diagnosed corresponds to a specific value (eg, 3, 4, or 5) of the Gleason pattern (ie, the disease is not expressed). If not, it may include information corresponding to the probability.

In any case, the first diagnostic system 10 may output a diagnosis result through the learned neural network 12-2.

Meanwhile, the consensus diagnosis system 100 according to the technical idea of the present invention may include the configuration shown in FIG. 2B.

The consensus diagnostic system 100 may also include a storage device 120 in which the processor 110 and the program 121 are stored.

The consensus diagnosis system 100 is connected to individual diagnosis systems (eg, diagnosis system 1 to diagnosis system N, 10 to 30) and/or standard diagnosis system 40 through a wired/wireless network to implement the technical idea of the present invention. In order to transmit and receive the necessary information. Depending on the implementation example, the consensus diagnostic system 100 may be installed and implemented in the individual diagnostic systems (eg, diagnostic systems 1 to 9, 10 to 30) or the standard diagnostic system 40. In this case, the consensus diagnostic system 100 may share the hardware of the individual diagnostic systems (eg, diagnostic system 1 to diagnostic system N, 10 to 30) or standard diagnostic system 40.

The programs 121 stored in the storage 120 of the consensus diagnostic system 100 are each from individual diagnostic systems (eg, diagnostic systems 1 to diagnostic systems N, 10 to 30) and/or standard diagnostic systems 40, respectively. Diagnostic results (eg, individual diagnostic results 1 to N and/or standard individual diagnostic results) can be received and confirmed.

The consensus diagnosis system 100 receives individual diagnosis results from only diagnosis systems (eg, diagnosis system 1 to diagnosis system N, 10 to 30) each learned with a set of learning data annotated by different annotation subjects. Alternatively, as described above, a standard individual diagnosis result may be further received from the standard diagnostic system 40. Of course, as described above, each of the diagnostic systems (e.g., diagnostic system 1 to diagnostic system N, 10 to 30) may be systems trained with the same learning data, and each of the diagnostic systems may be assigned to a specific diagnoser as described above. It can be prepared to be customized.

In any case, the consensus diagnosis system 100 may generate and output a consensus diagnosis result based on the received diagnosis results.

An example in which the consensus diagnosis system 100 generates a consensus diagnosis result will be described with reference to FIG. 7.

7 is a view for explaining the concept of deriving an agreement based on individual diagnosis results according to an embodiment of the present invention.

Referring to FIG. 7, in order to implement the technical concept of the present invention, diagnostic systems (eg, diagnostic systems 1 to diagnostic systems N, 10 to 30) may be implemented, and consensus diagnosis included in the reference diagnostic system 1 The system 100 may receive a diagnosis result for the same biological data from each of the diagnostic systems (eg, diagnostic systems 1 to 9, 10 to 30). In addition, FIG. 7 exemplarily describes a case in which the consensus diagnostic system 100 further receives standard individual diagnosis results from the standard diagnostic systems STD and 40.

Although FIG. 7 illustrates a case in which the diagnosis result indicates the expression of a specific disease, it is as described above that various types of information may be output as the diagnosis result according to an implementation example of the neural network 12-2.

According to an example, the consensus diagnosis system 100 may simply specify a type having a greater number of types of diagnosis results (eg, O with disease manifestation, X without disease manifestation) as the consensus diagnosis result. have.

Depending on the implementation example, the consensus diagnosis system 100 may assign weights to respective diagnosis systems (eg, diagnosis system 1 to diagnosis system N, 10 to 30, standard diagnosis system 40).

Such a weight may be adjusted based on the number of times the consensus diagnosis result and other individual diagnosis results are output while deriving the consensus diagnosis result for a plurality of biological data. For example, the diagnostic system 2 may have a weight of 0.9, and the diagnostic system 1 may have a weight of 1, which may mean that the diagnostic system 2 has output at least once when the result of the consensus diagnosis and other individual diagnosis are output. .

The consensus diagnosis system 100 may generate a consensus diagnosis result based on a diagnosis result whose weight is reflected as a weight factor. For example, if a specific diagnostic system outputs a diagnostic result to X, and the weight of the specific diagnostic system is a (for example, 0.9), in the process of deriving a consensus diagnosis result, the diagnostic result of the specific diagnostic system is a (0.9). ). By reflecting the weight as described above, more types of diagnosis results (eg, O or X) can be judged as consensus diagnosis results.

When the consensus diagnosis system 100 outputs individual diagnosis results different from the consensus diagnosis result, for each of the diagnosis systems (for example, the diagnosis system 1 to the diagnosis system N, 10 to 30, and/or the standard diagnosis system 40) The weight may be adjusted by lowering a predetermined value (for example, 0.1) for each time.

Alternatively, even if a specific diagnostic system (eg, diagnostic system 2) outputs an individual diagnosis result (eg, X) different from the consensus diagnosis result (eg, O) as described above, the other individual diagnosis result (eg, X) The number or specific gravity of the diagnostic systems (e.g., diagnostic system 1 to diagnostic system N, 10 to 30 and/or the standard diagnostic system 40) outputting (specific diagnostic system (e.g., diagnostic system) compared to the total number of diagnostic systems) The weight may be adjusted by further considering the number of diagnostic systems outputting the same type of diagnostic results as in 2). For example, when only one out of 10 diagnostic systems outputs different diagnostic results, the weight of the corresponding system is adjusted by a first value (eg, 0.1), and a certain number (eg, 3) or more out of 10 diagnostic systems In case of outputting a diagnosis result different in specific gravity (eg, 30%) or more, the weight of each of the corresponding systems may be adjusted only by the second pack (eg, 0.03). Alternatively, in the case of a certain number or more than a certain weight, the weight of the system may not be adjusted at all even if a diagnosis result and a diagnosis result are different.

That is, in the latter case, since there is room for interpretation to be made according to judgment, it is possible to derive a more reliable diagnosis result by adaptively adjusting the weight corresponding to the reliability of the diagnosis system.

In addition, when the diagnostic result of the standard diagnostic system 40 is further used, the consensus diagnostic system 100 has a weight of the standard diagnostic system 40 of other diagnostic systems (for example, the diagnostic systems 1 to 9, the diagnostic system N, 10). It may also be given a higher weight compared to 30).

On the other hand, in the case of the consensus diagnosis method according to the technical idea of the present invention, the diagnostic systems learned with different learning data sets (for example, the diagnostic system 1 to the diagnostic system N, 10 to 30 and/or the standard diagnostic system 40) In the case of using, it is also possible to derive a diagnosis result usefully. This may be more useful when the consensus diagnosis system 100 performs re-learning (or transfer learning) for a diagnosis system that outputs a diagnosis result different from the consensus diagnosis result, as will be described later.

8 is a view for explaining the concept of divided annotation of learning data according to an embodiment of the present invention.

Referring to FIG. 8, when a diagnostic method according to the technical concept of the present invention is applied, diagnostic systems (for example, diagnostic systems 1 to diagnostic system N) are used using learning data annotated differently for each annotator as described above. , 10-30 and/or the standard diagnostic systems 40, as well as each of the diagnostic systems (e.g., diagnostic systems 1-diagnostic system N, 10-30 and/or standard diagnostic systems 40) It may also be useful when learning using other learning data.

For example, as illustrated in FIG. 8, the performance of the trained engine, that is, the diagnostic system may vary depending on the tendency of the annotator to annotate the training data used for training, but the learning data may be changed even if the same annotator performs annotation. It also depends on whether you use it.

Therefore, as shown in FIG. 8, the diagnostic system 1 is a system trained with the training data set 1, the diagnostic system 2 is a system trained with the training data set 2, and the diagnostic system 3 is a system trained with the training data set 3, The diagnostic system 4 may be a system trained with the training data set 4.

Each training data set may or may not have duplicate training data, and the annotators may be the same or different for each training data set.

In any case, the diagnosis result of the diagnostic system trained for each training data set may be different. To this end, considerable efforts and efforts have been made in determining a wide range of training data sets including all of various cases from the time of selecting the training data set. It had to be costly.

However, according to the technical idea of the present invention, if learning is performed with only a sufficiently large set of learning data sets to train each of the diagnostic systems, diagnostic systems learned with different learning data sets (eg, diagnostic systems 1 to diagnostic systems) Even N, 10 to 30 and/or the standard diagnostic system 40, it is possible to derive a more accurate diagnosis result through the consensus diagnosis result.

That is, there is an effect that the bias of the learning data set itself can be significantly neglected in the process of deriving the consensus diagnosis result.

Moreover, for a diagnostic system that outputs a diagnosis result different from a consensus diagnosis result for specific biological data, re-learning may be performed by annotating the specific biological data as a consensus diagnosis result, in which case bias and/or learning by an annotator Deflection by data can be worn out over time, ultimately improving the performance of individual diagnostic systems (e.g., Diagnostic Systems 1 to Diagnostic System N, 10 to 30 and/or Standard Diagnostic System 40). There is.

An example of this will be described with reference to FIG. 9.

9 is a diagram for explaining the concept of re-learning an individual diagnostic system using a consensus diagnosis result according to an embodiment of the present invention.

Referring to FIG. 9, for example, specific biometric data is input to respective diagnostic systems (eg, diagnostic system 1 to diagnostic system N, 10 to 30, and/or standard diagnostic system 40), and diagnostic systems (eg, , Diagnostic system 1 to diagnostic system N, 10 to 30 and/or standard diagnostic system 40) each may output individual diagnostic results to consensus diagnostic system 100. And at this time, the consensus diagnosis system 100 may output O as the consensus diagnosis result, and the diagnosis system 2 (20) may output X as the individual diagnosis result.

In this case, the consensus diagnosis system 100 may include information on a diagnosis system (eg, diagnosis system 2) that outputs an individual diagnosis result different from the consensus diagnosis result, the specific biological data at this time, and the consensus diagnosis result (eg, O). Can be saved.

In addition, the stored diagnostic information may be used to re-learn the diagnostic systems (eg, diagnostic system 1 to diagnostic system N, 10 to 30, and/or standard diagnostic system 40).

For example, after a predetermined time has elapsed, the diagnostic system 2 may be re-learned using a re-learning data set including the specific biometric data labeled with the consensus diagnosis result (eg, O).

Of course, when the diagnostic system 2 is a diagnostic system customized for a specific diagnostician, consent of the specific diagnostician may be required.

According to the technical idea of the present invention, the diagnostic systems (e.g., diagnostic system 1 to diagnostic system N, 10 to 30 and/or standard diagnostic system 40) are collected by collecting biometric data that outputs a consensus diagnosis result and a different diagnostic result, respectively Can re-learn That is, biometric data stored by the consensus diagnosis system 100 and outputting different diagnosis results from the consensus diagnosis result may be fed back to the individual diagnosis system as learning data. In this case, the bias of each of the diagnostic systems (e.g., diagnostic system 1 to diagnostic system N, 10 to 30 and/or standard diagnostic system 40) (e.g., depending on the tendency of the annotator or the bias of the learning data itself) ) Has the effect of being improved.

The method for providing a customized diagnostic system according to an embodiment of the present invention can be implemented as computer readable codes on a computer readable recording medium. The computer-readable recording medium includes all types of recording devices in which data readable by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, hard disk, floppy disk, and optical data storage devices. In addition, the computer-readable recording medium can be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. In addition, functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers in the technical field to which the present invention pertains.

The present invention has been described with reference to one embodiment shown in the drawings, but this is merely exemplary, and those skilled in the art will understand that various modifications and other equivalent embodiments are possible therefrom. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the appended claims.

Claims (11)

In the method of providing a diagnostic system,
Transmitting, by the reference diagnostic system, M (M is an integer of 2 or more) unannotated customizing learning data to the first diagnostic system;
Receiving, by the reference diagnostic system, the M pieces of learning data for customization, annotated by the first diagnosis side, from the first diagnosis system; And
And reflecting the received M customized learning data for the reference to the reference diagnostic system,
A method for providing a customized diagnostic system, characterized in that the M customized learning data annotated by the first diagnoser side is reflected in the neural network for the reference diagnostic system as a condition of receiving a predetermined price.
The method of claim 1, wherein the method for providing a customized diagnostic system is provided.
A method for providing a customized diagnostic system comprising the step of receiving, by the reference diagnostic system, the diagnostic data reflecting the annotated M customizing learning data, receiving diagnostic data from the first diagnostic system, generating diagnostic results and transmitting the diagnostic results to the first diagnostic system. .
According to claim 1, The reference diagnostic system,
A method of providing a customized diagnostic system, characterized in that the larger the M pieces or the larger the M according to the weight of M pieces compared to the number of learning data learned by the reference diagnostic system.
According to claim 1, The reference diagnostic system,
A method for providing a customized diagnostic system, comprising a set of individual diagnostic systems including a plurality of individual diagnostic systems, and performing diagnosis through consensus based on individual diagnostic results output by a plurality of individual diagnostic systems.
The method of claim 4, wherein the step of reflecting the received learning data for the M customization of the annotation to the reference diagnostic system,
And re-learning at least some of the plurality of individual diagnostic systems using the annotated M customizing learning data.

The M-customized learning data according to claim 4,
A method of providing a customized diagnostic system including data in which at least two of the individual diagnostic systems included in the individual diagnostic system set output different diagnostic results.
In the method of providing a diagnostic system,
Reference diagnostic system-the reference diagnostic system includes a set of individual diagnostic systems including a plurality of individual diagnostic systems-transmits M (M is an integer of 2 or more) unannotated customizing learning data to the first diagnostic system To do;
Receiving, by the reference diagnostic system, the M pieces of learning data for customization, annotated by the first diagnosis side, from the first diagnosis system; And
Reflecting the received M customized learning data for the first individual diagnostic system included in the reference diagnostic system set;
Receiving, from a predetermined diagnostic request system, a diagnostic request including diagnostic data to be diagnosed by the reference diagnostic system;
According to the received diagnostic request, the type of the diagnostic result corresponding to the diagnostic request is determined as the first individual diagnostic result of the first individual diagnostic system, and the first individual diagnostic system receives and outputs the input data. A method of providing a customized diagnostic system, comprising transmitting a diagnostic result including a diagnostic result to the diagnostic request system.
A computer program installed in a data processing apparatus and stored in a computer-readable recording medium for performing the method according to any one of claims 1 to 7.
Processor; And
It includes a storage device where the program is stored,
A data processing system in which the method according to any one of claims 1 to 7 is performed by the program executed by the processor.
Processor;
And a storage device in which a program executed by the processor is stored,
The storage device includes at least one neural network,
The program,
M (M is an integer greater than or equal to 2) transmits unannotated customizing learning data to a first diagnostic system, and receives the M customizing learning data annotated by the first diagnostic system from the first diagnostic system. Receiving, and reflecting the received M learning data for customization in the reference diagnostic system,
A customizing diagnosis system characterized in that the M pieces of learning data for customization, which are annotated by the first diagnosis side, are reflected in the neural network as a condition of receiving a predetermined price.
Processor;
And a storage device in which a program executed by the processor is stored,
The storage device,
A neural network corresponding to a plurality of individual diagnostic systems,
The program,
M (M is an integer greater than or equal to 2) transmits unannotated customizing learning data to a first diagnostic system, and receives the M customizing learning data annotated by the first diagnostic system from the first diagnostic system. Upon reception, the received M customized learning data for reflection is reflected in a first neural network corresponding to a first individual diagnostic system among the plurality of individual diagnostic systems,
When a diagnostic request including diagnostic data to be diagnosed is received from a predetermined diagnostic request system, the type of the diagnostic result corresponding to the diagnostic request is determined as the first individual diagnostic result of the first individual diagnostic system according to the received diagnostic request. , A customized diagnostic system that transmits, to the diagnostic request system, a diagnostic result including the first individual diagnostic result that the first individual diagnostic system receives and outputs the input data.

Images