WO2017010612A1

WO2017010612A1 - System and method for predicting pathological diagnosis on basis of medical image analysis

Info

Publication number: WO2017010612A1
Application number: PCT/KR2015/009311
Authority: WO
Inventors: 김효은; 황상흠; 백승욱; 이정인; 장민홍; 유동근; 팽경현; 박승균
Original assignee: 주식회사 루닛
Priority date: 2015-07-10
Filing date: 2015-09-03
Publication date: 2017-01-19

Abstract

The present invention provides a pathological diagnosis prediction system on the basis of medical image analysis and a method using the system, the system comprising: an untagged image database for storing an untagged image having no tag information; a tagged image database for storing a tagged image having tag information; a classifier for outputting a result of classification of an input image; a learning unit for making the classifier learn on the basis of a tagged image stored in the tagged image database; an image processing unit for receiving a query image, storing the query image in the untagged image database, and transmitting the query image to the classifier; and an analyzing unit for generating an analysis result associated with a query image on the basis of a result of classification of the query image output from the classifier.

Description

Pathology Prediction System and Method Based on Medical Image Analysis

The present invention relates to a system and method for predicting pathological diagnosis based on medical image analysis. More particularly, the present invention relates to a medical image analysis-based pathological predictive prediction system capable of reliably and efficiently performing pathological diagnosis by analyzing medical images. And to a method.

Pathology is the field of medicine in which tissue samples are examined by naked eye or under a microscope and analyzed to determine abnormalities. For example, in order to diagnose cancer, a tissue sample of a suspected tissue is examined by a pathologist and a doctor through a microscope and diagnosed by determining the presence of cancer cells. This is called pathology diagnosis, and this pathology diagnosis is a final step of diagnosis as a confirmation procedure for diagnosis of a suspected lesion of a patient.

In order to automatically perform such a pathological diagnosis by a computer or the like, an existing image for comparing and analyzing an input query image and an image and pathological diagnosis data in which a corresponding pathological diagnosis result is databased are required.

The database to be analyzed should store the normal / abnormal images, the medical image including information on whether the lesion is present, the pathological diagnosis of the lesion, and the location of the lesion. Comparative analysis should be performed on query images. The presence or absence of such lesions, pathological diagnosis results and location information of the lesions may be referred to as tag information. The database to be subjected to the comparative analysis will have more reliability as more images including the tag information exist. In particular, by using the image including the tag information such as machine learning (learning) to learn from a large amount of data to ensure that only the information optimized for prediction to always maintain more accurate results It can be predicted.

However, as a conventional technique, a technique for predicting a pathological diagnosis result of a lesion has not yet been proposed by efficiently analyzing and diagnosing a medical image using a learning technique such as machine-learning in the medical imaging field. It is true.

In addition, various medical images used in existing medical facilities are often not databased or organized. In particular, even if the image itself is present, there are most cases in which there is no information on whether the lesion exists in the corresponding image, the pathological diagnosis result of the lesion and the location of the lesion, that is, tag information. In addition, even when the tag information is present, there are many cases in which only the tag information indicating only the pathological diagnosis result of the lesion or the presence of the lesion exists without location information of the lesion, and thus, there is a lot of difficulty in constructing a reliable database.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a system and a method capable of efficiently and efficiently analyzing a location of a lesion and a pathological diagnosis result of the lesion with respect to a medical image.

Another object of the present invention is to provide a system and method for improving analysis accuracy by keeping a classifier always up to date by performing learning based on machine learning for medical image analysis.

In addition, the present invention provides a system and method for improving the learning efficiency and reliability by appropriately utilizing the location information of the small amount of lesions, the presence or absence of relatively large amount of lesion information and the pathological diagnosis of the lesions for the learning of the classifier. To do it for another purpose.

In order to solve the above problems, the present invention provides a pathological diagnosis prediction system based on medical image analysis, comprising: an untagged image database storing untagged images without tag information; A tagged image database storing a tagged image having tag information; A classifier configured to output a classification result for the input image; A learning unit learning the classifier based on a tagged image stored in the tagged image database; An image processing unit for receiving the query image, storing the image in the untagged image database, and transmitting the image to the classifier; And an analyzer configured to generate an analysis result of the query image based on the classification result of the query image output from the classifier.

Here, the tag information is preferably first tag information indicating the presence or absence of a lesion on a corresponding image and a pathological diagnosis result of the lesion.

The tag information may further include second tag information indicating a location of a lesion with respect to the corresponding image.

The classifier may be configured to extract feature data of an input image, and to probabilistically predict and output a classification result of pathology based on the extracted feature data.

The classifier may further include a feature extractor which extracts feature data of an input image; A feature vector converter converting the extracted feature data into a feature vector; And a classification unit configured to output a classification result of pathological diagnosis on an input image based on the feature vector.

In addition, a feature extraction variable for extracting feature data from the feature extraction unit, a conversion variable for transforming a feature vector in the feature vector converter, and a classification variable for determining a classification result in the classification unit are tagged. The tag image stored in the image database may be configured to be learned by the learner.

The learning unit may estimate the feature extraction variable based on the second tag information of the tag image, and estimate the conversion variable and the classification variable based on at least one of the first tag information and the second tag information of the tag image. It may be configured to train the classifier.

The analysis result generated by the analysis unit may be configured to include final pathological diagnosis information including a classification result of the pathological diagnosis output from the classifier and information related to the classification result of the pathological diagnosis.

According to another aspect of the present invention, in the method for predicting pathology diagnosis in the pathology diagnosis prediction system based on the medical image analysis as described above, the image processing unit receives the query image, stored in the untagged image database and transmitted to the classifier A first step of doing; A second step of outputting a classification result of pathological diagnosis on an image input by the classifier; And a third step of generating an analysis result based on the classification result of the pathological diagnosis outputted by the analysis unit.

Here, before the first step, the image processing unit receives the tagged image and transmits it to the tagged image database; And training the classifier based on the tagged image stored in the tagged image.

According to the present invention, it is an object of the present invention to provide a system and method for efficiently and efficiently analyzing the location of a lesion and the pathological diagnosis of the lesion with respect to a medical image.

In addition, the present invention can provide a system and method that can improve the accuracy of analysis by keeping the classifier (up-to-date) by performing the learning based on machine learning for medical image analysis.

In addition, the present invention can provide a system and method that can increase the learning efficiency and reliability by appropriately using a small amount of location information of the lesion, the presence of a relatively large amount of lesions and pathological diagnosis information of the lesion for learning the classifier. Can be.

1 is a view showing a connection relationship of the pathological diagnosis prediction system 100 according to the present invention.

2 is a diagram illustrating an internal configuration of the pathological diagnosis prediction system 100.

3 is a diagram illustrating an embodiment of a configuration of the classifier 130.

4 is a flowchart illustrating an embodiment of a method performed by the pathological diagnosis prediction system 100 according to the present invention described with reference to FIGS. 1 to 3.

5 is a flowchart illustrating an example of a learning process of the classifier 130 performed in the pathological diagnosis prediction system 100 according to the present invention.

6 is a flowchart illustrating another example of a learning process of the classifier 130 performed in the pathological diagnosis prediction system 100 according to the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating a connection relationship between a pathology diagnosis prediction system 100 based on medical image analysis according to the present invention.

Referring to FIG. 1, a pathology diagnosis prediction system 100 based on medical image analysis (hereinafter, simply referred to as “pathology diagnosis prediction system 100”) is connected to a client terminal 200 through a network.

Here, the network means a known internet network, a wired or wireless communication network, or a combination thereof.

The client terminal 200 is a means for transmitting the medical image to be analyzed to the pathological diagnosis prediction system 100, for example, a medical image supply system such as a picture archiving and communication system (PACS), or an MRI, CT, X-ray photographing apparatus. It means various medical equipment such as. In addition, it may be a computer that is connected to such medical equipment or independently stores a medical image, or a smartphone or tablet PC that can be connected to them.

The client terminal 200 performs a pathology through a network on a query image, which is a medical image for analyzing lesion information such as the presence or absence of a lesion, the pathological diagnosis result of the lesion, or the location of the lesion, in the pathology diagnosis prediction system 100. The diagnostic prediction system 100 analyzes the received Curie image and analyzes the received Curie image and transmits analysis results including lesion information such as the presence or absence of the lesion and the pathological diagnosis result of the lesion or the location of the lesion. Send to 200.

Referring to FIG. 2, the pathological diagnosis prediction system 100 includes an untagged image database 110, a tagged image database 120, a classifier 130, a learner 140, an image processor 150, and an analyzer. 160.

The untagged image database 110 stores an untagged image without tag information, and the tagged image database 120 is a means for storing a tagged image with tag information. .

Here, tag information refers to additional information associated with a corresponding image, for example, information indicating whether a lesion exists in the image, information indicating a pathological diagnosis result of the lesion, information indicating a location of the lesion, Means information related to the corresponding image, such as comment information of the diagnoser.

In the present invention, an image having such tag information is called a tagged image, and an image without tag information is called an untagged image, and these are respectively tagged image database 120 and untagged image database. Stored separately in (110).

On the other hand, since the present invention relates to a system for predicting a diagnosis of pathology by analyzing a medical image, in particular, among the tag information, the presence or absence of pathology information and pathological diagnosis information indicating the presence or absence of a lesion on the image and the pathology diagnosis result of the first The location information indicating the location of the lesion in the image, called tag information, is called a second tag.

Here, the lesion presence information may be information on whether a tumor exists in the corresponding image, and the pathological diagnosis information refers to pathological diagnostic information on whether the tumor is malignant or benign.

Meanwhile, the second tag information, which is the lesion position information, may be omitted. That is, the tagged image has only the presence or absence information of the lesion and the first tag information that is pathological diagnosis information, or has the first tag information that is the lesion presence information and the pathological diagnosis information and the second tag information that is the lesion position information.

Meanwhile, in the present specification, a tag having only the presence tag information and the pathological diagnosis information, the first tag information, is called a weak tag, and the tag having the location information of the lesion together with the first tag information is called a rich tag. I'll call you. Also, an image having a weak tag is called a weak tag image and an image having a rich tag is called a rich tag image.

The untagged image database 110 is a means for storing an untagged image without the tag information as described above. The untagged image may be received from the client terminal 200 or collected from other medical imaging equipment. Save only untagged images without information.

The tagged image database 120 separately stores only tagged images having tag information. The tagged images may also be received from the client terminal 200 or collected from other medical imaging equipment.

The classifier 130 performs a function of outputting a classification result for the input image. The classifier 130 extracts a pattern or feature data of an input image, and probably predicts and outputs a classification result based on the extracted pattern or feature data.

In order to more accurately predict the classification result of the input image, the classifier 130 may be learned based on an image including reliable classification results accumulated in the past. This learning is made by the learning unit 140 described later.

The classifier 130 according to the present invention may include a tagged image database (eg, a tag image database) in the learning unit 140 according to a learning method such as, for example, an artificial neural network, a support vector machine (SVM), and the like known in the art. The classification result of the input image may be more accurately determined by periodically learning the images stored in the reference numeral 120.

Referring to FIG. 3, the classifier 130 includes a feature extractor 131, a feature vector converter 132, and a classifier 133.

The feature extractor 131 extracts feature data of an input image, and the feature vector converter 132 converts the extracted feature data into a feature vector. In addition, the classification unit 133 outputs a classification result of the input image based on the feature vector.

That is, when an image is input to the classifier 130, the feature extractor 131 extracts feature data representing a feature of the input image from the input image, and the feature vector converter 132 extracts the extracted feature data from a predetermined dimension ( to a feature vector of dimension). The classifier 133 outputs the classification result of the corresponding image based on the probability value based on the feature vector.

In order to operate the classifier 130, the classifier 130 includes a feature extraction parameter for extracting feature data and a classification parameter for predicting a classification result. The learner 140 learns periodically based on image data including reliable classification results accumulated in the past. This learning method will be described below with respect to the learning unit 140.

Meanwhile, the classifier 130 maintains the latest state through such learning, extracts feature data from an input image, converts the extracted feature data into a feature vector, and determines a classification result of the transformed feature vector. . In this case, the classification result may be information on the presence or absence of the lesion, such as the first tag information, pathological diagnosis information on the lesion, and location information of the lesion, such as the second tag information. At this time, the presence information of the lesion, pathological diagnosis information and location information of the lesion may be generated based on the probability value.

The learner 140 performs a function of learning the classifier 130 based on the tagged images stored in the tagged image database 120.

As described above, the classifier 130 includes a feature extraction variable for extracting feature data and a classification variable for determining a classification result. The learner 140 stores existing data, that is, the tagged image database 120. These variables are trained periodically or whenever needed.

As described above, the tagged image refers to an image having tag information, and the tag information includes first tag information indicating the presence or absence of a lesion and pathological diagnosis information of the lesion, and second tag information indicating the location information of the lesion. In this case, for more effective learning of the classifier 130, it may be preferable to use the second tag information including the location information of the lesion.

In general, most existing medical images have no tag information or only have first tag information indicating the presence or absence of a lesion and a pathological diagnosis result of the lesion, and have a second tag information which is location information of a lesion. It is less. Accordingly, although it may be difficult to train the classifier 130 to a degree of sufficient reliability with only a small number of second tag information, the present invention utilizes a small number of second tag information and a plurality of first tag information together to classify the classifier 130. ) Can be learned.

That is, the learner 140 estimates a feature extraction variable to be used by the feature extractor 131 based on the tag image having the second tag information, that is, the location information of the lesion, and at least one of the first tag information and the second tag information. After estimating the conversion variable in the feature vector converter 132 using the tag image having any one, the classifier 133 using the tag image having at least one of the first tag information and the second tag information. The classifier 130 may be trained by using the plurality of first tag information and the plurality of second tag information together by estimating the classification variable to be used in the.

Meanwhile, a specific method of estimating each variable may use a conventionally known learning algorithm, which is not a direct object of the present invention, and thus detailed description thereof will be omitted.

In addition, the above-described learning method in the learning unit 140 is also exemplary and other learning methods may be used.

Next, the image processor 150 will be described with reference to FIG. 2 again.

The image processor 150 receives a query image and stores the query image in an untagged image database. The query image is transmitted from the client terminal 200, and the query image is a medical image to be analyzed for lesion information such as the presence or absence of a lesion, the diagnosis result of the lesion, or the location of the lesion. It is a tagged image or a tagged image having only first tag information on the presence or absence of a lesion.

The image processor 150 receives the query image and stores it in the untagged image database 110. For this purpose, the image processor 150 is suitable for pre-processing the query image and storing the queried image in the untagged image database 110. It is desirable to perform the process of structuring into a form.

Here, as a preprocessing process, for example, interpolation, color / gamma correction, and color space conversion for converting a raw data for a query image into a high quality color image signal (eg, YCbCr) ) May be included. It may also include processes such as histogram equalization, image filtering, edge / contour detection, and the like. This preprocessing process may be configured as a combination of various detailed steps as necessary, and may be processed by the client terminal 200.

The image processor 150 transmits the preprocessed and structured image to the classifier 130, and the classifier 130 operates as described above to output the classification result for the query image.

The analyzer 160 is responsible for generating an analysis result for the query image based on the classification result for the query image output from the classifier 130. In addition, the analyzer 160 transmits the generated analysis result to the client terminal 200.

As described above, the classifier 130 outputs a classification result with respect to the input query image, wherein the classification result is lesion presence information indicating the presence or absence of the lesion and pathological diagnosis result information of the lesion, or the lesion indicating the position of the lesion together with these information. The location information can be output as a probability value. The analysis unit 160 receives the classification result, generates, processes, and processes other analysis information related to the classification result to generate a final analysis result for the query image, and transmits it to the client terminal 200.

Here, the other analysis information may include comprehensive diagnostic information (such as abnormality) including a final pathological diagnosis result in consideration of the number of lesion positions and the number of lesion positions on the query image. In addition, the final pathological diagnosis result information may be expressed as a probability value based on the probability value of the lesion output from the classifier 130.

Referring to FIG. 4, first, the client terminal 200 transmits a query image, that is, a medical image to be analyzed in the pathological diagnosis prediction system 100, to the pathological diagnosis prediction system 100 (S100).

The Curie image processing unit 150 of the pathological diagnosis prediction system 100 receives the query image, performs a predetermined preprocessing and structuring process as described above, and stores the query image in the untagged image database 110 ( S110, S120). The image processor 150 transmits the query image to the classifier 130.

Next, the classifier 130 receives the query image and outputs a classification result through the process as described above (S130).

The output classification result is transmitted to the analysis unit 140 and the analysis unit 140 generates an analysis result as described above (S140), and transmits the generated analysis result to the client terminal 200 (S150).

When the client terminal 200 receives the analysis result, the client terminal 200 performs necessary processing such as parsing the received analysis result (S160), for example, displays it on the display unit and provides the result to the user (S170).

Referring to FIG. 5, the pathology diagnosis prediction system 100 receives a tagged image from the client terminal 200 (S200). The tagged image may be a weak tag image having only the presence information and the first tag information, which is pathological diagnosis information, or the rich tag image having the second tag information, which is location information of the lesion, in addition to the first tag information. Can be.

The tagged image is stored in the tagged image database 120 through preprocessing and structuring in the Curie image processor 150 (S210 and S220).

The learner 150 trains the classifier 130 based on the tagged image including the tagged image newly stored in the tagged image database 120 (S230). This learning process means learning feature extraction variables, transformation variables, and classification variables used in the classifier 130 using the weak tag image or the rich tag image as described above. In addition, the learning process of the classifier 130 includes a verification process for evaluating whether learning has been well performed through the verification data and repeats the learning / verification process.

When this process is completed, the classifier 130 is updated to the latest version (S240).

In the embodiment of FIG. 6, the user terminal 200 obtains an untagged image from the untagged image database 110 of the pathological diagnosis prediction system 100 (S300), and the diagnostic information is tagged with the tag for the untagged image. Except for attaching the tag information in the form of recording (S310), the following process is the same as the embodiment of FIG.

According to the embodiment of FIG. 6, the tagged image may be secured by attaching tag information of a diagnosis person using an untagged image, and the classifier 130 may be trained using the tagged image.

In the above, the present invention has been described with reference to the preferred embodiment of the present invention, but the present invention is not limited to the above embodiment.

Claims

As a pathological diagnosis prediction system based on medical image analysis,

An untagged image database that stores an untagged image without tag information;

A tagged image database storing a tagged image having tag information;

A classifier configured to output a classification result for the input image;

A learning unit learning the classifier based on a tagged image stored in the tagged image database;

An image processing unit for receiving the query image, storing the image in the untagged image database, and transmitting the image to the classifier; And

An analyzer configured to generate an analysis result of the query image based on a classification result of the query image output from the classifier

Pathology diagnostic prediction system based on medical image analysis comprising a.
The method of claim 1,

The tag information is a medical image analysis-based pathological diagnosis prediction system, characterized in that the first tag information indicating the presence or absence of the lesion on the image and the pathological diagnosis result of the lesion.
The method of claim 2,

The tag information may further include second tag information indicating a location of a lesion with respect to the corresponding image.
The method of claim 1,

The classifier extracts feature data of an input image and predicts and outputs a classification result of the pathology diagnosis based on the extracted feature data.
The method of claim 4, wherein

The classifier,

A feature extractor which extracts feature data of an input image;

A feature vector converter converting the extracted feature data into a feature vector; And

A classification unit for outputting a classification result of pathological diagnosis on an input image based on the feature vector

Pathology diagnostic prediction system based on medical image analysis, characterized in that it comprises a.
The method of claim 5,

A feature extraction variable for extracting feature data from the feature extractor, a transform variable for transforming a feature vector in the feature vector converter, and a classification variable for determining a classification result in the classifier are the tagged image database. Pathology diagnosis prediction system based on medical image analysis, characterized in that learning by the learning unit through the tag image stored in the.
The method of claim 6,

The learning unit,

Estimate a feature extraction variable based on the second tag information of the tag image,

And predicting the conversion variable and the classification variable based on at least one of the first tag information and the second tag information of a tag image to train the classifier.
The method of claim 1,

The analysis result generated by the analysis unit includes pathology diagnosis based on medical image analysis, characterized in that it includes final classification information including information related to the classification result of the pathological diagnosis and the classification result outputted from the classifier. Prediction system.
A method for predicting pathological diagnosis in a pathological diagnosis prediction system based on medical image analysis according to any one of claims 1 to 8,

A first step of receiving, by the image processor, a query image, storing the query image in an untagged image database, and transmitting the image to a classifier;

A second step of outputting a classification result of pathological diagnosis on an image input by the classifier; And

A third step of generating an analysis result based on the classification result of the pathological diagnosis outputted by the analysis unit;

Pathological diagnostic prediction method comprising a.
The method of claim 9,

Before step one,

Receiving, by an image processor, a tagged image and transmitting the tagged image to a tagged image database; And

Learning unit trains the classifier based on the tagged image stored in the tagged image

Pathological diagnostic prediction method further comprising.