KR102202361B1 - System for detecting bone tumour - Google Patents

Abstract

The present invention relates to a bone tumor detection system, comprising: a structure learning unit configured to similarly reconstruct a shape of a bone included in an input image to learn a structural representation of a bone to generate first learning information; A discrimination learning unit that receives the first learning information and learns a tumor detection criterion to generate second learning information; And a tumor detection unit for generating bone tumor detection information by detecting the presence and location of a bone tumor by comparing the first learning information and the second learning information with the input X-ray image, wherein the structure learning unit An encoder that extracts and recombines the differential expression of the included bone shape; A decoder that stores and manages the extracted differential expression, and masks a component for any one of the femur, tibia, and fibula; And a SoftMax classifier that determines a bone state probability from the masked components and generates first learning information that classifies the bone state.
According to the present invention as described above, bone tumors are classified by segmenting the skeleton using ensemble half-supervised learning (ESSW) for a captured image, and determining a bone state probability through a softmax classifier from the segmented image. It can be detected early and used as a decision aid tool for bone tumor screening.

Description

Bone tumor detection system {System for detecting bone tumour}

The present invention relates to a bone tumor detection system, and more particularly, to a technology for automatically detecting a bone tumor by classifying and detecting whether a patient's bone is normal or a tumor using a radiograph (x-ray) image. will be.

Imaging diagnosis of bone tone, a side effect commonly found in asymptomatic patients, requires expert attention and evaluation. In particular, for knee pain, radiography is an essential diagnostic method for irradiation.

Among all tumors, the incidence of bone tumors is 2% to 3% and is constantly increasing, and doctors often use various methods such as imaging while observing clinical symptoms for accurate diagnosis. The basic elements of the differential diagnosis and evaluation of bone tumors using conventional radiography are divided into the patient's medical history and age, clinical symptoms, the anatomical location of the lesion, the definition of the lesion and the metastatic area of the elbow joint, and the radiographic characteristic lesion.

Deep Learning (DL) is a popular method used to perform many important tasks in radiology medical imaging. Deep learning architectures typically consist of a set of distinct layers that perform the task, including image patch convolutions and maximum response sampling.

Many studies are introducing deep learning methods in the field of medical image analysis, including the identification of bone tumors, among which the deep convolutional neural network (CNN) provides a great advantage for image classification.

However, in order to detect a bone tumor in the related art, various data such as radiography, patient's medical history, age, clinical symptoms, and the anatomical location of the lesion are required, and it is not easy to accurately detect early, or in parallel with auxiliary treatment. It is difficult to cure in the later stages of surgical resection. Therefore, early detection of puff tumors is very important in the treatment process.

Accordingly, the applicant of the present invention intends to propose a technology for early and accurate detection of bone tumors by classifying a captured image into a plurality of parts, classifying/learning according to its shape, extracting features, and classifying bone tumors through a neural network.

US published patent US20160110890 (published on April 21, 2016)

An object of the present invention is to classify a bone state by segmenting a skeleton using an ensemble semi-supervised learning (ESSW) for a captured image, and determining a bone state probability through a softmax classifier in the segmented image. It is intended to detect bone tumors early and use them as a decision aid tool for bone tumor screening.

An embodiment of the present invention for achieving this technical problem is a bone tumor detection system, a structure learning unit that learns structural representation of bone by similarly reconstructing the shape of a bone included in an input image to generate first learning information ; A discrimination learning unit that receives the first learning information and learns a tumor detection criterion to generate second learning information; And a tumor detection unit for generating bone tumor detection information by detecting the presence and location of a bone tumor by comparing the first learning information and the second learning information with the input X-ray image, wherein the structure learning unit An encoder that extracts and recombines the differential expression of the included bone shape; A decoder that stores and manages the extracted differential expression, and masks a component for any one of the femur, tibia, and fibula; And a SoftMax classifier for generating first learning information that classifies the bone state by determining a bone state probability from the masked components.

Preferably, the structural learning unit generates first learning information through unsupervised learning, the discriminant learning unit generates second learning information through supervised learning, and the tumor detection unit generates ensemble half-supervised learning. It is characterized by generating bone tumor detection information through (ESSW).

The structure learning unit receives an image including any one segment of the femur, tibia, or fibula through the encoder, and reconstructs the shape of the bone included in the image received through a plurality of convolutional layers and pooling layers to express the structure of the bone. Characterized in that learning to.

The SoftMax classifier is characterized in classifying a bone tumor by calculating a probability of a bone state from the output of the last layer of each of the encoder and the decoder.

According to the present invention as described above, bone tumors are classified by segmenting the skeleton using ensemble half-supervised learning (ESSW) for a captured image, and determining a bone state probability through a softmax classifier from the segmented image. It is detected early and can be used as a decision aid tool for bone tumor screening.

1 is a view showing the architecture of an ESSW for bone tumor detection of a bone tumor detection system according to an embodiment of the present invention.
2 is a block diagram showing a bone tumor detection system according to an embodiment of the present invention.
3 is an exemplary view showing a sample of bone tumor detection information generated by the bone tumor detection system according to an embodiment of the present invention.
4 is an exemplary view showing a bone segmentation process performed through an encoder and a decoder of the bone tumor detection system according to an embodiment of the present invention.
5 is an exemplary view showing a bone shape partially divided by the bone tumor detection system according to an embodiment of the present invention.
6 is a flow chart showing a method for detecting a bone tumor according to an embodiment of the present invention.
7 is a flow chart showing a detailed process for step S100 of the method for detecting a bone tumor according to an embodiment of the present invention.

Specific features and advantages of the present invention will become more apparent from the following detailed description based on the accompanying drawings. Prior to this, terms or words used in the present specification and claims are based on the principle that the inventor can appropriately define the concept of terms in order to describe his or her invention in the best way. It should be interpreted as a corresponding meaning and concept. In addition, when it is determined that a detailed description of known functions and configurations thereof related to the present invention may unnecessarily obscure the subject matter of the present invention, it should be noted that the detailed description has been omitted.

1 is a diagram showing the architecture of an ESSW for bone tumor detection of a bone tumor detection system S according to an embodiment of the present invention, and FIG. 2 is a bone tumor detection system S according to an embodiment of the present invention. ) Is a configuration diagram showing.

1 and 2, the bone tumor detection system S according to an embodiment of the present invention similarly reconstructs the shape of the bone included in the input image to learn the structural representation of the bone to obtain first learning information. A structure learning unit to generate 100, a discrimination learning unit 200 that receives the first learning information and learns a tumor detection criterion to generate second learning information, and the first learning information and the second learning information and input It is configured to include a tumor detection unit 300 for generating bone tumor detection information by detecting the presence and location of a bone tumor through comparison of the received X-ray images.

In this case, the bone tumor detection information may include any one of a normal lymphoma, a benign tumor, or a malignant tumor, as shown in the sample shown in FIG. 3.

In the following, a specific reference will be omitted, but the first learning information generated by the bone tumor detection system S according to an embodiment of the present invention is generated by unsupervised learning, and the second learning information is It is created by supervised learning. Therefore, since bone tumor detection information is a result of the combination of unsupervised learning and supervised learning, it is desirable to interpret it as being generated by ensemble half-supervised learning (ESSW) combining different learning models.

Specifically, the structure learning unit 100 stores and manages the encoders 102a and 102b that extract and recombine the differential expression of the bone shape included in the input image, and the extracted differential expression, and the femur, tibia or fibula Including a decoder (104a, 104b) for masking any one of the constituent elements, and a SoftMax classifier 106 that determines the bone state probability from the masked constituent elements to generate first learning information that classifies the bone state. Is composed.

In this case, the bone segmentation process performed by the encoder and decoder according to an embodiment of the present invention is the same as the example shown in FIG. 4, and the partially divided bone shape is the same as the example shown in FIG. 5.

Specifically, the encoders 102a and 102b extract a differential representation of the shape of a bone included in an image received through a plurality of convolutional layers and pooling layers.

In addition, the image received by the encoders 102a and 102b includes any one segment of the femur, tibia, or fibula, and similarly reconstructs the shape of the bone included in the image received through a plurality of convolutional layers and pooling layers. Study the structural representation of bones.

In addition, the extracted differential expression calculates the sum of the continuous convolutional layers connected to each of the encoders 102a and 102b before downsampling or upsampling, and reduces the size of the functional map through a pooling operation on the calculation result.

At this time, the cost function to be minimized is a mean squared error (MSE), and is expressed as [Equation 1] below.

[Equation 1]

In addition, the SoftMax classifier 106 classifies a bone tumor by calculating the probability of a bone state from the outputs of the encoder 102a and the decoder 104a, and the last layer of the encoder 102b and the decoder 104b. In this case, the cross entropy is used as a cost function for classification of bone tumors calculated as in [Equation 2] below.

[Equation 2]

On the other hand, the discrimination learning unit 200 generates second learning information for determining a bone state probability by learning a tumor detection criterion based on the bone state included in the first learning information approved from the structure learning unit 100.

At this time, an embodiment of the present invention generates the first learning information through unsupervised learning of the structure learning unit 100, and the second learning information through supervised learning of the discrimination learning unit 200. Because learning information is generated, that is, bone tumors are classified from two outputs, the total loss function is expressed as [Equation 3] below.

[Equation 3]

In addition, the tumor detection unit 300 detects the presence of a bone tumor and its location by comparing the average output of the first learning information and the second learning information by the ensemble half-supervised learning (ESSW) and the input X-ray image. Generate tumor detection information.

At this time, the bone tumor detection information generated by the tumor detection unit 300 is a result of the combination of the first learning information (unsupervised learning) and the second learning information (supervised learning), and the ensemble half-supervised learning combining different learning models Generated by (ESSW).

At this time, the bone state is classified by calculating the average probability for class i (the bone state probability from the masked components) as shown in [Equation 4] below through the average output by the ensemble half-supervised learning (ESSW).

[Equation 4]

A description of the factors used in the above equations is as follows.

는 ground true value(진정한 값)이고,

는 prediction value(예측 값)이며, n은 image resolution by(width x height)(이미지 해상도(너비 x 높이))이고, C는 number of classes(그룹 수)이며, M은 number of ensemble models(앙상블 모델 수)이고,

는 average probability of class

(클래스 I의 평균 확률)이며,

는 probability of class

using model

(j번째 클래스를 사용하는 클래스 I의 확률)이다.first,

Is the ground true value,

Is the prediction value, n is the image resolution by(width x height), C is the number of classes, and M is the number of ensemble models. Number),

Is the average probability of class

(Average probability of class I),

Is the probability of class

using model

(Probability of class I using the jth class).

The bone tumor detection system S according to an embodiment of the present invention as described above adjusts the size of the input image to 240x200, and the kernel size for each convolutional layer is set to 3x3, and the backpropagation algorithm is It is applied to calculate the slope of the error function for the variable, and the weights can be updated using the Adam algorithm with a learning rate of 0.0001.

In addition, the mini-batch strategy is used to converge faster than full-batch training, the weights of hidden layers are initialized randomly, and in the case of SoftMax layers, all weights are initialized to zero. And, the momentum method is used with a value of 0.9 to accelerate the training phase.

Hereinafter, a method for detecting a bone tumor according to an embodiment of the present invention will be described with reference to FIG. 6.

First, the structure learning unit 100 similarly reconstructs the shape of the bone included in the input image and learns the structural representation of the bone to generate first learning information (S100). At this time, the first learning information is generated through unsupervised learning.

Subsequently, the discrimination learning unit 200 receives the first learning information, learns a tumor detection criterion, and generates second learning information (S200). At this time, the second learning information is generated through supervised learning.

Then, the tumor detection unit 300 detects the presence of a bone tumor and its location by comparing the first learning information and the second learning information with the input X-ray image to generate bone tumor detection information (S300). At this time, bone tumor detection information is generated through an average output by ensemble half-supervised learning (ESSW).

Hereinafter, a detailed process of step S100 of the method for detecting a bone tumor according to an embodiment of the present invention will be described with reference to FIG. 7.

First, the structure learning unit 100 extracts and recombines the differential expression of the bone shape included in the input image (S102).

Subsequently, the structure learning unit 100 stores and manages the extracted differential expression, and masks a component for any one of the femur, tibia, and fibula (S104).

Then, the structure learning unit 100 determines a bone state probability from the masked constituent elements to generate first learning information that classifies the bone state (S106).

At this time, the structure learning unit 100 calculates the probability of the bone state from each output of the last layer of the encoder 102a and the decoder 104a, and the encoder 102b and the decoder 104b through the SoftMax classifier 106. Classify bone tumors.

As described above and shown in connection with a preferred embodiment for illustrating the technical idea of the present invention, the present invention is not limited to the configuration and operation as shown and described as described above, and deviates from the scope of the technical idea. It will be appreciated by those skilled in the art that many changes and modifications can be made to the present invention without. Therefore, all such appropriate changes and modifications and equivalents should be considered to be within the scope of the present invention.

S: bone tumor detection system
100: Structure Learning Department
102a, 102b: encoder
104a, 104b: decoder
106: SoftMax classifier
200: discrimination learning unit
300: tumor detection unit

Claims (4)

A structure learning unit that similarly reconstructs the shape of the bone included in the input image to learn a structural representation of the bone to generate first learning information;
A discrimination learning unit that receives the first learning information and learns a tumor detection criterion to generate second learning information; And
Comprising a tumor detection unit for generating bone tumor detection information by detecting the presence and location of a bone tumor by comparing the first learning information and the second learning information with the input X-ray image,
The structure learning unit,
An encoder that extracts and recombines the differential expression of the bone shape included in the input image;
A decoder that stores and manages the extracted differential expression, and masks a component for any one of the femur, tibia, and fibula; And
A SoftMax classifier that determines the bone state probability from the masked components and generates first learning information that classifies the bone state.
Bone tumor detection system comprising a. The method of claim 1,
The structure learning unit generates the first learning information through unsupervised learning,
The discriminant learning unit generates the second learning information through supervised learning,
The tumor detection unit generates the bone tumor detection information through ensemble half-supervised learning (ESSW). The method of claim 1,
The structure learning unit,
Receiving an image including any one segment among the femur, tibia, or fibula through the encoder, and reconstructing the shape of the bone included in the image received through a plurality of convolutional layers and pooling layers to learn structural representation of the bone Bone tumor detection system, characterized in that. The method of claim 1,
The SoftMax classifier,
Bone tumor detection system, characterized in that to classify the bone tumor by calculating the probability of the bone state from the output of each of the last layer of the encoder and the decoder.

Images