KR20200086406A - System for detecting bone tumour - Google Patents

Abstract

The present invention relates to a bone tumor detection system comprising: a structure learning unit which similarly reconstructs the shape of a bone included in an input image and learns the structural representation of the bone to generate first learning information; a discrimination learning unit which receives the first learning information and learns a tumor detection discrimination criterion to generate second learning information; and a tumor detection unit which detects the presence a bone tumor and the location of the bone tumor by comparing the first learning information and the second learning information with an inputted X-ray image to generate bone tumor detection information. The structure learning unit comprises: an encoder which extracts and recombines the differential representation of the bone shape included in the input image; a decoder which stores and manages the extracted differential representation, and masks a component with respect to any one of the femur, tibia, and fibula; and a softmax classifier which determines a bone state probability from masked components and generates first learning information in which bone states are classified. According to the present invention, with respect to a photographed image, the bone tumor detection system uses ensemble semi-supervised wnet (ESSW) to segment the skeleton, determines the bone state probability in the segmented image through the softmax classifier and classifies a bone state, thereby detecting a bone tumor in its early stage and being usable as a decision-making aid tool for screening of bone tumor.

Description

System for detecting bone tumour

The present invention relates to a bone tumor detection system, and more particularly, to a technique for automatically detecting a bone tumor by classifying and detecting whether a patient's bone is normal or a tumor using radiographic (x-ray) images will be.

Bone-lifting imaging, a side effect commonly found in asymptomatic patients, requires expert attention and evaluation. In the case of knee pain, radiography is an essential diagnostic method for irradiation.

Among all tumors, the incidence of bone tumors is 2% to 3%, and is constantly increasing, and doctors often use various methods such as imaging while observing clinical symptoms for accurate diagnosis. The basic elements of differential diagnosis and evaluation of bone tumors using traditional radiography are divided into the patient's medical history and age, clinical symptoms, the anatomical location of the lesion, the definition of the lesion and the metastatic region of the elbow, and the radiologic characteristic lesion.

Deep Learning (DL) is a universal method used to perform many important tasks in radiology and medical imaging. Deep learning architectures typically consist of a series of distinct layers of layers that perform tasks, including image patch convolution and maximum response sampling.

Many studies are introducing a deep learning method into the field of medical image analysis, including the identification of bone tumors, and among them, the Convolution Neural Network (CNN) provides a great advantage in image classification.

However, conventionally, in order to detect a bone tumor, various data such as radiographing, patient history, age, clinical symptoms, and anatomical location of the lesion are required, and it is not easy to detect accurately at an early stage. It is difficult to cure in the later stages of surgical resection. Therefore, early detection of cuckoo tumors is very important in the treatment process.

Accordingly, the present applicant intends to propose a technique for accurately detecting the presence or absence of a bone tumor by classifying the photographed image into a plurality of parts, classifying/learning according to the shape, and then extracting features and classifying the bone tumor through a neural network.

US Patent Publication US20160110890 (2016.04.21.Public)

The object of the present invention is to classify a bone state by determining a bone state probability through a softmax classifier in a segmented image by segmenting the skeleton using an ensemble semi-supervised Wnet (ESSW) on the captured image. , It is used to detect bone tumors early and to use them as a decision support tool for bone tumor screening.

One embodiment of the present invention for achieving such a technical problem is a bone tumor detection system, a structural learning unit that reconstructs the shape of a bone included in an input image similarly to learn the structural representation of the bone to generate first learning information. ; A discrimination learning unit receiving the first learning information and learning the tumor detection discrimination criteria to generate second learning information; And a tumor detection unit generating bone tumor detection information by detecting whether a bone tumor is present and its location through comparison of the first learning information and the second learning information with the received X-ray image. An encoder that extracts and recombines the differential expression of the included bone shape; A decoder for storing and managing the extracted differential expression and masking a component for any one of the femur, tibia or fibula; And a SoftMax classifier that generates first learning information that classifies bone states by determining the probability of bone states from the masked components.

Preferably, the structured learning unit generates the first learning information through unsupervised learning or supervised learning, and the discriminant learning unit through supervised learning or unsupervised learning. The second learning information is generated, and the tumor detection unit is characterized by generating bone tumor detection information through ensemble semi-supervised learning (ESSW).

The structural learning unit receives an image including any one of the femur, tibia, or fibula through an encoder, and reconstructs the shape of the bone included in the image received through multiple convolutional layers and pooling layers to express the structural structure of the bone Characterized by learning.

The SoftMax classifier is characterized by classifying bone tumors by calculating the probability of a bone condition from the output of the last layer of each of the encoder and decoder.

According to the present invention as described above, by dividing the skeleton using ensemble semi-supervised learning (ESSW) for the captured image, and by classifying the bone state by determining the probability of bone state through the softmax classifier in the segmented image, bone tumors It has an effect that can be found early and used as a decision-supporting tool for bone tumor screening.

1 is a diagram showing the architecture of ESSW for bone tumor detection in a bone tumor detection system according to an embodiment of the present invention.
Figure 2 is a block diagram showing a bone tumor detection system according to an embodiment of the present invention.
Figure 3 is an exemplary view showing a sample of the bone tumor detection information generated by the bone tumor detection system according to an embodiment of the present invention.
Figure 4 is an exemplary view showing a bone segmentation process performed through the encoder and decoder of the bone tumor detection system according to an embodiment of the present invention.
5 is an exemplary view showing a bone shape partially divided by a bone tumor detection system according to an embodiment of the present invention.
Figure 6 is a flow chart showing a bone tumor detection method according to an embodiment of the present invention.
7 is a flow chart showing a detailed process for step S100 of the bone tumor detection method according to an embodiment of the present invention.

Specific features and advantages of the present invention will become more apparent from the following detailed description based on the accompanying drawings. Prior to this, the terms or words used in the specification and claims are based on the principle that the inventor can properly define the concept of terms in order to best describe his or her invention in the technical spirit of the present invention. It should be interpreted as a matching meaning and concept. In addition, it should be noted that when it is determined that the detailed description of the known functions and configurations related to the present invention may unnecessarily obscure the subject matter of the present invention, the detailed description is omitted.

1 is a view showing the architecture of the ESSW for bone tumor detection of the bone tumor detection system (S) according to an embodiment of the present invention, Figure 2 is a bone tumor detection system (S) according to an embodiment of the present invention ).

1 and 2, the bone tumor detection system S according to an embodiment of the present invention similarly reconstructs the shape of the bone included in the input image to learn the structural representation of the bone to learn the first learning information. Structural learning unit 100 to generate, and discriminative learning unit 200 that receives the first learning information to learn tumor detection discrimination criteria to generate second learning information, and first learning information and second learning information and input It comprises a tumor detection unit 300 that detects whether or not a bone tumor and its location through comparison of the received X-ray image to generate bone tumor detection information.

In this case, the bone tumor detection information may include any one of normal lymphoma, benign tumor, or malignant tumor, as shown in the sample shown in FIG. 3.

Hereinafter, the specific reference will be omitted, but the first learning information generated by the bone tumor detection system (S) according to an embodiment of the present invention is generated by unsupervised learning, and the second learning information It is created by supervised learning. Therefore, bone tumor detection information is a result of a combination of non-supervised learning and supervised learning, and it is desirable to interpret it as being generated by ensemble semi-supervised learning (ESSW) that combines different learning models.

Specifically, the structure learning unit 100 stores and manages the encoders 102a and 102b for extracting and recombining the differential expression of the bone shape included in the input image, and the extracted differential expression, the femur, tibia or fibula The decoder 104a, 104b for masking a component for any one of the components, and the SoftMax classifier 106 for generating first learning information for classifying the bone condition by determining the probability of the bone condition from the masked components It is composed.

At this time, the bone splitting process performed through the encoder and decoder according to an embodiment of the present invention is the same as the example shown in FIG. 4, and the partially divided bone shape is the same as the example shown in FIG. 5.

Specifically, the encoders 102a and 102b extract differential expressions of the shape of the bones included in the image received through the plurality of convolutional layers and pooling layers.

In addition, the image input by the encoders 102a and 102b includes any one of the femur, tibia or fibula, and similarly reconstructs the shape of the bone included in the image received through multiple convolutional layers and pooling layers. Learn the structural representation of bones.

In addition, the extracted differential expression calculates the sum of the continuous convolutional layers connected to each of the encoders 102a and 102b before down-sampling or up-sampling, and reduces the size of the functional map by pulling the calculated results.

At this time, the cost function to be minimized is the mean squared error (MSE), and is expressed as [Equation 1] below.

[Equation 1]

In addition, the SoftMax classifier 106 classifies bone tumors by calculating the probability of bone condition from each output of the encoder 102a and decoder 104a, and the last layers of the encoder 102b and decoder 104b. At this time, the cross-entropy is used as a cost function for classification of the bone tumor calculated as shown in [Equation 2] below.

[Equation 2]

On the other hand, the discrimination learning unit 200 generates second learning information for determining the probability of a bone state by learning a tumor detection discrimination criterion based on the bone state included in the first learning information authorized by the structure learning unit 100.

At this time, an embodiment of the present invention generates the first learning information through the unsupervised learning of the structural learning unit 100, and the second through the supervised learning of the discriminant learning unit 200. The total loss function is expressed as [Equation 3] below because it generates the learning information, that is, the bone tumor is classified from the two outputs.

[Equation 3]

In addition, the tumor detection unit 300 detects a bone tumor and its location by comparing the average output of the first learning information and the second learning information by the ensemble semi-supervised learning (ESSW) and the received X-ray image. Tumor detection information is generated.

At this time, the bone tumor detection information generated by the tumor detection unit 300 is a result of combining the first learning information (non-supervised learning) and the second learning information (supervised learning), and ensemble semi-supervised learning combining different learning models. (ESSW).

At this time, the bone state is classified by calculating the average probability (the bone state probability from the masked components) for class i as shown in [Equation 4] below through the average output by the ensemble semi-supervised learning (ESSW).

[Equation 4]

The description of the factors used in the above equations is as follows.

는 ground true value(진정한 값)이고,

는 prediction value(예측 값)이며, n은 image resolution by(width x height)(이미지 해상도(너비 x 높이))이고, C는 number of classes(그룹 수)이며, M은 number of ensemble models(앙상블 모델 수)이고,

는 average probability of class

(클래스 I의 평균 확률)이며,

는 probability of class

using model

(j번째 클래스를 사용하는 클래스 I의 확률)이다.first,

Is the ground true value,

Is the prediction value, n is the image resolution by(width x height), C is the number of classes, and M is the number of ensemble models. Number),

Is the average probability of class

(Mean probability of class I),

Is the probability of class

using model

(Probability of class I using the jth class).

Bone tumor detection system (S) according to an embodiment of the present invention as described above, the size of the input image is adjusted to 240x200, the kernel size for each convolution layer is set to 3x3, the back propagation algorithm is a medium Applied to calculate the slope of the error function for the variable, the weight can be updated using the Adam algorithm with a learning rate of 0.0001.

In addition, the mini-batch strategy is used to converge faster than full batch training, the weight of the hidden layer is randomly initialized, and in the case of the SoftMax layer, all weights are initialized to zero. And, the momentum method is used with a value of 0.9 to accelerate the training phase.

Hereinafter, a method for detecting a bone tumor according to an embodiment of the present invention will be described with reference to FIG. 6.

First, the structure learning unit 100 similarly reconstructs the shape of the bone included in the input image, and learns the structural representation of the bone to generate first learning information (S100). At this time, the first learning information is generated through unsupervised learning.

Subsequently, the discrimination learning unit 200 receives the first learning information and learns the tumor detection discrimination criteria to generate the second learning information (S200). At this time, the second learning information is generated through supervised learning.

Then, the tumor detection unit 300 detects the presence and location of the bone tumor through comparison of the first learning information and the second learning information and the received X-ray image to generate bone tumor detection information (S300). At this time, bone tumor detection information is generated through an average output by ensemble semi-supervised learning (ESSW).

Hereinafter, a detailed process of step S100 of the bone tumor detection method according to an embodiment of the present invention will be described with reference to FIG. 7.

First, the structural learning unit 100 extracts and recombines the differential expression of the bone shape included in the input image (S102).

Subsequently, the differential learning extracted by the structure learning unit 100 is stored and managed, and a component for any one of the femur, tibia, or fibula is masked (S104).

Then, the structure learning unit 100 determines the probability of the bone state from the masked components to generate first learning information that classifies the bone state (S106).

At this time, the structure learning unit 100 calculates the probability of the bone state from each output of the encoder 102a and the decoder 104a, and the final layers of the encoder 102b and the decoder 104b through the SoftMax classifier 106. Classify bone tumors.

As described above, the present invention has been described and illustrated in connection with a preferred embodiment for illustrating the technical idea of the present invention, but the present invention is not limited to the configuration and operation as illustrated and described, and deviates from the scope of the technical idea. It will be understood by those skilled in the art that many changes and modifications to the present invention are possible without. Accordingly, all such suitable modifications and modifications and equivalents should be considered to be within the scope of the present invention.

S: bone tumor detection system
100: structural learning department
102a, 102b: encoder
104a, 104b: decoder
106: SoftMax classifier
200: discriminative learning unit
300: tumor detection unit

Claims (4)

A structure learning unit configured to similarly reconstruct the shape of the bone included in the input image to learn the structural representation of the bone to generate first learning information;
A discrimination learning unit receiving the first learning information and learning the tumor detection discrimination criteria to generate second learning information; And
The first learning information and the second learning information and a comparison of the received X-ray image to detect the presence and location of the bone tumor includes a tumor detection unit for generating bone tumor detection information,
The structure learning unit,
An encoder that extracts and recombines differential expressions of bone shapes included in the input image;
A decoder for storing and managing the extracted differential expression and masking a component for any one of the femur, tibia or fibula; And
SoftMax classifier that generates first learning information that classifies bone states by determining probability of bone state from masked components
Bone tumor detection system comprising a. According to claim 1,
The structure learning unit generates the first learning information through unsupervised learning or unsupervised learning,
The discrimination learning unit generates the second learning information through supervised learning or unsupervised learning,
The tumor detection unit is a bone tumor detection system characterized in that it generates the bone tumor detection information through ensemble semi-supervised learning (ESSW). According to claim 1,
The structure learning unit,
To receive the image including any one of the femur, tibia or fibula through the encoder, and reconstruct the shape of the bone included in the image received through the multiple convolutional layer and pooling layer to learn the structural representation of the bone Bone tumor detection system characterized in that. According to claim 1,
The SoftMax classifier,
A bone tumor detection system characterized by classifying bone tumors by calculating the probability of a bone condition from the output of the last layer of each of the encoder and decoder.

Images