KR102236340B1 - A method and apparatus for segmentation of Cartilages by using Bone-Cartilage Complex Modeling in MR images - Google Patents

Abstract

The present disclosure provides a method for segmenting cartilage using a bone-cartilage complex in a magnetic resonance image, in which a bone label and a predetermined cartilage label adjacent to the bone region are combined in a magnetic resonance image. Configuring Cartilage-Complex, BCC); Machine learning a first deep learning divider for segmenting the bone-cartilage complex using a plurality of training images and a second deep learning divider for segmenting the bone label, respectively; And dividing the cartilage label based on the difference between the bone-cartilage complex divided from the input image using the first deep learning divider and the bone label divided from the bone-cartilage complex using the second deep learning divider. It may include.

Description

A method and apparatus for segmentation of cartilages by using Bone-Cartilage Complex Modeling in MR images}

It relates to a method and apparatus for cartilage segmentation using bone-cartilage complex modeling in magnetic resonance images, and more particularly, in a method for segmenting and extracting cartilage using deep learning, bone-cartilage for improving the accuracy of segmentation of small-sized cartilage. The present invention relates to a method and apparatus for improved cartilage segmentation by applying complex modeling.

Knee arthritis is the leading cause of acquired disorders known worldwide. A recent study mentions that the lifetime risk of osteoarthritis of the knee is 40% and 47%, respectively, for men and women, and the risk increases to 60% if the BMI is 30 or higher. However, it is pointed out that the main causes of knee arthritis are mechanical wear of the knee cartilage and biochemical changes in the knee cartilage. Therefore, segmentation and extraction of knee cartilage in magnetic resonance images is used as an important pre-processing task for diagnosis, treatment planning, and prognosis of knee cartilage diseases including degenerative arthritis.

However, in magnetic resonance images, the knee cartilage appears in the form of a thin film surrounding the ends of the femur and tibia, and due to its thin thickness and small size, it is difficult to accurately segment and extract. In particular, conventionally, in the process of directly learning a cartilage label splitter, there is a limitation in that the accuracy of segmentation of cartilage that is relatively small compared to bone is poor, and in the case of a 3D image introduced for the evaluation of osteoarthritis, the complexity is further increased. Therefore, in order to improve the evaluation performance, automated cartilage segmentation is indispensable.

Korean Registered Patent Publication No. 10-1223681

A method and apparatus for segmenting cartilage using bone-cartilage complex modeling in a magnetic resonance image according to an embodiment is a multi-path segmentation network for semantic segmentation of a bone region and a bone-cartilage complex in a magnetic resonance image (hereinafter, RefineNet: Multi -Path Refinement) can be used to separate the cartilage area with high accuracy. In particular, it is possible to improve learning efficiency and improve cartilage segmentation accuracy by reducing size constraints by using bone-cartilage complex modeling in a conventional machine learning method for small-sized cartilage segmentation.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems that are not mentioned will be clearly understood by those skilled in the art from the following description.

A method for segmenting knee cartilage using bone-cartilage complex modeling in a magnetic resonance image according to an embodiment of the present invention is a bone-cartilage in which a bone label and a predetermined cartilage label adjacent to a bone region are combined in the learning image data, which is a magnetic resonance image. Constructing a complex (Bone-Cartilage-Complex, BCC); Machine learning a first deep learning divider for segmenting the bone-cartilage complex using a plurality of training images and a second deep learning divider for segmenting the bone label, respectively, wherein the plurality of training images is the training image data Including a first training image that is labeled with a bone label and used for learning the first deep learning splitter, and a second training image that is labeled with a bone-cartilage complex in the input image and used for learning the second deep learning splitter. Phosphorus, step; And segmenting the cartilage label based on the difference between the bone-cartilage complex divided from the input image using the first deep learning divider and the bone label divided from the bone-cartilage complex using the second deep learning divider, The input image includes a cartilage label segmentation step, which is a new magnetic resonance image in which cartilage extraction is performed.

In another embodiment, the bone-cartilage complex includes a femur and femur cartilage label, and a tibia and tibia cartilage label, and the bone label includes the femur label and the tibia label.

In another embodiment, in the machine learning step, in the three-dimensional image, which is a magnetic resonance image corresponding to the learning image data, learning by dividing each two-dimensional image in three directions of a coronal, axial, and sagittal plane. Then, the three result labels extracted from each 2D image are mixed in a majority voting method to extract a final 2.5D segmented label.

In another embodiment, in the step of segmenting the cartilage label, the bone label and the bone-cartilage complex region are segmented from the input image using a multipath segmentation network (RefineNet) algorithm for semantic segmentation.

An apparatus for segmenting knee cartilage using osteochondral complex modeling in a magnetic resonance image according to another embodiment of the present invention includes: at least one processor; And at least one memory in which instructions for causing the at least one processor to perform an operation when executed by the at least one processor are stored, and the operation performed by the at least one processor is An operation of constructing a bone-cartilage-complex (BCC) in which a bone label and a predetermined cartilage label adjacent to the bone region are combined; An operation of machine learning a first deep learning divider for segmenting the bone-cartilage complex using a plurality of training images and a second deep learning divider for segmenting the bone label, respectively; And an operation of dividing a cartilage label based on the difference between the bone-cartilage complex divided from the input image using the first deep learning divider and the bone label divided from the bone-cartilage complex using the second deep learning divider. Including, wherein the plurality of training images are labeled with a bone label in the training image data, and a bone-cartilage complex is labeled in the first training image used for learning the first deep learning divider and the input image, It includes a second training image used for learning the running splitter, and the input image is a new magnetic resonance image from which cartilage extraction is performed.

In another embodiment, the bone-cartilage complex includes femur and femur cartilage and tibia and tibia cartilage labels, and the bone label includes the femur label and the tibia label.

In another embodiment, the machine learning operation is performed by dividing each two-dimensional image in three directions of a coronal, axial, and sagittal plane in a three-dimensional image, which is a magnetic resonance image corresponding to the learning image data. , The final 2.5-dimensional segmentation label is extracted by mixing the three result labels extracted from each in a majority voting method.

In accordance with another embodiment of the present invention, a method for segmenting knee cartilage using bone-cartilage complex modeling in a magnetic resonance image includes: obtaining an input image including a bone-cartilage complex; And dividing a cartilage label based on a difference between the bone-cartilage complex divided in the input image and the bone label divided in the bone-cartilage complex, wherein the bone-cartilage complex uses a plurality of training images. Is divided by the learned first deep learning divider, the goal label is divided by a second deep learning divider learned using a plurality of training images, and the plurality of training images are divided into a goal label and Bone-Cartilage-Complex (BCC) to which a predetermined cartilage label adjacent to the bone region is bound is labeled.

In addition to this, another method for implementing the present invention, another system and a computer program for executing the method, and a computer-readable recording medium for recording the computer program may be further provided.

According to the present invention as described above, it has various effects as follows.

First, the knee cartilage division method according to an embodiment may improve the accuracy of knee cartilage division by using bone-cartilage complex division, thereby increasing the efficiency of diagnosing and tracking knee cartilage-related diseases.

In particular, it is possible to improve the extraction accuracy by applying a complex extraction method in a similar manner in the division of medical biological tissues, which are difficult to divide and extract due to their small size, in addition to the knee cartilage.

1 is a conceptual diagram illustrating a method of dividing cartilage according to an exemplary embodiment.
2 is a view for explaining the knee cartilage according to an embodiment.
3 is a flowchart illustrating a method of dividing cartilage according to an exemplary embodiment.
4 is a view for explaining in more detail a method of dividing knee cartilage using bone-cartilage complex modeling according to an embodiment.
5 to 7 are diagrams for explaining the accuracy of a cartilage segmentation method using a bone-cartilage complex according to an embodiment.
8 is a diagram schematically showing the internal configuration of a cartilage splitting device according to an embodiment.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments to be posted below, but may be implemented in various different forms, and only these embodiments make the posting of the present invention complete, and common knowledge in the technical field to which the present invention pertains. It is provided to completely inform the scope of the invention to those who have, and the invention is only defined by the scope of the claims. The same reference numerals refer to the same elements throughout the specification.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used with meanings that can be commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not interpreted ideally or excessively unless explicitly defined specifically.

The terms used in the present specification are for describing exemplary embodiments and are not intended to limit the present invention. In this specification, the singular form also includes the plural form unless specifically stated in the phrase. As used herein, “comprises” and/or “comprising” do not exclude the presence or addition of one or more other elements other than the mentioned elements.

1 is a conceptual diagram illustrating a cartilage segmentation method according to an exemplary embodiment.

Referring to FIG. 1, a cartilage splitting system according to an embodiment may include a cartilage splitting device 100 and a photographing device 120. In addition, the cartilage splitting device 100 and the photographing device 120 may be connected to the network 110.

First, the photographing apparatus 120 according to an embodiment photographs a magnetic resonance image. The magnetic resonance image (MR) photographed here may be a target image that requires segmentation of knee cartilage, or may be a training image for machine learning the cartilage segmentation apparatus 100. Here, the photographing apparatus 120 may mean a device capable of photographing bones and cartilage inside a human body, such as a device that photographs a magnetic resonance (MR) image.

In an embodiment of the present invention, the cartilage segmentation device 100 may be a computing device (eg, a server device) including a program for learning or analyzing a magnetic resonance image. For example, the cartilage segmentation apparatus 100 performs learning based on magnetic resonance image data to segment a bone region and a bone-cartilage complex, and divides a bone region and a bone-cartilage complex based on the learned result. It may be a computing device including a program for extracting a cartilage region through. In addition, for example, as the cartilage segmentation device 100 includes a learning model that has already been trained for segmentation of a bone region and a bone-cartilage complex, segmentation of a bone region and a bone-cartilage complex is performed based on the learning model. It may be one or more computing devices including a program for extracting a cartilage region through. Specifically, the process performed by the cartilage segmentation apparatus 100 may be performed by a processor (refer to FIG. 8) in the computing device. Specifically, when the cartilage segmentation apparatus 100 according to an embodiment is performed from the process of learning a magnetic resonance image, a multi-path segmentation network for semantic division of a bone region and a bone-cartilage complex in a magnetic resonance image (RefineNet : Multi-Path Refinement Networks for High-Resolution Semantic Segmentation) can be used to learn with high accuracy, and the cartilage area can be separated using the difference. In particular, it is possible to improve learning efficiency and improve cartilage segmentation accuracy by reducing size constraints using osteocartilage complex modeling in a conventional machine learning method for segmenting cartilage of a small size.

Therefore, not only can the efficiency of diagnosis and tracking of diseases related to knee cartilage be improved, but also the extraction accuracy is improved by applying a similar complex extraction method to the division of medical biological tissues, which are difficult to divide and extract due to their small size, in addition to knee cartilage. It can be improved.

Meanwhile, the network 110 according to an embodiment may include a wired network as well as a wireless network, and various means for transmitting an image captured by the photographing device 120 to the cartilage segmentation device 100 are all included. Can be understood as.

In addition, in another embodiment, the cartilage splitting apparatus 100 includes a first deep learning splitter for splitting a bone-cartilage complex and a second deep learning splitter for splitting the bone label. The first deep learning divider may be software (eg, a learning model or an analysis model) that divides a bone-cartilage complex from a new magnetic resonance image as the bone-cartilage complex learns a labeled magnetic resonance image. In addition, the second deep learning divider may be software (for example, a learning model or an analysis model) that divides a bone label from a new magnetic resonance image as it learns a magnetic resonance image labeled with a bone (i.e., a knee bone). have. In addition, for example, when the cartilage segmentation device 100 includes a plurality of computing devices, the first deep learning divider and the second deep learning divider may be included in other computing devices, or included in one computing device. May be.

2 is a view for explaining a knee bone and cartilage according to an embodiment.

2, between the femur (femur, 201) and the tibia (tibia, 202), the knee cartilage (Knee Cartilage, 203) including femoral cartilage and tibial meniscus is located, It distributes the load and protects the joints to move smoothly. Specifically, in the magnetic resonance image of FIG. 2 (b), a separate color (ie, chromatic colors such as purple, red, blue, yellow, etc.) is given and the displayed entire area corresponds to the knee cartilage (Knee Cartilage, 203). do.

However, referring to FIG. 2, in the magnetic resonance image, the knee cartilage 203 appears in the form of a thin film surrounding the ends of the femur 201 and the tibia 202, and it is difficult to accurately segment and extract because of its thin thickness and small size. In particular, conventionally, in the process of directly learning a cartilage label splitter, there is a limitation in that the accuracy of segmentation of cartilage that is relatively small compared to bone is poor, and in the case of a 3D image introduced for the evaluation of osteoarthritis, the complexity is further increased. Therefore, in order to improve the evaluation performance, automated cartilage segmentation is indispensable.

In particular, when a person manually divides cartilage, the cartilage has a thin, thin, and long curved structure that extends into several pieces, and it is difficult and subjective to divide it. Therefore, various methods have been proposed for automatic segmentation of cartilage. These include learning-based classifiers using handcrafted features and non-learning-based approaches that rely on statistical distributions of intensity or shape, such as active shapes/models, map-based methods, level sets, and graph cuts.

For example, magnetic resonance imaging for segmentation of osteoarthritis is an important process in diagnosing osteoarthritis and establishing treatment plans, and in various clinical tasks. Recently, deep segmentation networks (DSN) have been applied to cartilage segmentation, showing promising results. However, DSN has limitations in cartilage segmentation in that the network tends to ignore small objects such as cartilage while learning multi-class segmentation. Therefore, in bone-cartilage-complex (BCC) modeling according to an embodiment, DSN-based cartilage segmentation using a bone-BCC-difference (BCD) extraction method according to an embodiment If you do, you can overcome these limitations.

That is, in order to overcome the neglect of small cartilage in DSN, a bone-cartilage-complex (BCC) can be constructed that combines bone and cartilage in a single mask. Then, the BCC and the bone are divided, and the cartilage region can be extracted by subtracting the divided bone from the divided BCC. At this time, by applying 2.5D segmentation, the segmentation accuracy can be further improved by averaging multiple segmentation masks in multiple planes by majority vote. In particular, as will be described later in FIGS. 5 to 7, the cartilage segmentation method (BCD-Net) according to an embodiment obtained an average of 98.1% and 83.8% DSC in femur and tibia cartilage, respectively, and in the SKI10 public challenge verification data set The highest level of performance has been verified.

3 is a flowchart illustrating a method of dividing cartilage by the cartilage dividing apparatus 100 according to an embodiment of the present invention.

Referring to FIG. 3, in the cartilage segmentation method according to an embodiment of the present invention, in step S300, a bone label and the bone region in the learning image data, which is a magnetic resonance image, are A bone-cartilage-Complex (BCC) is formed in which adjacent predetermined cartilage labels are bound. Here, the bone-cartilage complex includes a femur and femur cartilage label, and a tibia and tibia cartilage label, and the bone label may mean a mask or region including the femur label and the tibia label. Of course, it is not limited to this configuration, and the bone label and the cartilage label are applied to all structures of the adjacent human body to constitute a bone-cartilage complex. In one embodiment, the training image data may be a previously acquired knee magnetic resonance image. In step S310, in the cartilage segmentation method, a first deep learning divider for segmenting a bone-cartilage complex using a plurality of training images and a second deep learning divider for segmenting a bone label, respectively, perform machine learning. In one embodiment, in the plurality of training images, a bone label is labeled in the training image data, and a bone-cartilage complex is labeled in the input image and a first training image used for learning the first deep learning divider. 2 It may include a second training image used for learning the deep learning divider.

For example, in the cartilage segmentation method according to an embodiment, the first deep learning divider and the second deep learning divider use a multipath segmentation network (RefineNet) algorithm for semantic segmentation described later in the description of FIG. By learning, it is possible to segment the bone-cartilage complex and the bone label from the input image.

In step S320, the cartilage segmentation method divides the cartilage label based on the difference between the bone-cartilage complex segmented using the first deep learning divider in the input image and the segmented bone label using the second deep learning segmenter. In this case, the input image is a new magnetic resonance image in which cartilage extraction is performed.

On the other hand, the cartilage segmentation method according to an embodiment learns by segmenting each two-dimensional image in three directions of coronal, axial, and sagittal instead of a three-dimensional image, and multiplying three result labels extracted from each two-dimensional image ( The final 2.5-dimensional division label can be extracted by mixing in the majority voting) method.

Therefore, the knee cartilage segmentation method according to an embodiment indirectly learns cartilage segmentation through bone-cartilage complex segmentation, but improves learning efficiency and improves cartilage segmentation accuracy by reducing the size constraint on cartilage that is relatively small compared to bone. have. In addition, a method for segmenting knee cartilage by a cartilage segmentation apparatus using bone-cartilage complex modeling in a magnetic resonance image according to an embodiment of the present invention includes: obtaining an input image including a bone-cartilage complex; And dividing the cartilage label based on the difference between the bone-cartilage complex divided from the input image using the first deep learning divider and the bone label divided from the bone-cartilage complex using a second deep learning divider. do. The first deep learning divider learns a plurality of training images to divide the bone-cartilage complex, the second deep learning divider learns a plurality of training images to divide a bone label, and the plurality of training images The image may be labeled with a bone-cartilage-Complex (BCC) in which a bone label and a predetermined cartilage label adjacent to the bone region are combined in a magnetic resonance image.

That is, the cartilage segmentation apparatus 100 may include a first deep learning divider and a second deep learning divider that have been trained based on a plurality of magnetic resonance images already labeled with a bone label or a bone-cartilage complex. At this time, the cartilage segmentation device 100 separates the bone label and the bone-cartilage complex from the newly acquired magnetic resonance image of the knee region based on the first deep learning divider and the second deep learning divider that have already been learned. , By calculating the difference, only the cartilage label can be segmented.

Hereinafter, a method and effect of splitting knee cartilage using bone-cartilage complex modeling according to an embodiment will be described in more detail with reference to FIGS. 4 to 7.

4, the cartilage segmentation method (BCD-Net) according to an embodiment includes (1) training two DSNs for BCCs and bone-only masks, and (2) cartilage masking with BCDs. (3) 2.5-dimensional segmentation by multi-view majority voting can be performed.

First, training two DSNs for each of the bone-cartilage complexes (BCCs) and bone-only masks may perform the following steps.

In order to transform the cartilage segmentation operation into two complex segmentation and extraction of the difference, masks for BCC, femur bone label, and tibia bone label were respectively constructed (401), and images of training set for two DSNs. Machine learning is repeatedly trained using. In the future, the trained DSN is used to segment more accurate cartilage when the test image 402 is input. Here, the BCC is constructed by creating a complex mask in which bone and cartilage are combined. Specifically, referring to the box image of step S403 for each training image 401 with a mask of manually divided femur (FB), tibia (TB), femoral cartilage (FC), and tibial cartilage (TC), femoral BCC Two BCCs {FBCC, TBCC} with (FBCC = FB + FC) and tibial BCC (TBCC = TB + TC) are shown. The box image of step S404 shows an example in which two goal label masks (FB and TB) are configured to train the DSN.

Accordingly, in steps S403 and S404, the two DSNs may be individually learned using two class data sets of BCC{FBCC, TBCC} and goal label mask {FB, TB}, respectively. Meanwhile, a DSN based on a convolutional neural network (CNN) structure can directly learn the mapping between an image and a segmentation mask in order to solve a high-resolution image classification problem. For example, Refine-net, one of the DSN models, can be used to learn the segmentation of BCC and goal label masks.

Next, in step S405, the cartilage mask is extracted using the bone-BCC-difference (BCD).

Specifically, DSN segmentation and BCD extraction are performed to extract a cartilage mask in the form of a difference between BCC and bone label. First, segmentation of the BCC and bone label mask for the test image is performed using two trained DSNs, respectively. For each test image, a cartilage mask is generated using the BCD obtained by subtracting the divided bone label mask {FB, TB} from the divided BCC {FBCC, TBCC}. Here, in order to further reduce the error in the division of femoral cartilage (FC = FBCC-FB) and tibia cartilage (TC = TBCC-TB), not all of the components of the BCD mask are used for calculation, but only the largest 3D connecting element It can be determined by the masks FC and TC.

In step S406, 2.5-dimensional segmentation may be performed according to a multi-view majority voting method.

In the case of 3D medical images, 3D segmentation is generally known to improve performance over 2D segmentation. However, since DSN learning for 3D images requires a lot of memory cost, 2.5-dimensional (2.5D) segmentation is used in many studies to reduce memory cost and improve accuracy. In particular, in the multi-view 2.5D segmentation, multiple 2D segmentation is performed on different image planes, and the segmentation mask is averaged by a majority vote as shown in step S406 of FIG. 4. In other words, the accuracy of BCD-Net can be further improved by performing multi-view 2.5-dimensional segmentation on three orthogonal planes including sagittal planes (S, sagittal), coronal planes (C, coronal), and axial planes (A, axial). have.

Meanwhile, BCD-Net segmentation for three planes {S, C, A} may be performed first. At this time, the parameters of BCD-Net on different image planes are all the same, and 2D division masks are accumulated for each plane to form a divided volume for each plane, and the three divided volumes are averaged by MV and finalized. A division mask 407 can be configured. If spatial information can be reflected in segmentation in this way, segmentation errors can be further reduced in 2.5-dimensional segmentation.

Hereinafter, the accuracy of a method for segmenting cartilage using a bone-cartilage complex according to an embodiment will be described with reference to FIGS. 5 to 7.

5 to 7 show experimental results in the SKI10 open challenge data set (FIG. 5A) for knee MRI cartilage segmentation in order to verify the effectiveness of the cartilage segmentation method according to an embodiment. The SKI10 dataset provides 100 knee MRI images consisting of 60 training and 40 validation images. In all images, the data set provides four classes of manual segmentation masks: FB, TB, FC and TC. Here, for 40 verification images, the data set provides the evaluation ROI mask. This data set provides mostly 1.5T T1-weighted sagittal MRI scans with an average voxel size of 0.4 mm x 0.4 mm x 1 mm and an average resolution of 290 x 340 x 110. In the experiment, the DSN is 60 training images and segmentation masks, and can have the number of 2D slices of 13,012, 13,439 and 16,407 in the sagittal, coronal, and axial planes, respectively.

The performance of the cartilage segmentation method (BCD-Net) according to an embodiment is evaluated quantitatively as well as qualitatively. First, in order to verify the BCD according to an embodiment, ReineNet learned in four classes of FB, TB, FC, and TC is shown (Fig. 5(b)). And, in order to confirm the 2.5D segmentation effect, three 2D segmentation results (S, C, A) and 2.5D segmentation results (MV) are also compared (Fig. 5(c)).

In addition, a BCD-Net (MV6) method of averaging a total of 6 division masks including three 2D RefinNets and three 2D BCD-Nets through a number of votes was also performed (Fig. 5(d)). As shown in FIG. 5, the qualitative evaluation of the segmentation result can be performed visually, and it can be seen that the segmentation method according to an embodiment has improved segmentation accuracy.

Next, in the quantitative evaluation, the Dice Similarity Coefficient (DSC) for the four grades was calculated and compared. We also evaluated the challenge delivery criteria, including mean symmetric surface distance (RMSD), volume overlap error (VOE), and mean error to compare the results of SKI10 participants with those of SKI10 participants.

Referring to FIG. 5, as can be seen in the first and third rows, FC tends to be under-differentiated in the RefineNet results, but it can be seen that such under-differentiation is largely compensated for in the BCD-Net according to an embodiment. Likewise, as indicated in the second and third rows, it can be seen that the segment is lost due to excessive subdivision of TB in the RefineNet result, while TC is mostly corrected in the BCD-Net according to an embodiment. That is, when the DSN performs machine learning on the existing 4 classes, the DSN tends to ignore the small cartilage that distinguishes bone and cartilage, resulting in a segmentation error. However, in the BCD-Net according to an embodiment, since there is no collision between the bone and cartilage classes, it can be seen that it is relatively robust to a segmentation error due to cartilage neglect. Looking at (c) and (d) of FIG. 5 to describe the improvement result of BCD-Net according to an embodiment, the results of BCD-Net (MV) and BCD-Ne (MV6) are shown in (e) of FIG. It can be seen that it is most visually similar to the manually divided correct answer value shown in FIG.

Meanwhile, FIG. 6 shows the comparison result of DSC (Dice similarity coefficients) according to the experiment as a numerical value. From the 2D RefineNet results, it can be observed that (1) DSCs of TB and TC are generally lower than those of FB and FC, and (2) the performance between planes can be aligned with S>C>A. In addition, in the 2D BCD results, (1) DSC of TB and TC generally improved than RefinNet, but FB and FC did not, and (2) performance in the coronal plane was significantly improved than ReineNet, and those in the sagittal and axial planes were You can see that it is not. It can be analyzed that the BCD-Net according to an embodiment improves the accuracy of relatively small structures such as TB and TC thanks to BCD extraction. On the other hand, 2.5D segmentation shows that bone and cartilage segmentation was improved overall in both RefineNet and BCD-Net methods. Finally, BCD-Net (MV6) achieved the best DSC for all mask classes, including RefinNet and BCD-Net results.

7 shows performance evaluation and comparison of the SKI10 challenge verification data set. Referring to Figure 7, in cartilage segmentation, BCD-Net (MV) and BCD-Net (MV6) can not only surpass other conventional DSNs such as SegNet and U-Net, but also achieve cutting-edge accuracy of FC and TC extraction. Can be seen. In bone segmentation, our method exhibits a state-of-the-art level of ASD (mm), but a slightly higher RMSD (mm) for FB and TB, and can be considered to be improved by supplementing the additional post-processing of bone masks in previous DSN-based methods have. In addition, it can be seen that (1) DSN is very effective in cartilage segmentation, and (2) BCD-Net according to an embodiment can improve the performance of DSN.

8 is a diagram schematically showing the internal configuration of a cartilage splitting device according to an embodiment.

The cartilage splitting apparatus 100 is a device capable of performing a function to be described later, and may be configured with, for example, a server computer, a personal computer, or the like. In an embodiment, the cartilage segmentation apparatus 100 may include one or more processors 130 and/or one or more memories 140.

According to an embodiment, an operation performed by one or more processors 130 is a bone-cartilage complex in which a bone label and a predetermined cartilage label adjacent to the bone region are combined in the training image data, which is a magnetic resonance image. -Cartilage-Complex, BCC), a first deep learning divider for segmenting the bone-cartilage complex using a plurality of training images, and a second deep learning divider for segmenting the bone label, respectively. And an operation of dividing a cartilage label based on a difference between the bone-cartilage complex divided from the input image using the first deep learning divider and the bone label divided using the second deep learning divider. have. In one embodiment, the plurality of training images are labeled with a bone label in the training image data, and a bone-cartilage complex is labeled in the first training image used for learning the first deep learning splitter and the input image, and the second It may include a second training image used for learning the deep learning divider. In addition, the input image is a new magnetic resonance image in which cartilage extraction is performed. That is, when it is necessary to extract the cartilage region from the magnetic resonance image of a new patient, the magnetic resonance image of the new patient may be input to the cartilage segmentation apparatus as an input image.

Of course, it is not limited to these operations, and it is obvious to a person skilled in the art that the cartilage segmentation apparatus 100 according to an exemplary embodiment can perform the cartilage segmentation methods described above in FIGS. 1 to 7.

In one embodiment, at least one of these components of the cartilage splitting device 100 may be omitted, or another component may be added to the cartilage splitting device 100. In addition, additionally or alternatively, some components may be integrated and implemented, or may be implemented as a singular or plural entity. At least some of the components inside and outside the cartilage segmentation apparatus 100 are connected to each other through a bus, general purpose input/output (GPIO), serial peripheral interface (SPI), mobile industry processor interface (MIPI), etc. As a result, data and/or signals can be exchanged.

One or more memories 140 may store various types of data. Data stored in the memory 140 is data acquired, processed, or used by at least one component of the cartilage segmentation apparatus 100, and may include software (eg, a program). The memory 140 may include volatile and/or nonvolatile memory. The one or more memories 140 may store instructions for causing the one or more processors 130 to perform an operation when executed by the one or more processors 130. In one embodiment, the one or more memories 140 may store personalized information for one or more users and/or recommendation information for one or more products. In the present disclosure, programs or commands are software stored in the memory 140, and various functions are applied to the application so that the operating system, application, and/or application for controlling the resources of the cartilage division apparatus 100 can utilize the resources of the device. It may include middleware and the like to be provided.

The one or more processors 130 may control at least one component of the cartilage splitting apparatus 100 connected to the processor 130 by driving software (eg, a program or command). In addition, the processor 130 may perform various operations related to the present disclosure, processing, data generation, and processing. In addition, the processor 130 may load data, etc. from the memory 140 or may store the data in the memory 140.

In one embodiment, the cartilage segmentation apparatus 100 may further include a communication interface (not shown). The communication interface may perform wireless or wired communication between the cartilage segmentation apparatus 100 and another server or another external device (eg, the photographing apparatus 120). For example, the communication interface is eMBB (enhanced mobile broadband), URLLC (Ultra Reliable Low-Latency Communications), MMTC (Massive Machine Type Communications), LTE (long-term evolution), LTE-A (LTE Advance), UMTS ( Universal Mobile Telecommunications System), GSM (Global System for Mobile communications), CDMA (code division multiple access), WCDMA (wideband CDMA), WiBro (Wireless Broadband), WiFi (wireless fidelity), Bluetooth, NFC (near field) communication), a global positioning system (GPS), or a global navigation satellite system (GNSS).

Various embodiments of the cartilage splitting apparatus 100 according to the present disclosure may be combined with each other. Each of the embodiments may be combined according to the number of cases, and embodiments of the cartilage splitting apparatus 100 made by combining are also within the scope of the present disclosure. In addition, the internal/external components of the cartilage splitting apparatus 100 according to the present disclosure described above may be added, changed, replaced, or deleted according to embodiments. In addition, the internal/external components of the aforementioned cartilage splitting apparatus 100 may be implemented as hardware components.

Meanwhile, the method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the present invention, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of the program instructions may include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

So far, the present invention has been looked at around its preferred embodiments. Those of ordinary skill in the art to which the present invention pertains will be able to understand that the present invention may be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered from an illustrative point of view rather than a limiting point of view. The scope of the present invention is shown in the claims rather than the above description, and all differences within the scope equivalent thereto should be construed as being included in the present invention.

100: cartilage segmentation device
110: network
120: photographing device

Claims (10)

In a method for segmenting cartilage using a bone-cartilage complex in a magnetic resonance image by a computer,
Constructing a bone-cartilage-complex (BCC) in which a bone label and a predetermined cartilage label adjacent to the bone region are combined from the learning image data, which is a magnetic resonance image;
Machine learning a first deep learning divider for segmenting the bone-cartilage complex and a second deep learning divider for segmenting the bone label using a plurality of training images labeled with a bone label from the training image data, respectively. ; And
The cartilage label is divided based on the difference between the bone-cartilage complex divided from the input image using the first deep learning divider and the bone label divided from the bone-cartilage complex using the second deep learning divider, The input image comprises a cartilage label segmentation step, which is a new magnetic resonance image in which cartilage extraction is performed. The method of claim 1,
The bone-cartilage complex comprises a femur and femur cartilage label, and a tibia and tibia cartilage label, and the bone label comprises the femur label and the tibia label. The method of claim 1,
The plurality of training images are labeled with a bone label in the training image data, and a bone-cartilage complex is labeled in the first training image and the input image to be used for learning the first deep learning divider. A method comprising a second training image to be used. The method of claim 1,
The machine learning step includes learning by dividing each two-dimensional image in three directions of a coronal, axial, and sagittal plane from a three-dimensional image, which is a magnetic resonance image corresponding to the learning image data, and extracting from each two-dimensional image. The method of extracting a final 2.5-dimensional segmentation label by mixing the three result labels in a majority voting method. The method of claim 1,
The cartilage label segmentation step is to segment the bone label and the bone-cartilage complex region in the input image using a multi-path segmentation network (RefineNet) algorithm for semantic segmentation. One or more processors; And
When executed by the at least one processor, the at least one processor includes at least one memory that stores instructions that cause the at least one processor to perform an operation,
The operation performed by the one or more processors,
An operation of constructing a bone-cartilage-complex (BCC) in which a bone label and a predetermined cartilage label adjacent to the bone region are combined from the learning image data, which is a magnetic resonance image;
Machine learning a first deep learning divider for segmenting the bone-cartilage complex and a second deep learning divider for segmenting the bone label using a plurality of training images labeled with a bone label from the training image data; And
An operation of dividing a cartilage label based on the difference between the bone-cartilage complex divided from the input image using the first deep learning divider and the bone label divided from the bone-cartilage complex using the second deep learning divider. Including, wherein the input image is a new magnetic resonance image in which cartilage extraction is performed, a cartilage segmentation apparatus using a bone-cartilage complex. The method of claim 6,
The bone-cartilage complex includes femur and femur cartilage and tibia and tibia cartilage label, and the bone label includes the femur label and the tibia label. The method of claim 6,
The machine learning operation includes learning by dividing each two-dimensional image in three directions of a coronal, axial, and sagittal plane from a three-dimensional image, which is a magnetic resonance image corresponding to the learning image data, and extracting from each two-dimensional image. The cartilage segmentation device to extract the final 2.5-dimensional segmentation label by mixing the three result labels in a majority voting method. In a method for segmenting cartilage using a bone-cartilage complex in a magnetic resonance image by a computer,
Obtaining an input image including a bone-cartilage complex; And
Including the step of dividing a cartilage label based on a difference between the bone-cartilage complex divided in the input image and the bone label divided in the bone-cartilage complex,
The bone-cartilage complex is divided by a first deep learning divider learned using a plurality of training images,
The goal label is divided by a second deep learning divider learned using a plurality of training images,
The plurality of training images is a bone-cartilage complex (Bone-Cartilage-Complex, BCC), in which a bone label and a predetermined cartilage label adjacent to a bone region are combined in a magnetic resonance image, is labeled. A cartilage segmentation program using a bone-cartilage complex in a magnetic resonance image, which is combined with a computer as hardware and stored in a medium to execute the method of any one of claims 1 to 5 and 9.

Images