KR102314455B1 - A Method and Apparatus for Segmentation of Orbital Bone in Head and Neck CT image by Using Deep Learning and Multi-Graylevel Network - Google Patents

Abstract

The orbital bone segmentation method in head and neck computed tomography (CT) includes a first label composed of a region of a first brightness value in the input image and a region of a second brightness value lower than the first brightness value. a first label and a second label setting step of setting the configured second label; and dividing the orbital bone by integrating the divided first label and the second label, wherein the first label and the second label are each using a network considering a training image and multiple brightness values. It is divided by the machine-learned first and second dividers.

Description

{A Method and Apparatus for Segmentation of Orbital Bone in Head and Neck CT image by Using Deep Learning and Multi-Graylevel Network}

The present invention relates to a method and apparatus for segmenting orbital bones in three-dimensional head and neck computed tomography (CT), and more particularly, deep learning in three-dimensional head and neck computed tomography (CT). and a method and apparatus for segmenting an orbital bone using a multi-luminance value network.

Segmentation and extraction of orbital bone from head and neck computed tomography images can be used as an important pre-processing operation for diagnosing orbital bone fractures and manufacturing plates required for reconstruction of fractured sites.

In the head and neck computed tomography image, the shape of the orbital bone varies from person to person, and the orbital bone consists of a thick bone area with high brightness values and a thin bone area with low brightness values similar to surrounding soft tissues. Since it shows a value, it is difficult to accurately divide and extract it by automatically learning it.

In particular, in the conventional machine learning method, in the process of learning the entire orbital bone to divide the entire orbital bone, there is a limitation in that the segmentation accuracy of the thin bone region is relatively poor.

The present invention discloses a method of segmenting an orbital bone using deep learning and a multi-brightness value network in a three-dimensional head and neck computed tomography (CT) image according to an embodiment. In particular, in the deep learning process, the cortical bone region having a relatively high brightness level and the thin bone and trabecular bone regions in the orbit inner wall and orbit bottom region having a low brightness level are machine-learned separately. Disclosed is a method for improving segmentation accuracy for thin bone regions in the orbital inner wall and orbital floor region.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

In the method for segmenting orbital bones in computed tomography (CT) of the head and neck according to an embodiment of the present invention, a first label composed of a region of a first brightness value in the input image and the first brightness value a first label and a second label setting step of setting a second label composed of an area having a low second brightness value; and dividing the orbital bone by integrating the divided first label and the second label, wherein the first label and the second label are each using a network considering a training image and multiple brightness values. It is divided by the machine-learned first and second dividers.

In another embodiment, the method further includes a pre-processing step of normalizing the brightness value and the pixel size of the input image.

Also, in another embodiment, performing the preprocessing includes changing a brightness value within a predetermined range into an unsigned 8-bit integer value for normalizing the brightness value.

Further, in another embodiment, the first label includes a cortical bone region of the orbital bone, and the second label includes a thin bone region of the inner wall of the orbit and the bottom of the orbit.

Also, in another embodiment, the first divider is machine-learned using a single brightness value network (SGB-Net) corresponding to the first label for the first label, and the second divider is For two labels, it is machine learning using a single brightness value network corresponding to the second label.

Also, in another embodiment, the single brightness value network (SGB-Net) generates a connection path and an extension path having six resolution steps, and parametric rectification for each resolution step in the connection path. Perform two 3X3 convolutions involving a linear unit (PReLU) and a 2x2 max-pooling operation with a stride of 2, and calculate the parametric rectified linear unit for each resolution step in the extension path. The following two 3X3 convolutions are performed, and 2x2 deconvolution with double amplitude is performed for upsampling for each resolution step.

In addition, in another embodiment, the single brightness value network (SGB-Net) labels the number of output channels of the image on which upsampling through deconvolution is performed in all resolution stages of the extension path. A last layer that performs 1x1 convolution to reduce the number of labels is included, and the last layer includes a softmax function that provides a probability of an output split label.

Also, in another embodiment, the second divider includes a 2D division module and a 3D division module, and the first label and second label setting step comprises: converting the input image into the 2D division module and the 3D division module. each input to the division module; and generating the second label by combining the 2D divided data and the 3D divided data divided by the 2D division module and the 3D division module).

In another embodiment, the 2D division module is a RefineNet-based 2D division network, and the 3D division module is a U-Net based 3D division network.

In another embodiment, in the step of setting the first label and the second label, the three-dimensional division module includes a two-dimensional area obtained from the two-dimensional division module, and is extracted from a three-dimensional input image. It is characterized in that a three-dimensional area is input.

Orbital bone dividing apparatus according to another embodiment of the present invention, one or more processors; and one or more memories storing instructions that, when executed by the one or more processors, cause the one or more processors to perform an operation, wherein the operation performed by the one or more processors comprises: a first brightness value in the input image an operation of setting a first label composed of an area of , and a second label composed of an area of a second brightness value lower than the first brightness value; and an operation of segmenting the final orbital bone by integrating the divided first label and the second label, wherein the first label and the second label are machined using a network considering a training image and multiple brightness values, respectively. It is divided by the learned first divider and the second divider.

In addition, the first label may include a cortical bone region of the orbital bone, and the second label may include a thin bone region of the orbital inner wall and orbital floor.

In addition to this, another method for implementing the present invention, another system and a computer program for executing the method, and a computer-readable recording medium for recording the computer program may be further provided.

According to the present invention as described above, it has various effects as follows.

According to an embodiment, the method and apparatus for segmenting orbital bone in a head and neck image may improve the accuracy of plate manufacturing required for diagnosis of an orbital bone fracture and reconstruction of a fracture site through improved segmentation and extraction of orbital bone. Furthermore, in performing segmentation on bone structures having various brightness values in various computed tomography images of a human body, it is possible to improve segmentation accuracy by applying a learning technique of multiple brightness values in a similar manner.

1 is a conceptual diagram for explaining a method of segmenting an orbital bone in a head and neck computed tomography image according to an embodiment.
2 is a diagram for explaining orbital bone segmentation in a head and neck computed tomography image according to an embodiment.
3 is a flowchart illustrating a method of segmenting an orbital bone according to an embodiment.
4 is a conceptual diagram illustrating a method of performing machine learning in a multi-brightness value network according to an embodiment.
5 and 6 are diagrams for explaining the effect of the orbital bone segmentation method according to an embodiment.
7 is a block diagram of an orbital bone division apparatus including a 2D division module and a 3D division module in the second divider according to another embodiment of the present invention.
8 is a flowchart of a method of segmenting a thin bone in a new input image by using a 2D segmentation network and a 3D segmentation network together according to another embodiment of the present invention.
9 and 10 are diagrams for explaining the effect of the orbital bone segmentation method using a second divider including a 2D segmentation network and a 3D segmentation network according to another embodiment of the present invention.
11 is a diagram schematically showing the internal configuration of an orbital bone dividing device according to an embodiment.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments published below, but can be implemented in various different forms, and only these embodiments make the publication of the present invention complete, and common knowledge in the art to which the present invention pertains It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout.

Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used with the meaning commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless clearly defined in particular.

In this specification, a 'label' means a specific area set in an image. For example, the label may mean a region directly set by a clinician, or a region divided by a machine-learning deep learning algorithm.

In this specification, the 'first label' is used for learning the first divider, and means that it is extracted from a head and neck computed tomography image.

In the present specification, the 'second label' is used for second divider learning, and means that it is extracted from a head and neck computed tomography image.

In this specification, 'image for learning' refers to image data constructed to train a machine learning model.

In the present specification, an 'input image' refers to image data for which a desired range extraction is to be performed through a machine learning model that has been trained.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. As used herein, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components.

1 is a conceptual diagram illustrating a method of segmenting an orbital bone in a head and neck computed tomography image according to an embodiment.

Referring to FIG. 1 , an orbital bone segmentation system in a head and neck computed tomography image according to an embodiment may include an orbital bone segmentation apparatus 100 and an imaging apparatus 120 . In addition, the orbital bone segmentation device 100 and the imaging device 120 may be connected to the network 110 .

First, the photographing apparatus 120 according to an embodiment serves to capture and acquire a head and neck image. Here, the imaging device 120 may refer to a device for imaging the inside of a human body in order to diagnose a disease, such as an X-ray or a computed tomography (CT) device. The imaging device 120 may generate a training image for machine learning or an input image that needs to segment the orbital bone.

The orbital bone segmentation apparatus 100 according to an embodiment performs a role of segmenting the orbital bone using deep learning and a multi-brightness value network in the head and neck computed tomography image captured by the photographing apparatus 120 . . In particular, the orbital bone segmentation apparatus 100 performs machine learning on a cortical bone region having a relatively high brightness level in the deep learning process for a head and neck computed tomography image, and an orbital inner wall having a low brightness level. Because machine learning is performed on the thin bone region in the and orbital floor region, that is, the cortical bone region and the thin bone region are separated and machine-learned, so the segmentation accuracy for the thin bone region of the trabecular bone is improved. can do. Here, when machine learning is performed by separating the thin bone area in the orbital inner wall and the orbital floor area, the trabecular bone area having a low brightness level may be separated together for machine learning.

That is, the orbital bone segmentation apparatus 100 learns about a cortical bone region having a relatively high brightness level in the head and neck computed tomography image, and then divides the cortical bone region in the new image ( After learning about the first segmentation part (or first divider) 101 to be segmented and the thin bone region in the orbit inner wall and orbit floor region having a low brightness level, thin bone (thin bone) in the new image It may include a second division part (or a second divider) 102 for segmenting the bone) region. A detailed description of the detailed learning model of the first division unit 101 and the second division unit 102 will be described later.

In one embodiment, the orbital bone dividing device 100 may be a separate computing device (eg, a server device or a user client device) physically separated from the imaging device, or may be a physically single device with the imaging device. have. For example, when software including an orbital bone segmentation algorithm (that is, an orbital bone segmentation model for which pre-learning is completed) is built-in or installed in the user's computer, the user's computer corresponds to the orbital bone segmentation apparatus 100, and photographing The device directly receives the computed tomography image of the head and neck through the network to perform segmentation of the orbital bone. In addition, for example, when the orbital bone segmentation apparatus includes a learning model that has been trained based on existing image data, the orbital bone segmentation apparatus 100 is a head and neck computer that is manufactured and photographed as a single device with the photographing apparatus. The result of separation from the tomography image can be transmitted to the user's computer.

The network 110 according to an embodiment may include a wired network as well as a wireless network, and various means for transmitting the image captured by the imaging device 120 to the orbital bone segmentation device 100 are all included. can be understood

In addition, in another embodiment, the orbital bone segmentation apparatus 100 includes a first divider 101 for dividing a first label composed of a region of a first brightness value in an input image and a brightness value lower than the first brightness value. and a second divider 102 for dividing a second label composed of an area of . The first divider may be software (eg, a learning model or an analysis model) that divides the first label in a new MR image as the cortical bone region learns the labeled MR image. In addition, the second divider may be software (eg, a learning model or an analysis model) that divides the second label in a new MR image as it learns the MR image in which the thin bone region is labeled. Also, for example, when the orbital bone dividing apparatus 100 includes a plurality of computing devices, the first divider and the second divider may be included in another computing device or may be included in one computing device.

2 is a diagram for explaining orbital bone segmentation in a head and neck computed tomography image according to an embodiment.

As shown in (a) of Figure 2, the orbital wall (orbital wall) (hereinafter referred to as orbital wall) surrounding the orbit is the orbital lateral wall (orbital lateral wall, 201), orbital roof (orbital roof, 202), orbital inner wall ( Orbital medial wall, 203), and the orbital floor (orbital floor, 204), consisting of a pyramidal bone structure, including, and protects the eye and adjacent nerves to the eye. In addition, the orbital wall is composed of cortical bone of high brightness value and trabecular bone and thin bone of low brightness value. On the other hand, an orbital fracture to be confirmed through the segmentation of the orbital bone composed of cortical bone and thin bone is one of the most common fractures, and in particular frequently occurs in the orbital inner wall 203 and orbital floor 204 . That is, in cranio-maxillofacial surgery for orbital wall reconstruction following orbital bone fracture, segmentation of the orbital bone is essential to support the position of the eyeball and restore the volume and shape of the orbit.

Meanwhile, a three-dimensional CT (3D CT) as shown in FIG. However, since the thin bones of the orbital inner wall 203 and the orbital floor 204 have low brightness values similar to those of neighboring soft tissues in the input image, as shown in FIG. Splitting bones is difficult. Here, the arrow points to the cortical bone and the triangle points to the thin bone.

Also, (c) of FIG. 2 shows that when a cortical bone having a high brightness value is divided using a threshold value of 300HU (Hounsfield Unit), the trabecular bone and thin bone having a low brightness value are hardly divided. On the other hand, Fig. 2(d) shows that when the threshold value is lowered to 50HU to divide the trabecular and thin bones having low brightness values, leakage into the soft tissue around the orbital bone occurs. The arrows shown in (c) and (d) of Fig. 2 indicate the inner wall of the orbit, and the triangle indicates the orbital floor region.

For surgery to reconstruct the orbital wall, segmentation of the orbital bone to restore the volume and shape of the orbit that maintains the position of the eyeball must be preceded. However, since the orbital bone has a broad spectrum of brightness values in CT images, it was difficult to accurately segment the thin bones of the orbital medial wall and orbital floor, which have weak brightness values, using the conventional U-Net-based segmentation method. Therefore, the orbital bone segmentation method according to an embodiment is a multi-layered method to improve segmentation accuracy of cortical bones having high intensity values and thin bones having low intensity values in CT images of the head and neck. A network (MGB-Net: multi-gray-bone network) using the brightness level is used. Specifically, the orbital bone segmentation device 100 uses a single orbital bone mask to prevent under-segmentation of the thin bones of the medial wall and the floor of the orbit. Convert to two masks for bones. Then, the orbital bone segmentation apparatus 100 trains two single brightness level networks (SGB-Net: Single-Gray-Bone-Net) for each mask, and then, in the input image, each cortical bone and thin Segment the entire orbital bone by integrating the results of segmentation on the bone mask.

Hereinafter, an operation of the orbital bone segmentation apparatus 100 and a method of segmenting an orbital bone in a head and neck computed tomography image will be described in detail with reference to FIGS. 3 and 4 .

3 is a flowchart illustrating a method of segmenting an orbital bone according to an embodiment.

Each step to be described below with reference to FIG. 3 is performed by the processor 130 of the orbital bone dividing apparatus 100 shown in FIG. 11 . For example, the orbital bone segmentation apparatus 100 may be one or more computers. That is, the orbital bone dividing device 100 may be a server device or a client device used by a user.

In step S300, the orbital bone segmentation apparatus 100 may perform pre-processing for normalizing the brightness value and the pixel size of the input image. In order to reduce protocol differences in various input images, normalization can be performed on brightness values and pixel sizes. For example, in order to normalize the brightness value, a brightness value within a predetermined range may be changed to an unsigned 8-bit integer value. It is also possible to extract a region of interest for extracting orbital bones from a head and neck computed tomography image. At this time, for observation of the orbital bone, the brightness value may be normalized based on the orbital bone.

In step S310, the orbital bone segmentation apparatus 100 may set a first label including an area having a relatively high brightness value and a second label including an area having a relatively low brightness value in the input image. Here, the first label may include a cortical bone region of the orbital bone, and the second label may include a thin bone region of the inner wall of the orbit and the bottom of the orbit. On the other hand, in general, the cortical bone of the orbital bone has a brightness value of 300 HU or more, and the thin bones of the orbital inner wall and the orbital floor have a brightness value of 200 HU or less. Accordingly, in order to complement the two masks, the first and second labels may be set based on the brightness values greater than 200HU and lower than 300HU by overlapping the brightness values of the masks of the cortical bone and the thin bone.

As an embodiment of the method for setting the first label and the second label, the orbital bone segmentation apparatus 100 is a first divider machine-learned using a training image (ie, an image for learning) and a network in consideration of multiple brightness values, respectively. and dividing the first label and the second label through a second divider, wherein machine learning may include performing a deep learning algorithm using a convolutional neural network network.

For example, the first divider performs machine learning for the first label using a single brightness value network (SGB-Net) suitable for the first label, and the second divider applies the second label to the second label. Machine learning can be performed using a suitable single brightness value network. That is, the network considering multiple brightness values (MSB-Net) according to an embodiment may mean a network in which a single brightness value network for each of the first label and the second label is combined. For example, the orbital bone segmentation apparatus 100 may include a plurality of head and neck computed tomography images (or head and neck computed tomography images on which pre-processing has been performed) in which the first label and the second label are set by the medical staff in the training image data. A process of learning the first divider and the second divider is performed using

Specifically, the process in which machine learning is performed using a single brightness value network (SGB-Net) is as follows. Similar to the conventional U-Net algorithm, a connected path and an extended path each having six resolution steps are generated, and in the connected path, two 3X3 Perform 2x2 max-pooling operation with convolution and stride equal to 2, and perform two 3x3 convolutions involving the parametric rectified linear unit at each resolution step in the extension path, each For resolution step-by-step upsampling, it may include a process of performing 2x2 deconvolution having a double amplitude. In addition, the single brightness value network (SGB-Net) reduces the number of output channels of an image subjected to upsampling through deconvolution to the number of labels at all resolution stages of the extension path. A last layer for performing a solution may be included, and the last layer may include a softmax function and a loss function that provide a probability of an output segmentation label.

In step S320, the orbital bone segmentation apparatus 100 may obtain the final orbital bone by integrating the divided first label and the second label. That is, the orbital bone segmentation device 100 uses the orbital bone segmentation model in which learning is completed based on the head and neck computed tomography image from which the first label and the second label are extracted and the first label (i.e., the first label in the input image). , the cortical bone region) and the second label (ie, the thin bone region including the trabecular bone) are divided, and the entire orbital bone is divided by combining them.

Accordingly, the method and apparatus for segmenting orbital bone in a head and neck image according to an embodiment may improve the accuracy of plate manufacturing required for diagnosis of an orbital bone fracture and reconstruction of a fracture site through improved segmentation and extraction of orbital bone. . Furthermore, in performing segmentation on bone structures having various brightness values in various computed tomography images of the human body, it is possible to improve segmentation accuracy by applying a learning technique of multiple brightness values in a similar manner. 4 is a conceptual diagram illustrating a method of performing machine learning in a multi-brightness value network according to an embodiment.

Referring to FIG. 4 , in the pre-processing step 502 , the original image 501 may be subjected to normalization of brightness values and pixel sizes in the input image in order to reduce the occurrence of differences between input images due to different acquisition protocols. have.

For normalization 503 of the brightness value, the brightness value is changed to an unsigned 8-bit integer value [0, 255] in the range [-200HU, 400HU]. Here, a region having a brightness value greater than 400HU may mean a certain cortical bone region, and a region having a brightness value lower than -200HU may mean air and fat.

Also, for pixel spacing normalization 504, the pixel size of all images equalizes the other pixel sizes of the data to the smallest pixel size among all data, 0.41x0. It can be changed to ^{41mm 2 .}

In addition, after preprocessing is performed on the original image 501 , the ROI image 505 may be generated by being cropped to a predetermined region of intent (ROI) region.

Meanwhile, the ROI image may be divided into a training image 506 or a set of input images 507 for actual orbital bone segmentation. Next, the input image can be machine-learned through a multi-luminance value network for thin bone regions with low brightness values and cortical bone regions with high brightness values, respectively.

Here, the multi-luminance value network (MGB-Net) is developed from the deep neural network, which is well known as the conventional U-Net algorithm. First, a multi-luminance value network is configured in a form including at least one single brightness value network (SGB-Net, 508, 509). The single brightness value network (SGB-Net) consists of a concatenating path and an expanding path, and each path consists of 6 layers. Each layer of the concatenated path consists of two convolution filters and a 2x2 max-pooling operation with a stride of 2, and each layer of the extension path consists of two convolution filters and a 2x2 upsampling operation. All convolutions have a kernel size of 3x3 and involve zero padding and a Rectified Linear Unit (ReLu). The last layer of the extension path consists of two filters {object, background} of a 1x1 convolution and a softmax function that provides the probability of the segmentation label. Also, the loss function may be calculated as the sum of pixel-wise cross entropy and dice coefficients.

Meanwhile, the input image 507 converts one orbital bone mask into two masks for cortical bone and thin bone in order to prevent under-segmentation of the orbital lining and thin bones of the orbital floor during the integration step. can be Therefore, in the machine learning step, two SGB-Nets perform deep learning (508, 509) on thin bone masks and cortical bone masks, respectively, respectively. That is, the two SGB-Nets are each trained on two types of data sets. For example, a cortical bone mask may include cortical bone and background, and a thin bone mask may include thin bone and background.

In the integration step 510 , the results of segmenting the cortical bone 511 and the thin bone 512 through the respective SGB-Nets are integrated 513 to obtain a final orbital segmentation result 520 .

On the other hand, in general, the cortical bone of the orbital bone has a brightness value of 300 HU or more, and the thin bones of the orbital inner wall and the orbital floor have a brightness value of 200 HU or less. Therefore, in order to complement the two masks, by superimposing the brightness values of the masks of the cortical bone and the thin bone, the final orbital bone segmentation result can be set to a brightness value greater than 200HU and lower than 300HU.

5 and 6 are diagrams for explaining the effect of the orbital bone segmentation method according to an embodiment.

5 shows qualitative and quantitative performance of a network using multiple brightness value levels according to an embodiment.

The segmentation performance evaluation results of the entire orbital bone ((a)-(e)) and the thin bones of the inner orbital wall and orbital floor ((f)-(j)) are shown (the area highlighted in color means the segmented area). ). Here, (a) and (f) are the original CT images, (b) and (g) are images acquired through the method (hereinafter, method A) using a threshold value of 300HU, (c) and (h) are RefineNet-101 Images obtained through the method (hereinafter, method B), (d) and (i) are images obtained through the U-Net method (hereinafter, method C) learned for the entire orbital bone, (e) and (j) ) shows an image obtained through the orbital bone segmentation method (hereinafter, method D) using multiple brightness value levels according to an embodiment.

First, for quantitative evaluation, the evaluation ROI to be evaluated is defined differently for each patient. The segmented orbital bone is evaluated within the evaluation ROI area, and performance measures within the evaluation ROI area are estimated using sensitivity, specificity and accuracy. Performance measures were estimated using sensitivity, dice similarity coefficient (DSC) and intersection-over-union (IoU) in thin bones of the orbital lining and orbital floor. On the other hand, performance measurement using only specificity and accuracy may not be suitable for comparing and evaluating detailed performance because the ratio of thin bone to the background of the ROI is very small.

6 shows the qualitative performance of a method for segmenting an orbital bone according to an embodiment. For qualitative evaluation, the segmentation results of the entire orbital bone and the thin bones of the orbital medial wall and orbital floor are displayed superimposed between the segmentation results and labels of the two clinical trials. For the whole orbital bone, the sensitivity of Method D is 92.50%, which shows the best performance. This is 2.21 %, 4.05 %, and 0.89 % improvement compared with methods A, B, and C, respectively.

However, the specificity and accuracy of method D is rather lower than that of method A. This is because Method A shows good performance on cortical bone but poor performance on thin bones of the orbital lining and orbital floor. Method A shows the worst performance in terms of sensitivity due to leakage of surrounding soft tissues because the threshold is fixed at 300HU. Method B and method C show that method A increases the sensitivity by 52.62% and 59.84%, respectively. However, both Method B and Method C represent a single brightness level network.

Therefore, the sensitivity and DSC of the orbital segmentation method (Method D) according to an embodiment are 67.61% and 61.96%, 13.16%p and 14.83%p compared to Method B, and 5.94%p and 3.26%p increase compared to Method C did. In particular, it can be seen that 65.78 %p and 58.23 %p increase compared to method A. In FIG. 6 , a value indicating a high performance evaluation result is displayed as large.

Meanwhile, referring back to FIG. 5 , it can be seen that, in Method A, the cortical bone is easily divided, but the thin bone is hardly divided. Also, methods B and C show a tendency to under-segmentation at the boundary of the cortical bone, but it can be seen that it is improved compared to method A. There is no significant difference between methods B and C. However, it can be seen that the orbital bone segmentation method (Method D) according to an embodiment explicitly shows a more stable segmentation result.

7 is a block diagram of an orbital bone division apparatus including a 2D division module and a 3D division module in the second divider according to another embodiment of the present invention.

8 is a flowchart of a method of segmenting a thin bone in a new input image by using a 2D segmentation network and a 3D segmentation network together according to another embodiment of the present invention.

Referring to FIG. 7 , in the orbital bone dividing apparatus 100 according to another embodiment of the present invention, in the process of separating the trabecular and thin bone (ie, the second divider 102 ), A 2D segmentation network and a 3D segmentation network are used together. For example, the second divider 102 accurately separates the trabecular bone and the thin bone by learning 2D RefineNet and 3D U-Net, respectively. That is, the second divider 102 includes a two-dimensional division module and a three-dimensional division module.

A detailed process for the second divider 102 to learn the 2D division network and the 3D division network is described as follows.

First, as shown in FIG. 8 , the 2D division module 103 in the second divider 102 sets a second label in which a Roll of Interest (ROI) of a thin bone in the image data for learning is set based on a low brightness value. to learn Specifically, the 2D segmentation module 103 performs learning by receiving a Region of Interset (ROI) as an input in a 2D image corresponding to the axial plane of the orbital bone.

The 3D division module 104 in the second divider 102 learns 3D learning data in which a thin bone volume of interest (VOI) is set based on a specific reference brightness value in 3D image data of the orbital bone. . At this time, the three-dimensional division module 104 may set the spatial range in which the thin bone exists in the two-dimensional second label as the three-dimensional learning data. As the range of 3D learning data is limited, the orbital bone segmentation apparatus 100 may improve the speed of segmenting the orbital bone, and may improve segmentation accuracy.

In an embodiment using a second divider including a two-dimensional segmentation module 103 and a three-dimensional segmentation module 104, in S310, the orbital bone segmentation apparatus 100 divides the input image into a second divider in two dimensions. It is input to the module 103 and the three-dimensional division module 104, respectively. At this time, the two-dimensional segmentation module 103 receives a two-dimensional image of the same axial plane as during learning. In addition, the 3D division module 104 receives a 3D area (eg, a rectangular parallelepiped area) extracted from the 3D input image as including the thin bone area obtained by the 2D division module 103 . Through this, the 3D segmentation module 104 may receive a 3D image extracted with the same size as 3D learning data used during learning.

Thereafter, the orbital bone segmentation apparatus 100 includes a two-dimensional segmentation module 103 (eg, a RefineNet-based 2D segmentation network) and a three-dimensional segmentation module 104 (eg, U -Combines the 2D segmented data and 3D segmented data segmented in a Net-based 3D segmentation network. That is, the orbital bone segmentation device 100 (in particular, the second segmenter 102) acquires the global context of thin bones and the detailed characteristics of the axial plane by using the two-dimensional segmentation data. , to acquire detailed spatial information of thin bones using 3D segmentation data.

Through this, as in FIGS. 9 and 10 , the orbital bone segmentation apparatus under-segmentation (under-segmentation: segmentation smaller than the actual size) that occurs when only a 2D segmentation network is used to segment thin bone Thus, it is possible to reduce the problem of missing a part of the object) and the problem of poor connectivity between the two-dimensional frame (or slice) of adjacent thin bones. That is, as the spatial information included in the 3D segmentation data obtained by the 3D segmentation network is utilized together, the orbital bone segmentation apparatus 100 increases the connectivity between adjacent two-dimensional frames (or slices), resulting in improved final segmentation data. can be obtained.

11 is a diagram schematically showing the internal configuration of the orbital bone dividing device 100 according to an embodiment.

The orbital bone segmentation apparatus 100 is a device capable of performing a function to be described later, and may be configured as, for example, a server computer, a personal computer, or the like. In one embodiment, the orbital segmentation apparatus 100 may include one or more processors 130 and/or one or more memories 140 .

According to an embodiment, the operation performed by the one or more processors 130 includes a preprocessing operation for normalizing a brightness value and a pixel size of an input image, a first label composed of a region having a high brightness value in the input image, and a low brightness value An operation for setting a second label consisting of an area of dividing the first label and the second label in the input image using the first divider and the second divider on which the operation and the machine learning have been performed, and the orbital bone by integrating the first label and the second label A division operation can be performed. In addition, a person skilled in the art can easily understand that, in addition to the above-described operation, the orbital bone segmentation apparatus 100 according to an embodiment can perform the orbital bone segmentation method described above with reference to FIGS. 1 to 4 .

In one embodiment, at least one of these components of the orbital bone division apparatus 100 may be omitted, or other components may be added to the orbital bone division apparatus 100 . In addition, additionally (additionally) or alternatively (alternatively), some of the components may be implemented as integrated, or may be implemented as a singular or a plurality of entities. At least some of the internal and external components of the orbital bone dividing device 100 are connected to each other via a bus, general purpose input/output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI). It can be connected to send and receive data and/or signals.

The one or more memories 140 may store various data. Data stored in the memory 140 is data obtained, processed, or used by at least one component of the orbital bone segmentation apparatus 100 , and may include software (eg, a program). Memory 140 may include volatile and/or non-volatile memory. The one or more memories 140 may store instructions that, when executed by the one or more processors 130 , cause the one or more processors 130 to perform an operation. In an embodiment, the one or more memories 140 may store personalization information for one or more users and/or recommendation information for one or more products. In the present disclosure, programs or commands are software stored in the memory 140 , and various functions are applied so that an operating system, an application, and/or an application for controlling the resources of the orbital bone division apparatus 100 can utilize the resources of the device. It may include middleware provided to .

The one or more processors 130 may control at least one component of the orbital bone dividing apparatus 100 connected to the processor 130 by driving software (eg, a program, a command). In addition, the processor 130 may perform various operations, processing, data generation, processing, etc. related to the present disclosure. In addition, the processor 130 may load data or the like from the memory 140 or store it in the memory 140 .

In one embodiment, the orbital bone segmentation apparatus 100 may further include a communication interface (not shown). The communication interface may perform wireless or wired communication between the orbital bone dividing device 100 and another server or other external device (eg, the imaging device 120 ). For example, the communication interface is eMBB (enhanced Mobile Broadband), URLLC (Ultra Reliable Low-Latency Communications), MMTC (Massive Machine Type Communications), LTE (long-term evolution), LTE-A (LTE Advance), UMTS ( Universal Mobile Telecommunications System), GSM (Global System for Mobile communications), CDMA (code division multiple access), WCDMA (wideband CDMA), WiBro (Wireless Broadband), WiFi (wireless fidelity), Bluetooth (Bluetooth), NFC (near field) communication), a global positioning system (GPS), or a global navigation satellite system (GNSS) may perform wireless communication.

Various embodiments of the orbital bone dividing apparatus 100 according to the present disclosure may be combined with each other. Each embodiment may be combined according to the number of cases, and the embodiment of the orbital bone dividing device 100 made in combination also falls within the scope of the present disclosure. In addition, the internal/external components of the orbital bone dividing apparatus 100 according to the present disclosure described above may be added, changed, replaced, or deleted according to embodiments. In addition, the internal/external components of the orbital segmentation apparatus 100 described above may be implemented as hardware components.

Meanwhile, the method according to an embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the present invention, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and carry out program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions may include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

So far, with respect to the present invention, the preferred embodiments have been looked at. Those of ordinary skill in the art to which the present invention pertains will understand that the present invention can be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments are to be considered in an illustrative rather than a restrictive sense. The scope of the present invention is indicated in the claims rather than the foregoing description, and all differences within the scope equivalent thereto should be construed as being included in the present invention.

100: orbital bone divider
110: network
120: shooting device

Claims (13)

In the method of segmenting the orbital bone in a head and neck computed tomography (CT) by an orbital bone segmentation device,
a first label and second label setting step of setting a first label composed of an area having a first brightness value and a second label composed of an area having a second brightness value lower than the first brightness value in the input image; and
Including; and dividing the orbital bone by integrating the set first label and the second label;
The method of claim 1, wherein the first label and the second label are divided by a first divider and a second divider that are machine-learned using a network considering a training image and multiple brightness values, respectively. The method of claim 1,
The method further comprising; a preprocessing step of normalizing the brightness value and pixel size of the input image. 3. The method of claim 2,
The pre-processing step is
and changing a brightness value within a predetermined range to an unsigned 8-bit integer value for normalizing the brightness value. The method of claim 1,
wherein the first label comprises a cortical bone region of the orbital bone and the second label comprises a thin bone region of the orbital lining and orbital floor. The method of claim 1,
The first divider is machine-learned by using a single brightness value network (SGB-Net) corresponding to a first label for the first label, and the second divider is machine learning using a corresponding single brightness value network. 6. The method of claim 5,
The single brightness value network (SGB-Net) is
generating a connection path and an extension path having 6 resolution levels;
performing two 3X3 convolutions involving a parametric rectified linear unit (PReLU) and a 2x2 max-pooling operation with a stride equal to 2 for each resolution step in the concatenated path; and
Performing two 3X3 convolutions involving the parametric rectified linear unit for each resolution step in the extension path, and performing 2×2 deconvolution with double amplitude for upsampling at each resolution step. Method, . 7. The method of claim 6,
The single brightness value network (SGB-Net) is
In all resolution steps of the extension path, including a last layer that performs 1x1 convolution to reduce the number of output channels of an image on which upsampling has been performed through deconvolution to the number of labels,
wherein the last layer comprises a softmax function giving the probability of the split label being output. The method of claim 1,
The second divider includes a two-dimensional division module and a three-dimensional division module,
The step of setting the first label and the second label,
inputting the input image to the 2D division module and the 3D division module, respectively; and
Creating the second label by combining the 2D divided data and the 3D divided data divided by the 2D division module and the 3D division module to generate the second label. 9. The method of claim 8,
The 2D division module is a 2D division network based on RefineNet,
The 3D division module is a U-Net-based 3D division network, the method 9. The method of claim 8,
In the first label and second label setting step,
The 3D division module includes a 2D area obtained by the 2D division module, characterized in that it receives a 3D area extracted from a 3D input image. one or more processors; and
one or more memories in which instructions that, when executed by the one or more processors, cause the one or more processors to perform operations are stored;
The operation performed by the one or more processors,
an operation of setting a first label composed of a region having a first brightness value and a second label composed of a region having a second brightness value lower than the first brightness value in the input image; and
Including the operation of dividing the final orbital bone by integrating the set first label and the second label,
The first label and the second label are divided by a first divider and a second divider machine-learned using a network considering a training image and multiple brightness values, respectively. 12. The method of claim 11,
wherein the first label includes a cortical bone region of the orbital bone, and the second label includes a thin bone region of the orbital lining and orbital floor. In combination with a computer as hardware, any one of claims 1 to 10 is stored in the medium to execute the method, the orbital bone segmentation program in the computed tomography image.

Images