KR102549694B1 - Apparatus for deep-learning based automatic pectoralis muscle segmentation and recording medium in which a program for excuting a prcess of the same is recorded - Google Patents

Abstract

Disclosed are a deep learning-based automatic pectoralis major muscle measuring device that automatically measures the pectoralis major muscle region using a deep learning technique using a chest CT image, and a recording medium on which a program for executing the method is recorded. Deep learning-based automatic pectoralis major muscle measuring device according to an embodiment of the present invention: the pectoralis major muscle is present from a first coronal plane image of a first chest computed tomography (CT) image obtained for an object by a classification artificial intelligence model a pectoralis major muscle region determination unit configured to determine first horizontal plane images; and a pectoralis major region segmentation unit configured to divide a region of the pectoralis major muscle through region segmentation that extracts features from the first horizontal plane images in which the pectoralis major muscle is determined to exist by the segmentation artificial intelligence model.

Description

Apparatus for deep-learning based automatic pectoralis muscle segmentation and recording medium in which a program for executing a process of the same is recorded}

The present invention relates to a technique for automatically measuring the pectoralis major muscle region using a deep learning technique using a chest CT image.

The pectoralis muscle is a muscle that plays an important role in the movement of the shoulder joint. The pectoralis major muscle helps to flex the arm and facilitates arm extension. The pectoralis major also serves as a respiratory muscle, helping to lift the chest and ribs when forced breathing is necessary, inhaling air and allowing the lungs to expand. Therefore, the size and muscle quality of the pectoralis major muscle inevitably has a great relationship with the level and progression of the disease related to breathing, and the progression and progress of various lung diseases, especially chronic obstructive pulmonary disease (COPD; Chronic Obstructive Pulmonary Disease) is known to have a great relationship.

On the other hand, chest computed tomography (CT; Computed Tomography) is a technique for acquiring cross-sectional images of the chest, and is widely used as a test method used for chest diagnosis. Currently, in the medical world, the size of the pectoralis major muscle is mainly measured using a measurement result in a 2D cross section corresponding to a specific slice position of a CT image. However, the measurement method in the 2D section has a limitation in that the size of the pectoralis major muscle of the patient is measured differently depending on the patient's posture or the skill level of the medical staff.

An object of the present invention is to provide a deep learning-based automatic pectoralis major muscle measuring device that performs automatic measurement of the pectoralis major muscle region using a deep learning technique using a chest CT image and a recording medium on which a program for executing the method is recorded.

In addition, the present invention is to provide a deep learning-based automatic pectoralis major measurement technology that does not change the pectoralis major measurement result according to the patient's posture or the clinical judgment of a medical expert and can increase the measurement accuracy of the pectoralis major muscle area.

The object of the present invention is not limited to the above-mentioned object, and other objects not mentioned will be clearly understood by those skilled in the art from the description below.

Deep learning-based automatic pectoralis major muscle measuring device according to an embodiment of the present invention: the pectoralis major muscle is present from a first coronal plane image of a first chest computed tomography (CT) image obtained for an object by a classification artificial intelligence model a pectoralis major muscle region determination unit configured to determine first horizontal plane images; and a pectoralis major region segmentation unit configured to divide a region of the pectoralis major muscle through region segmentation that extracts features from the first horizontal plane images in which the pectoralis major muscle is determined to exist by the segmentation artificial intelligence model.

The deep learning-based automatic pectoralis major muscle measuring device according to an embodiment of the present invention: trains the classification artificial intelligence model using a first learning image in which a pectoralis major muscle present region is labeled in a second coronal plane image of a second chest CT image. Classification artificial intelligence model learning unit configured to; and a segmentation artificial intelligence model learning unit configured to train the segmentation artificial intelligence model using a second training image in which the pectoralis major muscle region is labeled in a second horizontal plane image of the second chest CT image.

The pectoralis major muscle presence region determination unit: obtains the first coronal plane image processed by color synthesis from the first chest CT image, and uses the classification artificial intelligence model learned by the classification artificial intelligence model learning unit to determine the It may be configured to extract the first horizontal plane images from among the plurality of horizontal plane images of the first chest CT image by predicting the presence region of the pectoralis major muscle in the first coronal plane image.

The pectoralis major muscle presence region determination unit: identifies coronal plane positions corresponding to the middle portion based on the stacking order among the stacked coronal plane images of the first chest CT image; extracting a plurality of grayscale coronal plane images corresponding to the coronal plane locations from among the coronal plane images; synthesizing the plurality of grayscale coronal plane images into one to obtain a color coronal plane image; acquiring the first coronal plane image by matching the color coronal plane image to a set length; And it may be configured to determine the first horizontal plane images by the classification artificial intelligence model from the first coronal plane image.

The plurality of grayscale coronal plane images may include second coronal plane images corresponding to 3/8, 4/8, and 5/8 coronal plane positions based on the stacking order of the second coronal plane images. .

The pectoralis major region segmentation unit: generates a prediction image of the pectoralis major muscle using the segmentation artificial intelligence model learned by the segmentation artificial intelligence model learning unit; In addition, the pectoralis major muscle region may be segmented by excluding small regions corresponding to noise from the pectoralis major muscle prediction image.

The pectoralis major region segmentation unit: generates a binarized image having set pixel values by binarizing the pectoralis major muscle predicted image; dividing the binary image into a plurality of regions by connecting and grouping adjacent pixels in the binary image; In addition, the area of each of the plurality of areas may be calculated, and the pectoralis major muscle may be measured by excluding areas having an area less than or equal to a reference value.

The classification artificial intelligence model may be implemented by adding a fully connected layer based on the EfficientNet model. The segmentation artificial intelligence model includes: a contracting pass configured to sequentially and repeatedly perform a first convolution process and a pooling process through a plurality of contracting layers; an expansive pass configured to sequentially and repeatedly perform second convolution processing and up-sampling processing through a plurality of expanding layers; and a skip connection unit configured to transfer the first feature map extracted from the contracting path to the expansive path.

The expansive pass may include an attention module applied to each of the plurality of expanding layers. The attention module: receives the first feature map from the contracting layer of the contracting path by the skip connection unit; receiving a second feature map from the first expending layer of the expansive pass; generating an attention map by performing third convolution processing, relu activation processing, and sigmoid processing based on the first feature map and the second feature map; And it may be configured to transfer the attention map to a second expanding layer of the expensive pass.

According to an embodiment of the present invention, a computer-readable recording medium on which a program for executing a deep learning-based automatic pectoralis major muscle measurement method is recorded is provided. The deep learning-based automatic pectoralis major muscle measurement method: discriminates first horizontal plane images in which the pectoralis major muscle is present from a first coronal plane image of a first chest computed tomography (CT) image obtained for an object by a classification artificial intelligence model; doing; and segmenting a region of the pectoralis major muscle through region segmentation of extracting features from the first horizontal plane images in which the pectoralis major muscle is determined to be present by the segmentation artificial intelligence model.

The deep learning-based automatic pectoralis major muscle measurement method: The classification artificial intelligence model learning unit uses a first learning image in which the pectoralis major muscle presence region is labeled in a second coronal plane image of a second chest CT image by a classification artificial intelligence model learning unit. training the model; and learning the segmentation artificial intelligence model using a second training image in which the pectoralis major muscle region is labeled in a second horizontal plane image of the second chest CT image by a segmentation artificial intelligence model learning unit. .

The step of determining the first horizontal plane images may include: obtaining the first coronal plane image subjected to color synthesis processing from the first chest CT image, and the classification artificial intelligence model learned by the classification artificial intelligence model learning unit. The method may include extracting the first horizontal plane images from among a plurality of horizontal plane images of the first chest CT image by predicting a region in which the pectoralis major muscle exists in the first coronal plane image using .

The determining of the first horizontal plane images may include: identifying coronal plane positions corresponding to an intermediate portion based on stacking order among the stacked coronal plane images of the first chest CT image; extracting a plurality of grayscale coronal plane images corresponding to the coronal plane locations from among the coronal plane images; obtaining a color coronal plane image by combining the plurality of grayscale coronal plane images into one; acquiring the first coronal plane image by fitting the color coronal plane image to a set length; and determining the first horizontal plane images from the first coronal plane image by the classification artificial intelligence model.

The segmenting of the pectoralis major muscle region may include: generating a prediction image of the pectoralis major muscle using the segmentation artificial intelligence model learned by the segmentation artificial intelligence model learning unit; and dividing the region of the pectoralis major muscle by excluding small regions corresponding to noise from the prediction image of the pectoralis major muscle.

The dividing of the pectoralis major muscle region may include: generating a binarized image having set pixel values by binarizing the pectoralis major muscle prediction image; dividing the binary image into a plurality of regions by connecting and grouping adjacent pixels in the binary image; and calculating the area of each of the plurality of areas, and measuring the pectoralis major muscle by excluding areas having an area less than or equal to a reference value.

According to an embodiment of the present invention, a deep learning-based automatic pectoralis major muscle measuring device for performing automatic measurement of the pectoralis major muscle area using a deep learning technique using a chest CT image and a recording medium recording a program for executing the method are Provided.

In addition, according to an embodiment of the present invention, it is possible to prevent changes in measurement results of the pectoralis major muscle depending on a patient's posture or a clinical judgment of a medical professional, and it is possible to increase the measurement accuracy of the pectoralis major muscle region.

1 is a configuration diagram of a device for automatically measuring the pectoralis major muscle based on deep learning according to an embodiment of the present invention.
2 is a configuration diagram of a deep learning-based automatic pectoralis major muscle measuring unit constituting a deep learning-based automatic pectoralis major muscle measuring device according to an embodiment of the present invention.
3 is a conceptual diagram of a classification artificial intelligence model constituting the deep learning-based automatic pectoralis major measurement device according to an embodiment of the present invention according to an embodiment of the present invention.
4 is a conceptual diagram of a segmented artificial intelligence model constituting the deep learning-based automatic pectoralis major muscle measuring device according to an embodiment of the present invention according to an embodiment of the present invention.
5 is a flowchart of a deep learning-based automatic pectoralis major muscle measuring method executed by a program recorded on a computer-readable recording medium according to an embodiment of the present invention.
6 is a flowchart illustrating step S130 of FIG. 5 .
FIG. 7 is an exemplary diagram for explaining a process of acquiring a first coronal plane image according to the embodiment of FIG. 6 .
8 is a flowchart illustrating step S160 of FIG. 6 .
9 is a diagram showing a segmentation result of a predicted pectoralis major muscle region according to an embodiment of the present invention.
10 is a diagram showing the result of extracting a horizontal plane image in which the pectoralis major muscle is present based on deep learning according to an embodiment of the present invention.
11 is a diagram showing results of segmenting a pectoralis major muscle region from a horizontal plane image based on deep learning according to an embodiment of the present invention.

Objects and effects of the present invention, and technical configurations for achieving them will become clear with reference to the embodiments described later in detail in conjunction with the accompanying drawings. In describing the present invention, if it is determined that a detailed description of a known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description will be omitted. In addition, terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to the intention or custom of a user or operator.

However, the present invention is not limited to the embodiments disclosed below and may be implemented in a variety of different forms. Only these embodiments are provided to complete the disclosure of the present invention and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined by the scope of the claims. only become Therefore, the definition should be made based on the contents throughout this specification.

'~unit, ~module' used in this specification is a unit that processes at least one function or operation, and may mean, for example, software, FPGA, or hardware component. Functions provided by '~part, ~module' may be performed separately by a plurality of components or may be integrated with other additional components. '~unit, ~module' in this specification is not necessarily limited to software or hardware, and may be configured to be in an addressable storage medium or configured to be reproduced by one or more processors. Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

In the deep learning-based automatic pectoralis major muscle measurement device according to an embodiment of the present invention, the pectoralis muscle is present from the first coronal plane image of the first chest computed tomography image (3D chest CT image) acquired for the subject (patient). A deep learning-based classification artificial intelligence model that discriminates the first horizontal plane images of the pectoralis major muscle, and a deep learning-based division of the pectoralis major muscle region through region segmentation that extracts features from the first horizontal plane images in which the pectoralis major muscle is determined to exist. It includes a segmented AI model.

The deep learning-based automatic pectoralis major muscle measurement device according to an embodiment of the present invention determines axial images in which the pectoralis major muscle exists through a coronal image of a 3D ultrasound image, and detects the horizontal plane in which the pectoralis major muscle is present. In the CT image, the actual pectoralis major muscle region can be segmented.

The deep learning-based automatic pectoralis major muscle measuring device according to an embodiment of the present invention does not use only the horizontal plane image of the 3D chest CT image, but uses the coronal plane image together with the horizontal plane image of the 3D chest CT image to measure the pectoralis major muscle. The 3D pectoralis major muscle area, not the pectoralis major muscle, can be measured.

In addition, the deep learning-based automatic pectoralis major muscle measurement device according to an embodiment of the present invention uses a 3D CT image, but does not directly utilize a 3D volume, and extracts features in a corresponding area by an artificial intelligence model. D Using the patient information, the 3D region corresponding to the pectoralis major muscle can be extracted.

According to the deep learning-based automatic pectoralis major muscle measuring device according to an embodiment of the present invention as described above, the pectoralis major muscle area is measured based on a plurality of artificial intelligence models (classified artificial intelligence model, segmented artificial intelligence model) learned based on deep learning. automatic measurements can be made.

According to the deep learning-based automatic pectoralis major muscle measuring device according to an embodiment of the present invention, it is possible to prevent the pectoralis major muscle measurement result from changing depending on the patient's posture and to prevent the pectoralis major muscle measurement result from being influenced by the clinical judgment of a medical expert. It is possible to increase the measurement accuracy of the pectoralis major muscle area.

1 is a configuration diagram of a device for automatically measuring the pectoralis major muscle based on deep learning according to an embodiment of the present invention. Referring to FIG. 1 , the deep learning-based automatic pectoralis major muscle measurement device 10 according to an embodiment of the present invention includes a user interface unit 20, an input unit 30, a deep learning-based automatic pectoralis major muscle measurement unit 40, a storage unit ( 50), a display unit 60, and a control unit 70.

The user interface unit 20 may be provided so that a medical professional such as a doctor inputs various commands for executing a program for automatic deep learning-based measurement of the pectoralis major muscle. The user interface unit 20 may include, for example, an input interface such as a keyboard, a mouse, a touch pad, an electronic pen, and a voice recognition unit.

The input unit 30 is provided to receive various data necessary for deep learning-based automatic measurement of the pectoralis major muscle, for example, a chest computed tomography (CT) image of a patient designated by a medical professional using the user interface unit 20. can The input unit 30 may receive data from the chest CT device or data from the storage unit 50 .

The deep learning-based automatic pectoralis major measurement unit 40 uses a plurality of artificial intelligence models (classification artificial intelligence model, segmentation artificial intelligence model) learned based on deep learning, and the pectoralis major muscle area or the pectoralis major muscle size (area) from the patient's chest CT image. ) and the like can be performed automatically.

The storage unit 50 includes a chest CT image of the patient, a program for learning a plurality of artificial intelligence models (classification artificial intelligence model, a segmentation artificial intelligence model) based on deep learning, and a patient based on the learned plurality of artificial intelligence models. A program that performs a process of automatically measuring the pectoralis major muscle area and the size of the pectoralis major muscle may be stored.

The display unit 60 includes a patient's chest CT image, coronal images of the chest CT image, sagittal plane images, horizontal plane images, the pectoralis major muscle area predicted from the chest CT image by the deep learning-based automatic pectoralis major measurement unit 40, Information such as the pectoralis major muscle area may be provided to be displayed through the display screen of the medical terminal.

The control unit 70 controls various components such as the user interface unit 20, the input unit 30, the deep learning-based automatic pectoralis major measurement unit 40, the storage unit 50, and the display unit 60 to model the artificial intelligence model. It may include at least one processor that performs a process of measuring a pectoralis major muscle area or a pectoralis major muscle size from a chest CT image of a patient using .

2 is a configuration diagram of a deep learning-based automatic pectoralis major muscle measuring unit constituting a deep learning-based automatic pectoralis major muscle measuring device according to an embodiment of the present invention. Referring to FIG. 2 , the deep learning-based automatic pectoralis major measurement unit 40 includes a pectoralis major muscle region determination unit 100, a pectoralis major area segmentation unit 200, a classification artificial intelligence model learning unit 300, and a segmentation artificial intelligence model learning unit. It may include section 400 .

The pectoralis major muscle presence area determining unit 100 determines whether the pectoralis major muscle is selected from axial images of the first chest CT image from a first coronal image of the first chest CT image acquired of the object (patient). A classification artificial intelligence model 110 for determining existing first horizontal plane images may be included.

The pectoralis major muscle region segmentation unit 200 extracts features from the first horizontal plane images in which the pectoralis major muscle is determined to be present among the horizontal plane images of the first chest CT image through segmentation of the pectoralis major muscle in the first horizontal plane images. It may include a segmentation artificial intelligence model 210 that divides regions.

The classification artificial intelligence model learning unit 300 and the segmentation artificial intelligence model learning unit 400 include a plurality of learning images (eg, a plurality of coronal images of a 3D chest CT image for learning an artificial intelligence model and a plurality of coronal images). The classification artificial intelligence model 110 and the segmentation artificial intelligence model 210 may be trained using horizontal plane images of).

The classification artificial intelligence model learning unit 300 uses a plurality of first training images in which the pectoralis major muscle presence regions (coronal plane locations where the pectoralis major muscle exists) are labeled in the second coronal plane image of the second chest CT image, A classification artificial intelligence model 110 for classification of existing horizontal plane images may be trained.

The segmentation artificial intelligence model learning unit 400 generates a plurality of second learning images (horizontal images in which the region corresponding to the pectoralis major muscle and the region other than the pectoralis major muscle are distinguished) in which the pectoralis major muscle region is labeled in the second horizontal plane image of the second chest CT image. The segmentation artificial intelligence model 210 for segmentation of the pectoralis major muscle region in the first horizontal plane can be trained using the above.

3 is a conceptual diagram of a classification artificial intelligence model constituting a deep learning-based automatic pectoralis major muscle measuring device according to an embodiment of the present invention according to an embodiment of the present invention. The classification artificial intelligence model 110 may be implemented by adding a Fully Connected Layer 114 based on the EfficientNet model 112.

In an embodiment, the EfficientNet model may be implemented based on the “EfficientNet-b6” model. As is known, the adaptive net model can additionally operate AdaptiveAvgPool2d, Dropout, and Linear processes.

In the adaptive net model, the adaptive average pool plays a role in linearly changing the shape of the input value. Dropout is to prevent overfitting, and if the coefficient is set to 0.5, it can serve to omit half of the network. The linear process serves to determine the number of output features.

4 is a conceptual diagram of a segmented artificial intelligence model constituting a deep learning-based automatic pectoralis major muscle measuring device according to an embodiment of the present invention. The segmentation artificial intelligence model 210 may include a contracting pass 220 that reduces the image size, an expansive pass 230 that increases the image size, and a skip connection unit 240.

The contracting path 220 performs a first convolution process and a pooling process (eg, max pooling process, average pooling process, etc.) on a plurality of contracting layers. ), it can be performed sequentially and repeatedly.

The expansive path 230 may sequentially and repeatedly perform second convolution processing and up sampling processing through a plurality of expanding layers.

The skip connection unit 240 transfers the first feature map (convolution image) extracted from the contracting layer of the contracting pass 220 to the contracting layer and the corresponding expansive path 230 ) to the expanding layer.

The expansive path 230 may include a plurality of attention modules 232 . The attention module 232 may be applied to each of the plurality of expanding layers of the expansive pass 230 . In an embodiment, the segmentation artificial intelligence model 210 may be implemented as a U-Net model in which the attention module 232 is added to the expansive pass 230 .

The attention module 232 receives the first feature map from the contracting layer of the contracting path 220 through the skip connection unit 240, and receives the first feature map from the first expanding layer of the expansive path 230. 2 A feature map can be delivered.

The attention module 232 may generate an attention map by performing third convolution processing, ReLU activation processing, and sigmoid processing based on the received first and second feature maps. there is.

The Attention module 232 performs Convolution -> Batch Normalization/Convolution -> Batch Normalization -> Relu Activation Function (ReLu) -> Convolution -> Batch Normalization -> Processing in sigmoid order can be performed.

In addition, ReduceLROnPlateau may be applied to the UNET model as a scheduler for adjusting the learning rate. It can be driven to decrease the learning rate by 50% if no improvement is seen for the number of epochs, so that learning plateaus when the metric improvement stops, benefiting from the learning rate reduction.

The attention map generated by the attention module 232 of the first expanding layer of the expansive pass 230 may be transferred to the second expanding layer connected to the first expanding layer of the expansive pass 230. there is.

The segmentation artificial intelligence model 2120 may be configured to perform region segmentation processing corresponding to the pectoralis major muscle by increasing a weight of a central upper part of the image corresponding to the chest region using a custom loss function.

In an embodiment, the pectoralis major muscle presence region determining unit 100 may obtain a color synthesized first coronal plane image from a first chest CT image of a patient, which is an object of pectoralis major muscle measurement. The first coronal plane image obtained from the first chest CT image of the patient may be input to the classification artificial intelligence model 110 .

The pectoralis major muscle region determination unit 100 predicts the pectoralis major muscle region present in the first coronal plane image using the classification artificial intelligence model 110 learned by the classification artificial intelligence model learning unit 300, and the first chest CT First horizontal plane images may be extracted from among a plurality of horizontal plane images of the image.

In order to acquire a first coronal plane image from the first chest CT image, the pectoralis major muscle presence region determining unit 100 first selects the middle, based on the stacking order, among the stacked coronal plane images of the first chest CT image acquired for the patient. It is possible to check the coronal plane positions corresponding to the part.

Next, the pectoralis major muscle presence region determining unit 100 may extract a plurality of grayscale coronal plane images corresponding to coronal plane positions from among the coronal plane images of the first chest CT image.

In an embodiment, the plurality of grayscale coronal images extracted from the coronal plane images of the first chest CT image are 3 corresponding to 3/8, 4/8, and 5/8 coronal plane positions based on the stacking order. Coronal plane images of dogs may be included. However, the coronal plane positions may be set differently according to the size of the CT image.

Next, the pectoralis major muscle presence region determining unit 100 is configured to generate a plurality of grayscale coronal plane images corresponding to the set coronal plane positions (eg, 3/8, 4/8, and 5/8 coronal plane positions). A color coronal plane image can be obtained by attaching three coronal plane images) and synthesizing them into one so that each can take charge of RGB color.

Next, the pectoralis major muscle presence region determination unit 100 may obtain a first coronal plane image by matching the color coronal plane image to a set length. For example, in order to adjust the height of the coronal image to a set length (eg, 416), if the coronal image is longer than the set length 416, the lower part is cut, and if it is shorter than the set length 416, the same arrangement as the bottom row can be added to the lower part to match the set length of 416.

The pectoralis major muscle region determining unit 100 may determine first horizontal plane images in which the pectoralis major muscle is present among horizontal plane images of the first chest CT image by the classification artificial intelligence model 110 from the first coronal plane image.

The pectoralis major region segmentation unit 200 generates a pectoralis major muscle prediction image using the segmentation artificial intelligence model 210 learned by the segmentation artificial intelligence model learning unit 400, and small regions corresponding to noise in the pectoralis major muscle prediction image ( The area of the pectoralis major muscle can be segmented by excluding small-sized areas).

To describe the noise removal process in more detail, the pectoralis major region division unit 200 may binarize the pectoralis major prediction image output from the segmentation artificial intelligence model learning unit 400 to generate a binarized image having set pixel values.

The pectoralis major region segmentation unit 200 may generate a binarized image by leaving only pixels having a pixel value of '1' after, for example, binarization based on a threshold.

Next, the pectoralis major region division unit 200 may divide the binarized image into a plurality of regions (eg, regions having a pixel value of '1') by connecting and grouping adjacent pixels in the binarized image.

At this time, the pectoralis major region segmentation unit 200 may perform labeling after connecting and grouping adjacent pixels through, for example, Connected Component With Stats.

Next, the pectoralis major muscle region segmentation unit 200 calculates the area of each of the plurality of divided areas in the binarized image, and excludes the area whose area is less than a reference value (eg, the reference area corresponding to 10 pixels) to determine the pectoralis major muscle. can be measured

5 is a flowchart of a method for automatically measuring the pectoralis major muscle based on deep learning executed by a program recorded on a computer-readable recording medium according to an embodiment of the present invention. Referring to FIGS. 1, 2 and 5, when a plurality of training images are acquired, the classification artificial intelligence model learning unit 300 and the segmentation artificial intelligence model learning unit 400 classify using the plurality of training images. The segmentation artificial intelligence model 110 and the segmentation artificial intelligence model 210 may be trained (S110, S120).

The classification artificial intelligence model learning unit 300 may train the classification artificial intelligence model 110 using a plurality of first training images in which the pectoralis major muscle region is labeled in the second coronal plane image of the second chest CT image. there is.

The segmentation artificial intelligence model learning unit 400 may train the segmentation artificial intelligence model 210 using a plurality of second training images in which the pectoralis major muscle region is labeled in the second horizontal plane image of the second chest CT image.

The pectoralis major muscle presence area determining unit 100 obtains a first coronal image from the first chest CT image obtained for the subject (patient), and uses the classification artificial intelligence model 110 to obtain the first coronal image. From the image, it may be determined whether the pectoralis major muscle is present in the axial images of the first chest CT image (S130 and S140).

The pectoralis major region segmentation unit 200 performs region segmentation to extract features from the first horizontal plane images determined to have the pectoralis major muscle among the horizontal plane images of the first chest CT image by the segmentation artificial intelligence model 210. Through this, a pectoralis major muscle predicted image can be generated from the horizontal plane image of the 3D chest CT image, and the pectoralis major muscle region can be segmented by excluding the noise region from the predicted pectoralis major muscle predicted image (S150 and S160).

6 is a flowchart illustrating step S130 of FIG. 5 . FIG. 7 is an exemplary diagram for explaining a process of acquiring a first coronal plane image according to the embodiment of FIG. 6 . Referring to FIGS. 1, 2, and 5 to 7 , in order to acquire a first coronal plane image from a first chest CT image, the pectoralis major muscle presence area determining unit 100 first acquires a first chest CT image of the patient. Among the stacked coronal plane images of images, coronal plane positions corresponding to the middle part based on the stacking order may be checked (S132).

Next, the pectoralis major muscle presence region determination unit 100 extracts a plurality of grayscale coronal plane images 11, 12, and 13 corresponding to coronal plane locations from among the coronal plane images of the first chest CT image. It can (S134).

In an embodiment, the plurality of grayscale coronal images extracted from the coronal plane images of the first chest CT image are 3 corresponding to 3/8, 4/8, and 5/8 coronal plane positions based on the stacking order. It may include coronal plane images 11, 12, and 13 of the dog. Coronal plane positions may be set differently according to the CT image size.

Next, the pectoralis major muscle presence region determining unit 100 is configured to generate a plurality of grayscale coronal plane images corresponding to the set coronal plane positions (eg, 3/8, 4/8, and 5/8 coronal plane positions). A color coronal plane image 14 may be obtained by attaching three coronal plane images) and combining them into one so that each can take charge of RGB color (S136).

Next, the pectoralis major muscle presence region determination unit 100 may obtain a first coronal plane image 15 by matching the color coronal plane image 14 to a set length (S138). For example, in order to adjust the height of the coronal image to the set length, if the length of the coronal image is longer than the set length, the lower part may be cut off, and if the length is shorter than the set length, an array such as the bottom row may be added below to adjust the length.

Thereafter, the pectoralis major muscle region determination unit 100 may determine first horizontal plane images in which the pectoralis major muscle is present among horizontal plane images of the first chest CT image by the classification artificial intelligence model 110 from the first coronal plane image. .

The pectoralis major region segmentation unit 200 generates a pectoralis major muscle prediction image using the segmentation artificial intelligence model 210 learned by the segmentation artificial intelligence model learning unit 400, and small regions corresponding to noise in the pectoralis major muscle prediction image ( The area of the pectoralis major muscle can be segmented by excluding small-sized areas).

8 is a flowchart illustrating step S160 of FIG. 6 . 9 is a diagram showing a segmentation result of a predicted pectoralis major muscle region according to an embodiment of the present invention. Referring to FIGS. 1, 2, 6, 8, and 9 , the pectoralis major region segmentation unit 200 binarizes the pectoralis major muscle prediction image output from the segmentation artificial intelligence model learning unit 400 and has pixel values set therein. A binarized image may be generated (S162).

The pectoralis major region segmentation unit 200 may generate a binarized image by leaving only pixels having a pixel value of '1' after, for example, binarization based on a threshold.

Next, the pectoralis major region division unit 200 may divide the binarized image into a plurality of regions (eg, regions having a pixel value of '1') by connecting and grouping adjacent pixels in the binarized image (S164). ).

At this time, the pectoralis major region dividing unit 200 may perform labeling after connecting and grouping adjacent pixels through, for example, Connected Component With Stats.

Next, the pectoralis major muscle region segmentation unit 200 calculates the area of each of the plurality of divided areas in the binarized image, and excludes the area whose area is less than a reference value (eg, the reference area corresponding to 10 pixels) to determine the pectoralis major muscle. It can be measured (S166).

10 is a diagram showing the result of extracting a horizontal plane image in which the pectoralis major muscle is present based on deep learning according to an embodiment of the present invention. 10 (a) is a first coronal plane image extracted from a 3D chest CT image, (b) is a mask area, (c) is a predicted mask area, (d) is a pectoralis major muscle area (PM) and a non-pectoral muscle area ( Non-PM) indicates the predicted horizontal plane image slice range.

The mask area shown in (b) of FIG. 10 represents the area corresponding to the horizontal planes where the pectoralis major muscle exists, and the predicted mask area shown in (c) of FIG. 10 corresponds to the horizontal planes classified by the classification artificial intelligence model. indicates the corresponding area. According to an embodiment of the present invention, automatic pectoralis major measurement was performed by a deep learning-based artificial intelligence model to determine a horizontal plane image slice in which the pectoralis major muscle was present, and it was measured to have a high accuracy of about 95%.

11 is a diagram showing results of segmenting the pectoralis major muscle region from a horizontal plane image based on deep learning according to an embodiment of the present invention. Figure 11 (a) is a mask corresponding to the pectoralis major muscle area in the horizontal plane image, Figure 11 (b) is the pectoralis major muscle area predicted by the deep learning-based segmentation artificial intelligence model, (c) is the result of (a) is the difference between the result of and (b). According to an embodiment of the present invention, as a result of segmenting the pectoralis major muscle area in a horizontal plane image by a deep learning-based artificial intelligence model, the pectoralis major muscle area was predicted with a high accuracy of about 90%.

According to the recording medium on which the program for executing the deep learning-based automatic pectoralis major muscle measurement method according to the embodiment of the present invention as described above is recorded, a plurality of artificial intelligence models learned based on deep learning (classification artificial intelligence model, segmentation Based on artificial intelligence model), automatic measurement of the pectoralis major muscle area can be performed.

According to an embodiment of the present invention, it is possible to discriminate axial images in which the pectoralis major muscle exists through a coronal image of a 3D ultrasound image, and divide an actual pectoralis major muscle area in the horizontal plane CT image in which the pectoralis major muscle is determined to exist. can

In addition, according to an embodiment of the present invention, it is possible to measure the 3D pectoralis major muscle rather than the 2D pectoralis major muscle by using the coronal plane image together with the horizontal plane image to measure the pectoralis major muscle, rather than using only the horizontal plane image among the 3D CT images.

In addition, according to an embodiment of the present invention, while using a 3D CT image, without directly utilizing a 3D volume, features in the corresponding area are extracted by an artificial intelligence model, and 2.5D patient information is used to detect the pectoralis major muscle. 3D regions can be extracted.

In addition, according to an embodiment of the present invention, it is possible to prevent the pectoralis major muscle measurement result from being changed according to a patient's posture change or a medical expert's clinical judgment, and it is possible to increase the measurement accuracy of the pectoralis major muscle region.

The above description is merely an example of the technical idea of the present invention, and various modifications and variations can be made to those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed according to the following claims, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

10: Deep learning-based automatic pectoralis major measuring device
20: user interface unit
30: input unit
40: deep learning-based automatic pectoralis major measurement unit
50: storage unit
60: display unit
70: control unit
100: Pectoralis major muscle presence area determining unit
110: Classification artificial intelligence model
200: pectoralis major area division
210: Segmented AI Model
300: classification artificial intelligence model learning unit
400: Segmented artificial intelligence model learning unit

Claims (14)

a pectoralis major muscle presence region determining unit configured to determine first horizontal plane images in which the pectoralis major muscle is present from a first coronal plane image of a first chest computed tomography (CT) image acquired of the object by a classification artificial intelligence model; and
A deep learning-based automatic pectoralis major muscle area segmentation unit configured to segment a region of the pectoralis major muscle through region segmentation that extracts features from the first horizontal plane images in which the pectoralis major muscle is determined to exist by the segmentation artificial intelligence model; measuring device. According to claim 1,
a classification artificial intelligence model learning unit configured to train the classification artificial intelligence model using a first training image in which a pectoralis major muscle presence region is labeled in a second coronal image of a second chest CT image; and
Further comprising a deep learning-based segmentation artificial intelligence model learning unit configured to train the segmentation artificial intelligence model using a second training image in which the pectoralis major muscle region is labeled in a second horizontal plane image of the second chest CT image. Automatic pectoralis major measurement device. According to claim 2,
The pectoralis major muscle presence region determination unit:
The first coronal plane image processed by color synthesis from the first chest CT image is obtained, and the pectoralis major muscle in the first coronal plane image is obtained by using the classification artificial intelligence model learned by the classification artificial intelligence model learning unit. A deep learning-based automatic pectoralis major muscle measuring device configured to predict an existing region and extract the first horizontal plane images from among a plurality of horizontal plane images of the first chest CT image. According to claim 3,
The pectoralis major muscle presence region determination unit:
identifying coronal plane positions corresponding to the middle portion based on the stacking order among the stacked coronal plane images of the first chest CT image;
extracting a plurality of grayscale coronal plane images corresponding to the coronal plane locations from among the coronal plane images;
synthesizing the plurality of grayscale coronal plane images into one to obtain a color coronal plane image;
acquiring the first coronal plane image by matching the color coronal plane image to a set length; and
Deep learning-based automatic pectoralis major measurement device configured to determine the first horizontal plane images by the classification artificial intelligence model from the first coronal plane image. According to claim 4,
The plurality of grayscale coronal plane images include second coronal plane images corresponding to coronal plane positions of 3/8, 4/8 and 5/8 based on the stacking order. According to claim 2,
The pectoralis major region division is:
generating a prediction image of the pectoralis major muscle using the segmentation artificial intelligence model learned by the segmentation artificial intelligence model learning unit; and
A deep learning-based automatic pectoralis major measuring device configured to divide the region of the pectoralis major muscle by excluding small regions corresponding to noise in the pectoralis major muscle prediction image. According to claim 6,
The pectoralis major region division is:
generating a binarized image having set pixel values by binarizing the predicted pectoralis major image;
dividing the binary image into a plurality of regions by connecting and grouping adjacent pixels in the binary image; and
A deep learning-based automatic pectoralis major measuring device configured to calculate the area of each of the plurality of areas and measure the pectoralis major muscle excluding areas in which the area is less than or equal to a reference value. According to claim 1,
The classification artificial intelligence model is implemented by adding a fully connected layer based on the EfficientNet model,
The segmented artificial intelligence model is:
A contracting pass configured to sequentially and repeatedly perform a first convolution process and a pooling process through a plurality of contracting layers;
an expansive pass configured to sequentially and repeatedly perform second convolution processing and up-sampling processing through a plurality of expanding layers; and
A skip connection unit configured to transfer the first feature map extracted from the contracting path to the expansive path;
The expansive pass includes an attention module applied to each of the plurality of expanding layers,
The Attention module:
receiving the first feature map from the contracting layer of the contracting path by the skip connection unit;
receiving a second feature map from the first expending layer of the expansive pass;
generating an attention map by performing third convolution processing, relu activation processing, and sigmoid processing based on the first feature map and the second feature map; and
Deep learning-based automatic pectoralis major measurement device configured to transfer the attention map to a second expanding layer of the expensive pass. A computer-readable recording medium on which a program for executing a deep learning-based automatic pectoralis major measurement method is recorded,
The deep learning-based automatic pectoralis major measurement method:
determining first horizontal plane images in which the pectoralis major muscle is present from a first coronal plane image of a first chest computed tomography (CT) image acquired of the object by a classification artificial intelligence model; and
A computer-readable recording medium comprising: segmenting a region of the pectoralis major muscle through region segmentation of extracting features from the first horizontal plane images in which the pectoralis major muscle is determined to be present by a segmentation artificial intelligence model. According to claim 9,
The deep learning-based automatic pectoralis major measurement method:
learning the classification artificial intelligence model using a first training image in which a pectoralis major muscle region is labeled in a second coronal plane image of a second chest CT image by a classification artificial intelligence model learning unit; and
Learning the segmentation artificial intelligence model by a segmentation artificial intelligence model learning unit using a second training image in which the pectoralis major muscle region is labeled in a second horizontal plane image of the second chest CT image; A readable recording medium. According to claim 10,
The step of determining the first horizontal plane images is:
The first coronal plane image processed by color synthesis from the first chest CT image is obtained, and the pectoralis major muscle in the first coronal plane image is obtained by using the classification artificial intelligence model learned by the classification artificial intelligence model learning unit. and extracting the first horizontal plane images from among a plurality of horizontal plane images of the first chest CT image by predicting an existing region. According to claim 11,
The step of determining the first horizontal plane images is:
identifying coronal plane positions corresponding to an intermediate portion based on a stacking order among the stacked coronal plane images of the first chest CT image;
extracting a plurality of grayscale coronal plane images corresponding to the coronal plane locations from among the coronal plane images;
obtaining a color coronal plane image by combining the plurality of grayscale coronal plane images into one;
acquiring the first coronal plane image by fitting the color coronal plane image to a set length; and
and determining the first horizontal plane images from the first coronal plane image by the classification artificial intelligence model. According to claim 10,
The step of dividing the region of the pectoralis major muscle is:
generating a prediction image of the pectoralis major muscle using the segmentation artificial intelligence model learned by the segmentation artificial intelligence model learning unit; and
A computer-readable recording medium comprising the step of dividing the pectoralis major muscle region by excluding small regions corresponding to noise from the pectoralis major muscle prediction image. According to claim 13,
The step of dividing the region of the pectoralis major muscle is:
generating a binarized image having set pixel values by binarizing the predicted pectoralis major image;
dividing the binary image into a plurality of regions by connecting and grouping adjacent pixels in the binary image; and
Calculating the area of each of the plurality of areas, and measuring the pectoralis major muscle excluding areas whose area is less than or equal to a reference value, computer-readable recording medium.

Images