KR102615323B1 - System for automatic segmentation for maxillary sinus and lesion, method for operation thereof and method for artificial intelligent training of automatic segmentation model - Google Patents

Abstract

A method of operating a segmentation system operated by at least one processor, comprising acquiring a medical image of a normal person and a medical image of a patient with a maxillary sinus lesion, and generating a medical image of the normal person based on the characteristics of the maxillary sinus lesion included in the medical image of the patient. Create an artificial maxillary sinus lesion. A synthetic maxillary sinus lesion is added to the maxillary sinus of a normal person's medical image to generate learning data including the synthetic maxillary sinus lesion. When the learning data is input, a segmentation model is learned so that the maxillary sinus and maxillary sinus air area are output in the medical image. When a patient's medical image is input to the learned segmentation model, the patient's maxillary sinus and air area are segmented from the patient's medical image, and the maxillary sinus lesion is segmented by removing the maxillary sinus air area from the maxillary sinus.

Description

Automatic segmentation system for maxillary sinus and lesion, operation method thereof, and method for learning automatic segmentation model {System for automatic segmentation for maxillary sinus and lesion, method for operation thereof and method for artificial intelligent training of automatic segmentation model}

The present invention relates to an automatic maxillary sinus and lesion segmentation system for automatically segmenting the maxillary sinus and lesions in CT images of patients with maxillary sinus lesions based on artificial intelligence, its operation method, and an automatic segmentation model learning method.

If missing teeth are neglected, the patient's digestive function deteriorates due to weakened masticatory function. Difficulties in communication arise due to inaccurate pronunciation, and various problems such as misalignment of teeth around missing teeth and loss of gum bones occur. Therefore, to replace a patient's lost teeth, various treatment methods such as bridges, implants, and complete dentures are used.

Implant surgery is one of the treatment methods to replace lost natural teeth when teeth are lost due to various dental diseases, aging, congenital defects, or accidents. Directly placing a dental prosthesis in the upper or lower jaw is the most effective functional and aesthetically effective prosthetic treatment method.

The components of an implant are comprised of a fixture that is implanted directly into the maxilla or mandible bone to replace the root of a natural tooth, an abutment that connects the fixture to the crown, and a tooth-shaped prosthesis. . An important factor in implant surgery is determining the placement location and direction considering various factors such as the position of the patient's teeth, the condition of the alveolar bone, and the location of the maxillary sinus or inferior canal.

In general, 3D tomographic images of the patient are acquired through imaging diagnostic equipment (e.g., CBCT (Cone-Beam Computed Tomography) or MDCT (Multi-Detector Computed Tomography), etc.), and surrounding anatomical structures are identified through implant surgery planning. Considering this, the location and direction of implant placement are determined.

In particular, if the thickness of the alveolar bone remaining after extracting the maxillary molar is thin, or if the maxillary sinus descends to the area where the extracted tooth root was, making it unsuitable for implant placement, the size and location of the maxillary sinus located in the upper part of the maxilla must first be accurately determined. . After that, a maxillary sinus lift surgery must be performed to secure the amount of bone that can be used to place an implant by lifting the lower part of the maxillary sinus upward and filling the space with bone.

Therefore, in the implant surgery planning process, a three-dimensional anatomical model of the patient is required, and for this, radiological imaging and anatomical structures and lesion segmentation must be preceded. However, CBCT images have a limitation in that they have lower image contrast than MDCT.

Additionally, in CBCT images, metal shadows may occur due to braces, dental prosthesis, amalgam fillings, dental implants, etc., and movement noise may occur due to the patient's movement during imaging. Additionally, since the maxillary sinus is adjacent to the nasal cavity, ethmoid sinus, and frontal sinus, it is difficult to accurately segment the target structure among various anatomical structures, and manual segmentation takes a lot of time.

Recently, with the development of deep learning technology, many technologies for automatically segmenting anatomical structures in CBCT or MDCT have been reported. However, in order to learn a deep learning model, accurate labeling data for anatomical structures and lesions is required, and this process requires a significant amount of time and labor from medical staff. Therefore, without accurate labeling of the maxillary sinus and lesions, a deep learning model cannot be learned, and unsupervised learning-based deep learning models have the limitation of low segmentation performance because they cannot learn various aspects of the maxillary sinus and lesions.

Therefore, the present invention provides an automatic maxillary sinus and lesion segmentation system for automatically segmenting the maxillary sinus and lesions in a patient's medical image using a label-free learning method, an operating method thereof, and an automatic segmentation model learning method. .

As a method of operating a partitioned system operated by at least one processor, which is a feature of the present invention for achieving the technical object of the present invention,

Obtaining a medical image of a normal person and a medical image of a patient with a maxillary sinus lesion, and generating a synthetic maxillary sinus lesion from the medical image of the normal person based on the characteristics of the maxillary sinus lesion included in the medical image of the patient, the artificial maxillary sinus lesion Generating learning data including an artificial maxillary sinus lesion by adding a maxillary sinus lesion to the maxillary sinus of the medical image of the normal person, and learning a segmentation model so that the maxillary sinus and maxillary sinus air area are output in the medical image when the learning data is input. , and when the patient's medical image is input to the learned segmentation model, the patient's maxillary sinus and the patient's maxillary sinus air region are divided from the patient's medical image, and then the patient's maxillary sinus air region is removed from the patient's maxillary sinus. It includes the step of dividing the maxillary sinus lesion.

The step of generating the learning data includes generating a maxillary sinus area label by labeling the maxillary sinus area of the normal person based on the medical image of the normal person, and generating a synthetic maxillary sinus lesion label based on representative features of the maxillary sinus lesion. It may include a step, and a step of creating an initial artificial maxillary sinus lesion.

In the step of creating the initial artificial maxillary sinus lesion, the initial artificial maxillary sinus lesion may be created based on at least one of a filling method, a circle drawing method, and a border drawing method.

In the step of creating the initial artificial maxillary sinus lesion, the initial artificial maxillary sinus lesion may be created by filling the entire maxillary sinus area in the medical image of the normal person with the pixel intensity of the maxillary sinus lesion.

In the step of generating the initial artificial maxillary sinus lesion, a plurality of circles are generated within the maxillary sinus area in the medical image of the normal person, and among the generated circles, a circle in an area smaller than a preset standard size is generated as the initial artificial maxillary sinus lesion. You can.

In the step of creating the initial artificial maxillary sinus lesion, a membrane of a certain thickness may be created at the border of the maxillary sinus in the medical image of the normal person, and the initial artificial maxillary sinus lesion that mimics the maxillary sinus lesion may be created to be in contact with the maxillary sinus border.

After the step of creating the initial artificial maxillary sinus lesion, the pixel intensity of the initial artificial maxillary sinus lesion may be adjusted to create the artificial maxillary sinus lesion.

After the step of creating the initial artificial maxillary sinus lesion, the step of confirming the maxillary sinus air area by calculating the difference between the maxillary sinus area label of the normal person and the artificial maxillary sinus lesion label may be further included.

In the learning step, when the maxillary sinus air area excluding the artificial maxillary sinus lesion, the maxillary sinus label, and the medical image of the normal person with the artificial maxillary sinus lesion inserted are input into the segmentation model, the maxillary sinus and the maxillary sinus air area are The segmentation model can be trained to be output.

In the dividing step, the maxillary sinus lesion may be divided through the difference between the maxillary sinus air area in the maxillary sinus area containing the maxillary sinus lesion.

As a device that is another feature of the present invention for achieving the technical object of the present invention,

It includes a communication device, a memory, and at least one processor that executes instructions of a program loaded in the memory, wherein the processor displays an artificial maxillary sinus lesion in a medical image of a normal person based on the characteristics of the patient's maxillary sinus lesion. Generate learning data in which the artificial maxillary sinus lesion is added to the medical image of the normal person, and train a segmentation model with the learning data so that when the medical image is input, the maxillary sinus and maxillary sinus air area are output. When the patient's medical image is input to the learned segmentation model, the patient's maxillary sinus and maxillary sinus air area are output from the patient's medical image, and then the maxillary sinus air area is removed from the patient's maxillary sinus to create a maxillary sinus lesion. Divide.

The processor may label the maxillary sinus area based on the medical image of the normal person and generate an initial artificial maxillary sinus lesion based on at least one of a filling method, a circle drawing method, and a boundary drawing method.

The processor generates the initial artificial maxillary sinus lesion by filling the entire maxillary sinus area in the medical images of normal people with the pixel intensity of the maxillary sinus lesion, or randomly generates a plurality of circles in the maxillary sinus area in the medical images of normal people, and generates Among the circles, a circle with an area smaller than the radius of the circle preset as a standard is created as the initial artificial maxillary sinus lesion, or a membrane of a certain thickness is created at the border of the maxillary sinus in the medical images of the normal people, and the maxillary sinus lesion is made to contact the border of the maxillary sinus. The initial artificial maxillary sinus lesion that mimics the above can be created.

The processor may adjust the pixel intensity of the initial artificial maxillary sinus lesion to create the artificial maxillary sinus lesion, and determine the maxillary sinus air area by calculating the difference between the maxillary sinus label and the artificial maxillary sinus lesion label.

The processor is configured to output the maxillary sinus and maxillary sinus air areas when the maxillary sinus air area excluding the artificial maxillary sinus lesion, the maxillary sinus label, and the medical image of the normal person into which the artificial maxillary sinus lesion is inserted are input to the segmentation model. A segmentation model can be trained.

The processor may segment the maxillary sinus lesion through the difference between the maxillary sinus air area in the maxillary sinus area containing the maxillary sinus lesion.

According to the present invention, a maxillary sinus lesion is created in the maxillary sinus area through a prior knowledge-based artificial lesion creation unit in a medical image without a maxillary sinus lesion using a label-free learning method, and then learned to create an actual maxillary sinus lesion. The maxillary sinus and lesions can be automatically segmented from medical images of patients with .

In addition, compared to the existing method of dentists manually segmenting the maxillary sinus and lesions in medical images, labor is saved and segmentation variation between dentists is reduced.

In addition, it shortens the time required to plan an implant surgery and is greatly helpful in improving the implant success rate.

Figure 1 is an exemplary diagram of an environment in which an automatic maxillary sinus and lesion segmentation system according to an embodiment of the present invention is implemented.
Figure 2 is a flowchart of the operation of the automatic maxillary sinus and lesion segmentation system according to an embodiment of the present invention.
Figure 3 is a flowchart of a method for learning a maxillary sinus and lesion segmentation model according to an embodiment of the present invention.
Figure 4 is an example diagram of learning a segmentation model according to an embodiment of the present invention.
Figure 5 is an example of a segmentation model automatically segmenting a maxillary sinus lesion according to an embodiment of the present invention.
Figure 6 is an example of applying data augmentation technology to a medical image generating an artificial maxillary sinus lesion according to an embodiment of the present invention.
Figure 7 is an example of automatic segmentation of a maxillary sinus lesion in a medical image according to an embodiment of the present invention.
Figure 8 is a structural diagram of an automatic maxillary sinus and lesion segmentation system according to an embodiment of the present invention.

Below, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. In order to clearly explain the present invention in the drawings, parts that are not related to the description are omitted, and similar parts are given similar reference numerals throughout the specification.

Throughout the specification, when a part is said to “include” a certain element, this means that it may further include other elements rather than excluding other elements, unless specifically stated to the contrary.

Hereinafter, the automatic maxillary sinus and lesion segmentation system, its operation method, and automatic segmentation model learning method according to an embodiment of the present invention will be described with reference to the drawings.

Figure 1 is an exemplary diagram of an environment in which an automatic maxillary sinus and lesion segmentation system according to an embodiment of the present invention is implemented.

The automatic maxillary sinus and lesion segmentation system (hereinafter referred to as the 'segmentation system' for convenience of explanation) 100 divides the maxillary sinus and the lesion within the maxillary sinus in a medical image containing a maxillary sinus lesion.

The segmentation system 100 uses a label-free learning method to generate artificial maxillary sinus lesions through a synthetic lesion generator based on prior knowledge in medical images without maxillary sinus lesions. The artificial maxillary sinus lesion created by the lesion creation unit 110 is inserted into the maxillary sinus area of a normal person without a maxillary sinus lesion.

To this end, the segmentation system 100 is configured with a segmentation model learned to segment the maxillary sinus and the maxillary sinus air area from the maxillary sinus where the artificial maxillary sinus lesion is inserted. The learned segmentation model automatically segments the maxillary sinus and sinus lesions in medical images of patients with maxillary sinus lesions.

That is, in order to accurately segment the maxillary sinus and the maxillary sinus lesion in the medical image, the segmentation system 100 includes a segmentation model learned to segment the maxillary sinus air area and the maxillary sinus area including the lesion. Then, in the post-processing process, the segmentation system 100 obtains the difference between the maxillary sinus air area in the maxillary sinus area containing the lesion and automatically segments only the maxillary sinus lesion in the maxillary sinus area.

To perform the above-described functions, the segmentation system 100, as shown in FIG. 1, includes a lesion creation unit 110 and a lesion segmentation unit 120.

The lesion creation unit 110 creates a maxillary sinus lesion. The maxillary sinus lesion created at this time is an artificial maxillary sinus lesion to be added to the maxillary sinus extracted from medical images of normal people without maxillary sinus lesions.

That is, the lesion generator 110 adds artificial maxillary sinus lesions to medical images of normal people and generates learning medical images and label data including the artificial maxillary sinus lesions. Then, the learning medical images and label data are mapped and managed.

Here, in order to create an artificial maxillary sinus lesion, the lesion creation unit 110 uses a method of creating a maxillary sinus lesion by imitating representative aspects (domain knowledge) obtained from maxillary sinus lesions of patients with actual maxillary sinus lesions.

The lesion creation unit 110 creates an artificial maxillary sinus lesion and label using at least one of the filling method, the circle drawing method, and the maxillary sinus border drawing method. The artificial maxillary sinus lesion first created by the lesion creation unit 110 is referred to as an initial artificial maxillary sinus lesion.

In an embodiment of the present invention, the creation of an artificial maxillary sinus lesion using at least one of the filling method, the circle drawing method, and the maxillary sinus border drawing method will be described as an example. However, all three methods can be used to create artificial maxillary sinus lesions, and the method is not limited to any one method.

In order to create an artificial maxillary sinus lesion, the lesion creation unit 110 first creates an artificial maxillary sinus lesion label. The lesion creation unit 110 adjusts the pixel intensity of the initially created artificial maxillary sinus lesion to create an artificial maxillary sinus lesion that mimics the pixel aspect of an actual maxillary sinus lesion.

The reference pixel used by the lesion creation unit 110 to adjust the pixel intensity of the initial artificial maxillary sinus lesion is information corresponding to a representative aspect obtained from an actual maxillary sinus lesion. Since there are various methods for collecting information about the reference pixel used by the lesion generator 110, embodiments of the present invention are not limited to any one method.

The lesion creation unit 110 may generate an artificial maxillary sinus lesion in medical images of normal people by filling the entire maxillary sinus region with the pixel intensity of the maxillary sinus lesion through a filling method. That is, considering the case where the size of the actual patient's lesion is so large that the maxillary sinus is completely filled with the maxillary sinus lesion, the lesion creation unit 110 creates an artificial maxillary sinus lesion to fill the entire maxillary sinus area with the maxillary sinus lesion through a filling method. You can.

Additionally, the lesion generator 110 randomly generates a plurality of circles within the maxillary sinus area in medical images of normal people through a circle drawing method. In addition, among the generated circles, a circle smaller than the preset standard size, that is, the size set as the radius of circle in the hyperparameter to be explained later, can be created as an artificial maxillary sinus lesion. At this time, the number of circles generated is 10 to 30, and the size is explained as an example in which the circles are randomly generated within the range of 10 to 20 pixels in radius, but it is not necessarily limited to this.

That is, considering the case where the actual patient's lesion is relatively small and the maxillary sinus lesion occupies only a portion of the maxillary sinus, the lesion creation unit 110 may use a small area of a circle created through a circle drawing method as an artificial maxillary sinus lesion.

Additionally, the lesion creation unit 110 creates a film of a certain thickness at the border of the maxillary sinus in medical images of normal people through a border drawing method. Additionally, the lesion creation unit 110 may create an artificial maxillary sinus lesion that mimics the maxillary sinus lesion so that it contacts the border of the maxillary sinus. At this time, the thickness is explained as an example of 3 to 10 pixels, but it is not necessarily limited to this.

In this way, the lesion creation unit 110 inserts the artificial maxillary sinus lesion created by the above-described method into the maxillary sinus region extracted from the medical image of a normal person, and inserts the artificial maxillary sinus lesion into the medical image containing the artificial maxillary sinus lesion (hereinafter referred to as 'medical image for learning'). ) and maxillary sinus lesion labels can be obtained. In this way, from the learning medical image containing the artificial maxillary sinus lesion, the lesion generator 110 can extract the maxillary sinus air area as the difference between the maxillary sinus and the artificial lesion area.

The learning data including the maxillary sinus air area, maxillary sinus label data, and medical images for learning extracted from the lesion creation unit 110 are the maxillary sinus and maxillary sinus lesion segmentation model (hereinafter, for convenience of explanation) constituting the lesion segmentation unit 120. It is used as input to the 'segmentation model'). In the embodiment of the present invention, the lesion segmentation unit 120 is described as an example of being configured as a segmentation model, but it is not necessarily limited to this.

When learning data is input, the lesion segmentation unit 120 trains a segmentation model to segment the air area excluding the maxillary sinus and artificial maxillary sinus lesions in the training medical image. In order to enable the lesion dividing unit 120 to accurately segment the maxillary sinus and the maxillary sinus lesion from the medical image, the lesion dividing unit 120 divides the maxillary sinus air area and the maxillary sinus area containing the artificial maxillary sinus lesion in the learning medical image. Learn a segmentation model. This is to enable the lesion segmentation unit 120 to accurately segment the maxillary sinus and the maxillary sinus lesion from the medical image.

In order to segment the artificial maxillary sinus lesion, the lesion segmentation unit 120 trains a segmentation model into the maxillary sinus area containing the artificial maxillary sinus lesion and the maxillary sinus air area included in the learning data. Through this, the lesion dividing unit 120 can obtain the difference in the air area in the maxillary sinus area.

In the embodiment of the present invention, the use of SinusNet as a segmentation model is described as an example, but it is not necessarily limited to this. SinusNet is a U-Net-based segmentation model, and the encoder part consists of 3 × 3 convolution layers + batch normalization + ReLU, and 2 × 2 Maxpooling. And the decoder part consists of 3×3 convolution layer + batch normalization + ReLU, 2D transposed convolution layer, and skip connector layer.

When medical images of patients with maxillary sinus lesions are input, the lesion segmentation unit 120 divides the maxillary sinus and sinus lesions in the medical images through a learned segmentation model.

The operating method of the division system 100 described above will be described with reference to FIG. 2.

Figure 2 is a flowchart of the operation of the automatic maxillary sinus and lesion segmentation system according to an embodiment of the present invention.

As shown in FIG. 2, the segmentation system 100 acquires medical images to learn a segmentation model (S100). Here, the medical image includes not only a CBCT image or MDCT image of a normal person, but also a CBCT image or MDCT image of a patient with a maxillary sinus lesion. This will be explained using an example where medical images of normal people are used as learning data and verification data, and medical images of patients with maxillary sinus lesions are used to test the learned segmentation model.

The segmentation system 100 creates an artificial maxillary sinus lesion using various medical images of the patient acquired in step S100. The segmentation system 100 adds the created artificial maxillary sinus lesion to the medical image of a normal person, generating training medical images and label data including the artificial maxillary sinus lesion, and learning data including the maxillary sinus air area. Then, the segmentation system 100 learns a segmentation model using the generated training data (S200).

When medical images of patients with maxillary sinus lesions are input, the segmentation system 100, which includes a segmentation model learned using learning data, divides the maxillary sinus and the maxillary sinus air area in the input medical images. Then, the segmentation system 100 divides the maxillary sinus lesion by finding the difference in the air area of the maxillary sinus in the segmented maxillary sinus (S300).

At this time, a method in which the segmentation system 100 creates an artificial maxillary sinus lesion and learns a segmentation model in step S200 will be described with reference to FIG. 3 .

Figure 3 is a flowchart of a method for learning a maxillary sinus and lesion segmentation model according to an embodiment of the present invention.

As shown in FIG. 3, in order to create an artificial maxillary sinus lesion, the segmentation system 100 labels the maxillary sinus area based on a medical image of a normal person among a plurality of medical images collected first (S201). Since there are various methods by which the segmentation system 100 labels the maxillary sinus region, embodiments of the present invention are not limited to any one method.

The segmentation system 100 labels the maxillary sinus region and then generates a synthetic maxillary sinus lesion label (S202). That is, the maxillary sinus area is identified from the medical image of a patient with a maxillary sinus lesion, and an artificial maxillary sinus lesion label is created by imitating the aspects that typically appear in the maxillary sinus lesion (S202).

The segmentation system 100 performs one of the filling method, the circle drawing method, and the border drawing method to generate an initial artificial maxillary sinus lesion (hereinafter referred to as an 'initial artificial maxillary sinus lesion' for convenience of explanation) (S203 ).

That is, the segmentation system 100 can generate an initial artificial maxillary sinus lesion in medical images of normal people by filling the entire maxillary sinus region with the pixel intensity of the maxillary sinus lesion through the filling method.

Additionally, the segmentation system 100 randomly generates a plurality of circles within the maxillary sinus region in medical images of normal people through a circle drawing method. Also, among the created circles, a circle area whose size is smaller than the size determined by the radius of the circle in the hyperparameters shown in Table 1 below can be created as an artificial maxillary sinus lesion.

Additionally, the segmentation system 100 generates a film of a certain thickness at the boundary of the maxillary sinus in medical images of normal people through a boundary drawing method. Additionally, the segmentation system 100 may create an artificial maxillary sinus lesion that mimics a maxillary sinus lesion so that it contacts the maxillary sinus border.

In an embodiment of the present invention, the creation of an artificial maxillary sinus lesion is described as an example using the hyperparameters shown in Table 1 below, but it is not necessarily limited to this.

hyper
parameter Number of shapes
(circles) Pixel value range(pixels) Radius of Circle (pixels) Edge width (pixels) Kernel size of Median filter range 10~30 40~180 10~20 3~10 5

That is, when the segmentation system 100 generates a synthetic maxillary sinus lesion in a medical image using a circle drawing method, the number of circles is 10 to 30 and the size of the circles is randomly within the range of 10 to 20 pixels in radius. creates lesions. Additionally, the segmentation system 100 can create a synthetic maxillary sinus lesion that mimics a maxillary sinus lesion adjacent to the maxillary sinus border by creating a membrane 3 to 10 pixels thick at the border of the maxillary sinus in a medical image through a boundary drawing method.

Additionally, the segmentation system 100 randomly assigns pixel values in the range of 40 to 180 for the pixel intensity of the artificial maxillary sinus lesion and then applies a 5×5 median filter.

In this way, the segmentation system 100 creates an artificial maxillary sinus lesion in the maxillary sinus area in a medical image of a normal person, and generates a training medical image including the artificial maxillary sinus lesion and an artificial maxillary sinus lesion label.

The segmentation system 100 adjusts the pixel intensity of the initial artificial maxillary sinus lesion created in step S203 to create an artificial maxillary sinus lesion that mimics the pixel aspect of an actual maxillary sinus lesion (S204). At this time, the segmentation system 100 sets only the minimum and maximum pixel intensity values and randomly adjusts the pixel intensity for each initial artificial maxillary sinus lesion within the range of the minimum and maximum values. Since the segmentation system 100 can adjust pixel intensity using various techniques or methods, embodiments of the present invention are not limited to any one method.

In this way, the segmentation system 100 creates a synthetic maxillary sinus lesion in the maxillary sinus area in the medical image of a normal person and then adds the artificial maxillary sinus lesion to the medical image of a normal person (S205). Then, the segmentation system 100 determines the maxillary sinus air area by obtaining the difference between the maxillary sinus label and the artificial maxillary sinus lesion label (S206).

This allows the segmentation model to segment the air region more easily than segmenting the maxillary sinus lesion in the maxillary sinus. Therefore, in order to find a maxillary sinus lesion by segmenting the maxillary sinus and the air area in the medical image and finding the difference between the air areas in the maxillary sinus, the segmentation system 100 identifies the maxillary sinus air area in the maxillary sinus area.

The segmentation system 100 trains a segmentation model using training data (S207). That is, the segmentation system 100 uses the maxillary sinus air area excluding the artificial maxillary sinus lesion, maxillary sinus label data, and medical images as inputs for learning the segmentation model.

And the segmentation system 100 trains the segmentation model so that when learning data is input to the segmentation model, the maxillary sinus and maxillary sinus air area are output. Additionally, the lesion segmentation unit 120 learns a segmentation model to segment the maxillary sinus lesion through the difference in the maxillary sinus air area in the maxillary sinus containing the lesion.

An example in which the segmentation system 100 learns a segmentation model through the above-described procedure and an example in which a maxillary sinus lesion is segmented in a medical image will be described with reference to FIGS. 4 to 6 . In an embodiment of the present invention, recognition of a synthetic maxillary sinus lesion and a maxillary sinus air area in the maxillary sinus area will be described as an example using the three artificial maxillary sinus lesions shown in FIG. 4.

Figure 4 is an example diagram of learning a segmentation model according to an embodiment of the present invention.

As shown in FIG. 4, the segmentation system 100 generates a maxillary sinus label (②) by labeling the maxillary sinus area of a normal person in a medical image (①) of a normal person. Then, the segmentation system 100 creates an initial artificial maxillary sinus lesion (③) and then adjusts the pixel intensity of the initial artificial maxillary sinus lesion to create an artificial maxillary sinus lesion (④).

The segmentation system 100 generates a medical image for learning (⑤) by adding the artificial maxillary sinus lesion (④) to the medical image (①) of a normal person. Then, the segmentation system 100 determines the maxillary sinus air area (⑥) by calculating the difference between the label of the artificial maxillary sinus lesion (④) and the maxillary sinus label (②).

In this way, after the training medical image (⑤) with the maxillary sinus label (②) and the artificial maxillary sinus lesion added is generated from the normal person's medical image (①), the segmentation system 100 uses the normal person's medical image (①) for learning purposes. When a medical image (⑤) is input, a segmentation model is trained to output the maxillary sinus and maxillary sinus air area (⑦).

The maxillary sinus and maxillary sinus air areas obtained as output are used as input to the loss function to learn the segmentation model. And by comparing the correct data in the verification data, that is, the maxillary sinus label (②) and maxillary sinus air area (⑥), with the output of the maxillary sinus and maxillary sinus air area (⑦) predicted by the segmentation system 100, the currently predicted image is the correct answer. Update the segmentation model by calculating how much it is different from the other. For this purpose, the medical image (①), maxillary sinus label (②), learning image (⑤), and maxillary sinus air area (⑥) in FIG. 4 are referred to as verification data.

In order to learn a segmentation model, the embodiment of the present invention uses the MDL (Multi-label Dice Loss) loss function, and the MDL loss function can be defined as shown in Equation 1 below.

Here, n is the number of pixels, K is the number of classes (background, maxillary sinus area containing maxillary sinus lesions, air area), G is the correct answer data, and P is the prediction result. Here, the MDL loss function is an already known technology, and detailed description is omitted in the embodiments of the present invention.

The partition system 100 repeats learning of the partition model 400 times (epoch), calculates a loss function after learning once, and updates the weight of the partition model in a direction that minimizes the loss value.

In an embodiment of the present invention, an example of securing the number of data used to learn a segmentation model in consideration of maxillary sinus and lesion characteristics will be described with reference to FIG. 5.

Figure 5 is an example of applying data enhancement technology to a medical image generating an artificial maxillary sinus lesion according to an embodiment of the present invention.

As shown in Figure 5, in an embodiment of the present invention, the number of training data used for learning is increased through data augmentation technology and used to train a segmentation model.

Figure 5 (a) shows the original medical image, and (b) is an image created by adjusting the brightness of the original medical image. (c) is a medical image that is blurrier than the original medical image by applying Gaussian blur to the original medical image, and (d) is a training medical image obtained by rotating the original medical image.

In an embodiment of the present invention, a new medical image can be obtained by rotating the original image within a range of -20° to +20°. Additionally, a new medical image can be obtained by brightening or darkening the brightness of the original image within the range of -20% to +20%, and a medical image that is blurred compared to the original image can be obtained by using Gaussian blur (σ=0.5). there is. Ranges stated in embodiments of the present invention are not always limited as such.

Figure 6 is an example of a segmentation model automatically segmenting a maxillary sinus lesion according to an embodiment of the present invention.

After the above-described segmentation model learning procedure of FIG. 4 is completed, when the medical image (⑧) of a patient with a maxillary sinus lesion is input as shown in FIG. 6, the segmentation system 100 generates the patient's medical image (⑧) through the segmentation model. Extract the maxillary sinus (⑨) and maxillary sinus air area (⑩) from ⑧). Then, the segmentation system 100 obtains an inverted image of the maxillary sinus air area (⑩) and detects the maxillary sinus lesion from the patient's medical image (⑧) through pixel-wise multiplication of the maxillary sinus (⑨) and the inverted maxillary sinus air area. (⑪) is automatically divided.

Figure 7 is an example of automatic segmentation of a maxillary sinus lesion in a medical image according to an embodiment of the present invention.

Each row shown in Figure 7 represents the division of the maxillary sinus (Sinus), the maxillary sinus air area (Air), and the maxillary sinus lesion (Lesion). The part marked with the first display means (yellow, ⑫) represents the correct answer area, and the portion marked with the second display means (red, ⑬) represents the lesion prediction area.

In an embodiment of the present invention, F1-score (F1), Precision (PR), and Recall (RC) are used to compare the accuracy of automatic segmentation of maxillary sinus and sinus lesions. PR is the ratio of those predicted to be correct to those that are actually correct. If only a part of the correct answer is predicted, it becomes 100%. RC is a number that indicates how successful the prediction was in the correct answer that was supposed to be correct. Contrary to PR, if the prediction is greater than the correct answer, it becomes 100%. And F1 is the average of PR and RC.

At this time, the results of automatic maxillary sinus lesion segmentation according to the artificial lesion creation technique used in label-free learning are shown in Table 2.

Lesion creation technique maxillary sinus maxillary sinus air area Maxillary sinus lesions replenishment boundary one F1 PR R.C. F1 PR R.C. F1 PR R.C. √ 97.0±1.8 96.7±1.8 97.4±2.4 79.3±18.0 97.5±1.4 70.3±23.8 41.4±18.0 86.1±63 29.6±17.0 √ 97.1±1.4 96.4±2.1 97.9±1.3 87.3±10.1 97.8±1.2 80.2±15.6 61.5±13.5 89.4±8.2 48.4±15.9 √ 96.4±2.9 97.2±1.6 95.7±4.6 88.6±8.2 97.8±1.3 81.9±12.9 60.2±21.1 86.9±13.8 48.5±21.1 √ √ √ 96.6±1.7 95.8±2.4 97.4±2.1 95.0±2.2 94.7±3.5 95.4±2.9 80.9±11.6 82.7±9.1 80.1±15.0

In the embodiment of the present invention, it can be seen that the highest maxillary sinus lesion segmentation performance is achieved when all three artificial lesion creation techniques are used.

Figure 8 is a structural diagram of a computing system according to an embodiment of the present invention.

Referring to FIG. 8, the automatic maxillary sinus and lesion segmentation system 100 operated by at least one processor may be implemented as a computing system 200, and is described to execute the operations of the present invention in the computing system 200. Run a program containing instructions. Programs can be stored and distributed in computer-readable storage media.

The hardware of the computing system 200 may include at least one processor 210, memory 220, storage 230, and communication interface 240, and may be connected through a bus. In addition, hardware such as input devices and output devices may be included. The computing system 200 may be equipped with various software, including an operating system capable of running programs.

The processor 210 is a device that controls the operation of the computing system 200 and may be various types of processors that process instructions included in a program, for example, a Central Processing Unit (CPU) or a Micro Processor Unit (MPU). ), MCU (Micro Controller Unit), GPU (Graphic Processing Unit), etc.

The memory 220 loads the corresponding program so that instructions described to execute the operations of the present invention are processed by the processor 210. The memory 220 may be, for example, read only memory (ROM), random access memory (RAM), etc. The storage 230 stores various data, programs, etc. required to execute the operations of the present invention. The communication interface 240 may be a wired/wireless communication module.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements can be made by those skilled in the art using the basic concept of the present invention defined in the following claims. It falls within the scope of rights.

Claims (16)

1. A method of operating a partitioned system operated by at least one processor, comprising:
Acquiring a medical image of a normal person and a medical image of a patient with a maxillary sinus lesion, and generating an artificial maxillary sinus lesion from the medical image of the normal person based on the characteristics of the maxillary sinus lesion included in the medical image of the patient;
Adding the artificial maxillary sinus lesion to the maxillary sinus of the medical image of the normal person, generating an initial artificial maxillary sinus lesion based on at least one of a filling method, a circle drawing method, and a border drawing method, and generating the initial artificial maxillary sinus lesion from the initial artificial maxillary sinus lesion. Generating training data including a maxillary sinus lesion of a synthetic human,
When the learning data is input, learning a segmentation model to output the maxillary sinus and maxillary sinus air area in the medical image, and
When the patient's medical image is input to the learned segmentation model, the patient's maxillary sinus and the patient's maxillary sinus air area are divided from the patient's medical image, and then the patient's maxillary sinus air area is removed from the patient's maxillary sinus. Steps to segment the lesion
A method of operation, including. According to paragraph 1,
The step of generating the learning data is,
Generating a maxillary sinus label by labeling the maxillary sinus area of the normal person based on the medical image of the normal person, and
A step of generating an artificial maxillary sinus lesion label based on the representative characteristics of the maxillary sinus lesion.
A method of operation, including.
delete According to paragraph 2,
The step of creating the initial artificial maxillary sinus lesion is,
An operation method of generating the initial artificial maxillary sinus lesion by filling the entire maxillary sinus region in the medical image of the normal person with the pixel intensity of the maxillary sinus lesion. According to paragraph 2,
The step of creating the initial artificial maxillary sinus lesion is,
An operation method of generating a plurality of circles in the maxillary sinus area in the medical image of the normal person, and generating a circle smaller than a preset standard size among the generated circles as the initial artificial maxillary sinus lesion. According to paragraph 2,
The step of creating the initial artificial maxillary sinus lesion is,
An operation method of generating a film of a certain thickness at the border of the maxillary sinus in the medical image of the normal person, and generating the initial artificial maxillary sinus lesion that mimics the maxillary sinus lesion to contact the border of the maxillary sinus. According to paragraph 2,
After the step of creating the initial artificial maxillary sinus lesion,
An operating method for generating the artificial maxillary sinus lesion by adjusting the pixel intensity of the initial artificial maxillary sinus lesion. According to paragraph 2,
After the step of generating the artificial maxillary sinus lesion label,
Confirming the maxillary sinus air area by calculating the difference between the maxillary sinus lesion label of the normal person and the maxillary sinus lesion label of the artificial person.
An operation method further comprising: According to clause 8,
The learning step is,
When the maxillary sinus air area excluding the artificial maxillary sinus lesion, the maxillary sinus label, and the medical image of the normal person with the artificial maxillary sinus lesion inserted are input into the segmentation model, the segmentation model is output so that the maxillary sinus and the maxillary sinus air area are output. Learning how to operate. According to clause 9,
The dividing step is,
An operating method that divides a maxillary sinus lesion through the difference in the maxillary sinus air area from the maxillary sinus area containing the maxillary sinus lesion. communication device,
memory, and
At least one processor that executes instructions of a program loaded into the memory,
The processor is
Label the maxillary sinus area based on the medical image of the normal person, create an initial artificial maxillary sinus lesion based on at least one of the filling method, circle drawing method, and border drawing method, and create an initial artificial maxillary sinus lesion based on the characteristics of the patient's maxillary sinus lesion. A synthetic maxillary sinus lesion is created from a medical image of a normal person to generate learning data in which the artificial maxillary sinus lesion is added to the medical image of a normal person, and a segmentation model is trained using the learning data to generate a maxillary sinus and maxillary sinus air area when the medical image is input. The segmentation model is trained to output, and when the medical image of the patient is input with the learned segmentation model, the patient's maxillary sinus and maxillary sinus air area are output from the patient's medical image, and then the maxillary sinus of the patient is output. Maxillary sinus and sinus lesion segmentation system, which divides the maxillary sinus lesion by removing the air region.
delete According to clause 11,
The processor,
The initial artificial maxillary sinus lesion is created by filling the entire maxillary sinus region in the medical images of the normal people with the pixel intensity of the maxillary sinus lesion, or
In the medical images of the normal people, a plurality of circles are randomly generated in the maxillary sinus area, and among the generated circles, a circle in an area smaller than the radius of the circle preset as a standard is generated as the initial artificial maxillary sinus lesion, or
A maxillary sinus and maxillary sinus lesion segmentation system that generates a film of a certain thickness at the border of the maxillary sinus in the medical images of normal people and creates the initial artificial maxillary sinus lesion that mimics the maxillary sinus lesion to contact the border of the maxillary sinus. According to clause 13,
The processor,
A maxillary sinus and maxillary sinus lesion segmentation system that adjusts the pixel intensity of the initial artificial maxillary sinus lesion to create the artificial maxillary sinus lesion, and determines the maxillary sinus air area by calculating the difference between the maxillary sinus label and the artificial maxillary sinus lesion label. According to clause 14,
The processor,
When the maxillary sinus air area excluding the artificial maxillary sinus lesion, the maxillary sinus label, and the medical image of the normal person with the artificial maxillary sinus lesion inserted are input to the segmentation model, the segmentation model is learned so that the maxillary sinus and the maxillary sinus air area are output. Shiki, maxillary sinus and maxillary sinus lesion segmentation system. According to clause 15,
The processor,
A maxillary sinus and sinus lesion segmentation system that divides a maxillary sinus lesion through the difference in the maxillary sinus air area in the maxillary sinus area containing the maxillary sinus lesion.

Images