KR20210045577A - Method for planning a non-invaseve treatment using ct image generated from brain mri image based on artificial intelligence - Google Patents

Abstract

Provided is a non-invasive treatment planning method based on artificial intelligence. The non-invasive treatment planning method using CT image generated from a brain MRI image based on artificial intelligence, according to an aspect of the present invention, comprises the following steps of: acquiring an MRI image of the patient's brain; generating a synthetic CT image corresponding to the brain MRI image using a learning model generated by machine learning in advance based on an artificial intelligence algorithm; extracting bone tissue information from the synthetic CT image; and setting a non-invasive treatment plan by providing an external energy source based on the lesion information extracted from the MRI image and the bone tissue information.

Description

Non-invasive treatment planning method using CT images generated from artificial intelligence-based brain MRI images{METHOD FOR PLANNING A NON-INVASEVE TREATMENT USING CT IMAGE GENERATED FROM BRAIN MRI IMAGE BASED ON ARTIFICIAL INTELLIGENCE}

The present invention relates to a non-invasive treatment planning method using a CT image generated from an artificial intelligence-based brain MRI image, and more particularly, to a method of establishing a non-invasive treatment plan using only MRI imaging without CT imaging.

Brain disease treatment increases the risk of treatment because the skull must be opened when invasive treatment is applied. Therefore, non-invasive treatment using external energy sources (eg, radiation, ultrasound) is emerging. However, in terms of treating neurological disorders, if the error is large in the process of setting a non-invasive treatment plan, and a dose from an external energy source is given to a wider range than the area where the lesion is located, damage to the brain tissue may occur. If an external energy source is provided within a narrow range, there is a problem in that treatment is not properly performed.

Meanwhile, in the treatment of brain diseases, the clinical application of gamma knife radiation surgery is gradually expanding as a treatment for minimizing damage to major brain regions and maximizing the therapeutic effect. Currently, gamma knife radiosurgery is used to treat a variety of diseases, from neoplastic diseases such as malignant and benign brain tumors to functional brain diseases such as Parkinson's disease, ophthalmic diseases and cerebrovascular diseases, and gamma knife radiation surgery for each disease. The application of the application is also continuously increasing. This gamma knife radiosurgery consists of four steps: frame fixation, brain image acquisition, treatment plan establishment, and treatment implementation, and the treatment plan is 1mm error based on brain MRI (magnetic resonance) image data acquired while the frame is fixed to the head. This is done under the assumption of a limit. In particular, in calculating the dose distribution of a radiation treatment plan, the thickness and location of individual bone tissues (eg, skull) are important factors to consider.

Specifically, during the process of establishing a treatment plan for gamma knife radiation surgery-during the course of treatment implementation, the dose distribution during intracranial irradiation is calculated by an algorithm using an index of tissue maximum ratio (TMR) 10. There was a limitation in that it was an algorithm that did not reflect the difference in individual skulls, which had the greatest effect on energy absorption during the slow line survey. In other words, the TMR 10 algorithm presupposes that the patient's head is all made up of the same density as water, but does not reflect this in that the radiation is attenuated from the skull as it enters the skull because the attenuation constant of the actual bone is higher than that of water. The TMR 10 algorithm has a problem that there is a possibility that the radiation dose may be incorrectly entered into an incorrect location. The brain is an important structure that is in charge of all functions of the body as a whole, and all parts are important because each part of the brain has different functions, but if radiation is irradiated to normal tissues due to inaccurate radiation surgery, the patient may suffer from neurological disorders. Radiation-induced tissue toxicity that may cause or cause life-threatening may appear, and if an inaccurate dose is entered into a tumor tissue that requires a radiation dose above a certain level, the tumor may recur. Therefore, the above problem has a fatal result in patient treatment. Can result.

Recently, GammaPlanㄾ Convolution, which calculates the dose by reflecting the difference between individual skulls using CT images, has been developed, but 1) additional brain CT scans are required due to the nature of gamma knife radiation surgery that establishes a treatment plan based on MRI. 2) If the basic image of the brain stereotactic position calculation performed after frame fixation is selected as a brain CT image and then a treatment plan is established by matching the existing brain MRI images, the treatment error due to the registration error between the images is reduced. There was a problem that it occurred. That is, feature points are extracted from the CT image and the MRI image, and image matching is performed between the CT image and the MRI image through matching of the extracted feature points. However, the CT image and the MRI image have a large difference in resolution, image quality, field of view, and contrast for each region, and due to this difference, it is difficult to match images between different images, and thus there is a problem that a treatment error is likely to occur.

Meanwhile, a technology called deep learning among machine learning algorithms has recently been in the spotlight in various fields. In particular, in the field of object recognition, a technology called CNN (convolutional neural network), which is a kind of deep learning, is in the spotlight. CNN is a model that simulates human brain function based on the assumption that when a person recognizes an object, it extracts basic features of an object, then performs complex calculations in the brain and recognizes the object based on the result. CNNs are generally used with various filters for extracting features of an image through a convolution operation, and a pooling or non-linear activation function for adding nonlinear features (e.g., sigmoid, ReLU (rectified)). linear unit), etc.) can be used.

Along with the increasing interest in deep learning, interest in deep learning is also increasing in various medical devices (eg, ultrasound, computed tomography (CT), magnetic resonance imaging (MRI), etc.).

(Patent Document 0001) KR 10-2016-0050326

The problem to be solved by the present invention is to provide a CT image generated from an artificial intelligence-based brain MRI image that can prevent additional external energy source irradiation to the patient and prevent treatment errors due to inter-image registration errors in applying non-invasive treatment. It is to provide a non-invasive treatment planning method used.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems that are not mentioned will be clearly understood by those skilled in the art from the following description.

A non-invasive treatment planning method using a computed tomography (CT) image generated from an artificial intelligence-based brain magnetic resonance imaging (MRI) image according to an aspect of the present invention for solving the above problems Silver, obtaining a brain MRI image of the patient; Generating a synthetic CT image corresponding to the brain MRI image using a learning model generated by machine learning in advance based on an artificial intelligence algorithm; Extracting bone tissue information from the composite CT image; And setting a non-invasive treatment plan by providing an external energy source based on the lesion information extracted from the MRI image and the bone tissue information.

Other specific details of the present invention are included in the detailed description and drawings.

According to the present invention as described above, it has various effects as follows.

The present invention can obtain bone tissue data without exposure to radiation in the treatment of neurological diseases to which non-invasive treatment according to an external energy source is applied without opening the skull.

In addition, according to the present invention, since a separate CT image is not taken to establish a non-invasive treatment plan, additional radiation exposure to the patient can be prevented and the safety of the patient can be improved.

In addition, according to the present invention, an accurate non-invasive treatment plan can be established and treatment efficiency can be improved by reflecting a bone structure using an MRI image and a synthetic CT image.

In addition, the present invention can prevent unnecessary external energy source irradiation to normal tissues around the lesion.

In addition, the present invention can generate a learning model with very robustness by using an artificial intelligence algorithm.

In addition, the present invention generates a composite CT image using an artificial intelligence algorithm instead of matching an MRI image and a CT image, thereby increasing an image processing speed and reducing an error.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a block diagram showing a non-invasive treatment planning apparatus according to an embodiment of the present invention.
2 is a flowchart illustrating a method of planning a non-invasive treatment according to an embodiment of the present invention.
3 is a flowchart illustrating a method of generating a learning model according to an embodiment of the present invention.
4 is a block diagram illustrating a CGAN according to an embodiment of the present invention.
5 is an exemplary diagram for explaining augmentation of learning data according to an embodiment of the present invention.
6 and 7 are exemplary diagrams for explaining the precision of a composite CT image according to an embodiment of the present invention.

Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in a variety of different forms. It is provided to fully inform the skilled person of the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terms used in the present specification are for describing exemplary embodiments and are not intended to limit the present invention. In this specification, the singular form also includes the plural form unless specifically stated in the phrase. As used herein, “comprises” and/or “comprising” do not exclude the presence or addition of one or more other elements other than the mentioned elements. Throughout the specification, the same reference numerals refer to the same elements, and "and/or" includes each and all combinations of one or more of the mentioned elements. Although "first", "second", and the like are used to describe various elements, it goes without saying that these elements are not limited by these terms. These terms are only used to distinguish one component from another component. Therefore, it goes without saying that the first component mentioned below may be the second component within the technical idea of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used with meanings that can be commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not interpreted ideally or excessively unless explicitly defined specifically.

Spatially relative terms "below", "beneath", "lower", "above", "upper", etc. It can be used to easily describe the correlation between a component and other components. Spatially relative terms should be understood as terms including different directions of components during use or operation in addition to the directions shown in the drawings. For example, if a component shown in a drawing is turned over, a component described as "below" or "beneath" of another component will be placed "above" the other component. I can. Accordingly, the exemplary term “below” may include both directions below and above. Components may be oriented in other directions, and thus spatially relative terms may be interpreted according to the orientation.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing a non-invasive treatment planning apparatus according to an embodiment of the present invention.

Referring to FIG. 1, a non-invasive treatment planning apparatus 100 according to an embodiment of the present invention may establish a non-invasive treatment plan using an acquired MRI image and existing learning data. For example, the non-invasive treatment planning apparatus 100 may input MRI images and CT images of each of a plurality of patients as learning data into an artificial intelligence algorithm to generate a learning model, and use the generated learning model to generate a brain MRI. CT images can be generated from images, and non-invasive treatment plans can be established using brain MRI images and CT images.

In one embodiment, the non-invasive treatment planning apparatus 100 may be a server, and a dedicated program for setting a non-invasive treatment plan may be installed. For example, the non-invasive treatment planning apparatus 100 may pre-process (e.g., augment data, crop images) the acquired medical images and learning data, and the data acquisition unit 110 capable of acquiring medical images and learning data. A data processing unit 120, a deep learning unit 130 that performs machine learning using learning data and artificial intelligence algorithms, and generates a learning model, and an image generation unit that generates a CT image from an MRI image using a learning model. (140), a treatment plan setting unit 150 that establishes a non-invasive treatment plan using lesion information and bone tissue information (e.g., skull data), and converts learning results, learning models, patient data, and treatment result data into big data. It may include a database 160 that can be stored.

2 is a flowchart illustrating a method of planning a non-invasive treatment according to an embodiment of the present invention. The operations of FIG. 2 may be performed by each of the components of FIG. 1.

Referring to FIG. 2, in an embodiment, in operation 21, the data acquisition unit 110 may acquire a brain MRI image of a patient. For example, the data acquisition unit 110 may receive a brain MRI image from an MRI imaging apparatus through a network. In addition, the data acquisition unit 110 may acquire medical images such as CT images. Here, the CT image is computed tomography (CT) and the MRI image is magnetic resonance imaging (MRI). For example, the MRI image and the CT image are voxel data, and may be formed of a plurality of slices, that is, a plurality of unit images.

In an embodiment, in operation 22, the image generator 140 may generate a composite CT image corresponding to the brain MRI image using a learning model generated by machine learning in advance based on an artificial intelligence algorithm. For example, the artificial intelligence algorithm may be a Condicitonal Generative Adversarial Network (GAN). The operation of generating a learning model by machine learning will be described in detail later in FIG. 3.

In an embodiment, in operation 23, the treatment plan setting unit 150 may extract bone tissue information from the composite CT image. For example, bone tissue may refer to all bone tissues included in the brain, such as a skull. The bone tissue information may include at least one of the thickness of the bone tissue, the density of the bone tissue, and the location of the bone tissue. In particular, important factors in a treatment plan may be the thickness, density and location of the skull. This is because the skull is the factor that has the greatest influence on the irradiation dose of an external energy source.

In an embodiment, in operation 24, the treatment plan setting unit 150 may set a non-invasive treatment plan by providing an external energy source based on the lesion information and bone tissue information extracted from the MRI image. For example, the external energy source may be ultrasound or radiation, as long as it can treat a patient's lesions and emit electromagnetic waves into the patient's brain. The lesion information may include at least one of the location, size, and thickness of the lesion, and the location, size, and thickness of the normal tissue around the lesion.

In an embodiment, the treatment plan setting unit 150 may determine an irradiation dose, an irradiation range, an irradiation direction, an irradiation time, and a three-dimensional irradiation coordinate of an energy source for treating neurological diseases based on the lesion information and bone tissue information. For example, the treatment plan setting unit 150 may input lesion information and bone tissue information into the Leksell GammaPlanㄾ-Convolution algorithm to determine the irradiation dose, irradiation range, irradiation direction, irradiation time, and three-dimensional irradiation coordinates of the radiation.

Meanwhile, although not shown in the drawing, the treatment plan setting unit 150 may verify the validity of the composite CT image before establishing a treatment plan. For example, after calculating the Hounsfield Unit (HU) value of the patient's original CT image and calculating the Hounsfield Unit value of the composite CT image, the Hounsfield unit value of the original CT image and the composite CT image By comparing the Hounsfield unit values, the validity of the composite CT image can be verified. For example, as the Hounsfield unit value of the composite CT image is the same as or closer to the Hounsfield unit value of the original CT image, the composite CT image can be confirmed as valid data. In brain treatment using radiation, the skull has the most influence on the radiation dose reaching the lesion of the brain, and the Hounsfield unit best represents the radiation absorbed dose of the skull, so it can be used for validation.

3 is a flowchart illustrating a method of generating a learning model according to an embodiment of the present invention. 4 is a block diagram illustrating a CGAN according to an embodiment of the present invention. 5 is an exemplary diagram for explaining augmentation of learning data according to an embodiment of the present invention. 6 and 7 are exemplary diagrams for explaining the precision of a composite CT image according to an embodiment of the present invention. The operations of FIG. 3 may be performed by each of the components of FIG. 1.

The operations described in FIG. 3 may be operations of inputting training data including MRI images and original CT images of each of a plurality of patients into an artificial intelligence algorithm, and generating a learning model through the artificial intelligence algorithm. Here, the artificial intelligence algorithm uses Condicitonal GAN (Generative Adversarial Network).

3 to 7, in an embodiment, in operation 31, the data acquisition unit 110 may acquire training data including MRI images and original CT images of each of a plurality of patients. For example, the learning data may be a pair of brain MRI images and brain CT images of a plurality of patients with brain diseases, and may include all medical images containing information on diseases related to the brain.

In an embodiment, in operation 32, the data processing unit 120 may augment the training data. For example, the data processing unit 120 may augment the learning data by rotating or flipping the MRI image and the original CT image included in the training data, and accordingly, the amount of learning data is abundant. Can be secured. For example, as disclosed in FIG. 5, the data processing unit 120 may increase the amount of data by rotating or flipping the original image. Of course, operation 32 may be omitted depending on the amount of previously secured learning data.

In an embodiment, in operation 33, as illustrated in FIG. 4, the deep learning learning unit 130 conditionally inputs random noise data to a generator based on the MRI image included in the training data. For example, the generator may be a fully convolutional network (FCN).

In an embodiment, in operation 34, the deep learning learning unit 130 may generate a composite CT image through a generator. For example, the generator may conditionally generate a composite CT image from random noise data using an MRI image.

In an embodiment, in operation 35, the deep learning learning unit 130 may input the generated synthetic CT image (pseudo CT) and the original CT image corresponding to the MRI image included in the training data into the discriminator. have. For example, the discriminator may be a convolutional neural network (CNN). Of course, the other discriminator is not limited to CNN, and may vary depending on the purpose of use of the image processing device, such as a deep neural network (DNN) or a recurrent neural network (RNN).

In an embodiment, in operation 36, the deep learning learning unit 130 may determine whether the discriminator is the original composite CT image. For example, the discriminator can output a probability value of 1 if the composite CT image is real, and output a probability value of 0 if it is considered fake, and if it is difficult to distinguish whether it is real or fake, it can output a probability value of 0.5. .

In an embodiment, in operation 37, the deep learning learning unit 130 may generate a learning model by repeating operations 31 to 36. For example, the deep learning learning unit 130 may generate a learning model by repeatedly performing an operation of generating a synthesized CT image through a generator and an operation of determining whether the synthesized CT image is an original through a discriminator. For example, if the discriminator continuously outputs the synthesized CT image generated by the generator with a probability value close to 0.5, the learning model can be a very sophisticated learning model. Accordingly, the image generator 140 may generate an accurate composite CT image based on an MRI image through a learning model including a generator.

As such, the existing analytic methods (such as one-to-one matching of MRI intensity and CT intensity (HU)) have limitations in that they cannot reflect all of the various variability of the image quality of MRI and CT. When using a deep learning algorithm represented by CGAN (Conditional Generative Adversarial Network) as in the present invention, a model with very robustness can be created. In addition, the present invention is non-invasive because the inference process of the deep learning model learned based on the GPU (graphics processing unit) is very fast (MR->CT conversion is very fast, less than 1 sec per slice). The speed of planning surgery can be significantly increased.

Meanwhile, in order to confirm the effect of the present invention, a CT/MR image pair of 19 patients with glioblastoma cerebral glioblastoma of TCGA (The cancer genome atlas) is used to learn with Conditional Generative Adversarial Network (CGAN) to synthesize CT images from MRI images. Preliminary research was conducted, data augmentation was performed to compensate for insufficient data, and learning was conducted by securing a total of 1,580 slice image pairs, and when compared with actual CT images, Dice similarity coefficient (DSC) As a standard, it was confirmed that the synthesis accuracy of 0.934 was shown. That is, the images shown in FIG. 6 are examples of MR-CT conversion research results. In each, (left) patient's MR image, (middle) CT image converted to a learned network, (right) patient's actual CT image

A non-invasive treatment planning method using a computed tomography (CT) image generated from an artificial intelligence-based brain magnetic resonance imaging (MRI) image according to an aspect of the present invention includes an MRI image of a patient's brain. Obtaining a; Generating a synthetic CT image corresponding to the brain MRI image using a learning model generated by machine learning in advance based on an artificial intelligence algorithm; Extracting bone tissue information from the composite CT image; And setting a non-invasive treatment plan by providing an external energy source based on the lesion information extracted from the MRI image and the bone tissue information.

According to various embodiments, in the setting of the non-invasive treatment plan, the irradiation dose, irradiation range, irradiation direction, irradiation time, and three-dimensional irradiation coordinates of an energy source for treating neurological diseases are determined based on the lesion information and the skull information. It may include the step of determining.

According to various embodiments, the step of setting the non-invasive treatment plan includes inputting the lesion information and the skull information to the Leksell GammaPlan ^ㄾ -Convolution algorithm to receive the radiation dose, the radiation range, the radiation direction, the radiation time, and the three-dimensional radiation. It may include the step of determining the coordinates.

According to various embodiments, calculating a Hounsfield Unit (HU) value of the original CT image of the patient; Calculating a Hounsfield unit value of the composite CT image; And verifying validity of the synthesized CT image by comparing a Hounsfield unit value of the original CT image with a Hounsfield unit value of the synthesized CT image.

According to various embodiments, it may include inputting training data including MRI images and original CT images of each of a plurality of patients into the artificial intelligence algorithm, and generating the learning model through the artificial intelligence algorithm. .

According to various embodiments, the artificial intelligence algorithm may be a Condicitonal Generative Adversarial Network (GAN).

According to various embodiments of the present disclosure, generating the learning model may include inputting random noise data to a generator conditionally on an MRI image included in the training data; Generating a composite CT image through the generator; Inputting the generated composite CT image and an original CT image corresponding to the MRI image included in the training data into a discriminator; Determining, by the discriminator, whether the synthesized CT image is an original; And generating the learning model by repeatedly performing an operation of generating a composite CT image through the generator and an operation of determining whether the composite CT image is an original through the discriminator.

According to various embodiments, the generator may be a fully convolutional network (FCN), and the discriminator may be a convolutional neural network (CNN).

According to various embodiments, the learning data may be augmented by rotating or flipping the MRI image and the original CT image included in the training data.

A non-invasive treatment planning program using a CT image generated from an artificial intelligence-based brain MRI image according to an aspect of the present invention is combined with a computer that is hardware, and a medium to execute the method of any one of claims 1 to 9 Can be stored in.

The steps of a method or algorithm described in connection with an embodiment of the present invention may be implemented directly in hardware, implemented as a software module executed by hardware, or a combination thereof. Software modules include Random Access Memory (RAM), Read Only Memory (ROM), Erasable Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), Flash Memory, hard disk, removable disk, CD-ROM, or It may reside on any type of computer-readable recording medium well known in the art to which the present invention pertains.

In the above, embodiments of the present invention have been described with reference to the accompanying drawings, but those skilled in the art to which the present invention pertains can be implemented in other specific forms without changing the technical spirit or essential features. You will be able to understand. Therefore, the embodiments described above are illustrative in all respects, and should be understood as non-limiting.

100: non-invasive treatment planning device
110: data acquisition unit
120: data processing unit
130: Deep Learning Department
140: image generator
150: treatment plan setting unit
160: database

Claims (10)

In the non-invasive treatment planning method using computed tomography (CT) images generated from artificial intelligence-based brain magnetic resonance imaging (MRI) images,
Acquiring a brain MRI image of the patient;
Generating a synthetic CT image corresponding to the brain MRI image using a learning model generated by machine learning in advance based on an artificial intelligence algorithm;
Extracting bone tissue information from the composite CT image;
Setting a non-invasive treatment plan by providing an external energy source based on the lesion information extracted from the MRI image and the bone tissue information; How to plan invasive treatment. The method of claim 1,
The setting of the non-invasive treatment plan includes determining an irradiation dose, an irradiation range, an irradiation direction, an irradiation time, and a three-dimensional irradiation coordinate of an energy source for treating neurological diseases based on the lesion information and the bone tissue information. A non-invasive treatment planning method using a CT image generated from an artificial intelligence-based brain MRI image, characterized in that. The method of claim 1,
The step of setting the non-invasive treatment plan includes inputting the lesion information and the bone tissue information into the Leksell GammaPlanㄾ-Convolution algorithm to determine the irradiation dose, irradiation range, irradiation direction, irradiation time, and three-dimensional irradiation coordinates. A non-invasive treatment planning method using a CT image generated from an artificial intelligence-based brain MRI image, comprising: a. The method of claim 1,
Calculating a Hounsfield Unit (HU) value of the original CT image of the patient;
Calculating a Hounsfield unit value of the composite CT image; And
Comparing the Hounsfield unit value of the original CT image with the Hounsfield unit value of the synthesized CT image to verify the validity of the synthesized CT image. Non-invasive treatment planning method using CT images. The method of claim 1,
And generating the learning model through the artificial intelligence algorithm by inputting learning data including MRI images and original CT images of each of a plurality of patients into the artificial intelligence algorithm. Non-invasive treatment planning method using CT images generated from MRI images. The method of claim 5,
The artificial intelligence algorithm is a non-invasive treatment planning method using a CT image generated from an artificial intelligence-based brain MRI image, characterized in that the condicitonal GAN (Generative Adversarial Network). The method of claim 6, wherein generating the learning model comprises:
Inputting random noise data into a generator conditionally on the MRI image included in the training data;
Generating a composite CT image through the generator;
Inputting the generated composite CT image and an original CT image corresponding to the MRI image included in the training data into a discriminator;
Determining, by the discriminator, whether the synthesized CT image is an original; And
And generating the learning model by repeatedly performing an operation of generating a synthetic CT image through the generator and an operation of determining whether the synthesized CT image is the original through the discriminator. Non-invasive treatment planning method using CT images generated from MRI images. The method of claim 7, wherein the generator is a fully convolutional network (FCN) and the discriminator is a convolutional neural network (CNN). The artificial intelligence-based brain MRI of claim 6, wherein the learning data is augmented by rotating or flipping the MRI image and the original CT image included in the learning data. Non-invasive treatment planning method using CT images generated from images. A non-invasive treatment planning program using a CT image generated from an artificial intelligence-based brain MRI image, which is combined with a computer as hardware and stored in a medium to execute the method of any one of claims 1 to 9.

Images