KR20220052166A - Diagnosis method for Graves’orbitopathy using orbital computed tomography (CT) based on neural network-based algorithm - Google Patents

Abstract

The present invention relates to a Graves' orbitopathy (GO) diagnosis method using orbital computed tomography (CT) based on a neural network-based algorithm and, more specifically, to a Graves' orbitopathy diagnosis method using orbital computed tomography (CT) based on a neural network-based algorithm which develops a new neural network which can be applied to Graves' orbitopathy (GO) diagnosis through CT, thereby reducing time required for GO diagnosis while maintaining diagnosis accuracy (AUC) through a CT image.

Description

Diagnosis method for Graves’orbitopathy using orbital computed tomography (CT) based on neural network-based algorithm}

The present invention relates to a method for diagnosing thyroid ophthalmopathy using orbital computed tomography (CT) based on a neural network-based algorithm. ), it relates to a method for diagnosing thyroid eye disease using orbital CT based on a neural network-based algorithm that can reduce the time required for thyroid eye disease diagnosis while maintaining diagnostic accuracy (AUC) through CT images.

Thyroid ophthalmopathy is an autoimmune disorder mainly associated with Graves' disease, and is a relatively common extrathyroid complication observed in 25% to 50% of Graves' disease patients. In general, symptoms such as eyelid retraction, protrusion, glare, pressure, and diplopia appear and improve over time. However, in 3-5% of patients with thyroid eye disease, serious complications such as blindness may occur. Therefore, the computed tomography (CT) method is being used to clarify the differential diagnosis and prognosis of patients suffering from thyroid ophthalmopathy. CT was previously used for the purpose of diagnosing thyroid ophthalmopathy and determining an appropriate treatment method, but as the indications gradually increased, it is widely used to evaluate the degree of ocular protrusion, to confirm the presence of compressive optic neuropathy, and even to evaluate the activity of thyroid ophthalmopathy. Despite the expansion of these indications, the diagnosis of thyroid ophthalmopathy based on orbital CT results has several drawbacks.

First of all, it is difficult to identify minute changes in orbital fat and extraocular muscle only by CT findings. Since there is no widely accepted study on the normal range of structures within the orbit, it is difficult to clearly distinguish changes in the orbit only with CT results. Although several software programs are being developed for CT analysis, a more intuitive and easy method is needed to analyze CT results and provide objective results for predicting the diagnosis and prognosis of thyroid eye disease patients.

Recently, various neural network (NN)-based methods for image analysis have been developed and applied to diagnose various eye diseases such as glaucoma, diabetic retinopathy, and age-related macular degeneration. However, since this pre-developed artificial neural network was developed for general image analysis, it may not be sufficient for diseases that require dozens of specialized images in multiple axial directions to detect structural changes, such as orbital disease.

In view of such problems, the present invention demonstrates the applicability of an artificial neural network (NN) to the diagnosis and severity evaluation of thyroid ophthalmopathy using orbital CT, and orbital computed tomography (CT) based on a neural network-based algorithm. To provide a method for diagnosing thyroid ophthalmopathy using

A method for diagnosing thyroid ophthalmopathy using orbital computed tomography (CT) based on a neural network-based algorithm according to embodiments of the present invention comprises: collecting CT image images of a patient suffering from thyroid ophthalmopathy; and the collected CT images Pre-processing the CT image by creating an information processing pipeline for each of the CT images; and constructing data assuming a diagnosis situation based on the pre-processed CT image; and creating a combined data set of the collected images Learning a neural network (NN); and repeating the neural network (NN) learning n times, averaging n diagnostic accuracy values, and optimal diagnosis of the neural network (NN) performance; and diagnosing thyroid ophthalmopathy using the CT image through an algorithm of a neural network (NN) that produces the optimal diagnostic performance.

According to embodiments of the present invention, the pre-processing of the CT image of the thyroid eye disease patient is based on the collected CT image image dataset of the thyroid eye disease patient, and the axial (axial) image of the thyroid eye disease CT image image. ), obtaining image information data of coronal, sagittal parts; and pre-processing by creating an image information processing pipeline based on the image information of the axial, coronal, and sagittal portions of the CT image image.

According to embodiments of the present invention, the step of pre-processing the CT image is to normalize the size by using interpolation for the initially input axial, coronal, and sagittal CT images. and extracting a portion related to the eyeball from the CT image image; and displaying an unnecessary black area in the vicinity other than the CT image image from which only the portion related to the eye is extracted; and a black area in the CT image image Deleting the area marked with ; and normalizing the size of the image through interpolation with respect to the CT image from which images around the eyeball have been deleted; and normalizing the pixel values of the image to between 0 and 1 after removing the remaining parts except for fat and muscle through the Hounsfield unit value.

According to the embodiments of the present invention, the CT image normalization step using the Hounsfield unit value is based on the fact that a person has two eyes, so that a large number of information obtainable from each eye is included and processed as one piece of information without a standard. and adding a Half Depthwise Convolution structure to improve the limit of diagnostic performance.

According to embodiments of the present invention, the step of creating the combined data set comprises: configuring data based on the pre-processed CT image, assuming four diagnostic situations; and generating a data set of 7 combinations through the axial, coronal, and sagittal CT images.

According to embodiments of the present invention, the image information processing pipeline of the axial, coronal, and sagittal parts of the CT image image is designed so that the image information processing pipeline can be placed separately. It includes; a step of flexibly adjusting the amount of calculation required for learning.

According to embodiments of the present invention, in the step of training the neural network (NN), the artificial neural network (NN: Neural Network) is used by using as much as 80% of the CT images of the respective CT image data. and measuring the diagnostic accuracy (AUC) using the remaining 20% of the CT images.

According to embodiments of the present invention, the algorithm of the artificial neural network (NN) includes the steps of diagnosing whether the thyroid ophthalmopathy is present, and generating a calculation formula for diagnosing the severity of the thyroid ophthalmopathy.

According to embodiments of the present invention, the learning of the neural network (NN) is repeated n times to derive an accuracy value based on the n diagnosis results, and an average of the derived n accuracy values and deriving the optimal diagnostic performance specification (SPEC) and performance of the artificial neural network (NN).

According to embodiments of the present invention, the diagnosis of thyroid ophthalmopathy includes diagnosing thyroid ophthalmopathy with the CT image based on an algorithm of an artificial neural network (NN) that has undergone the repeated learning. .

According to the method for diagnosing thyroid ophthalmopathy using orbital computed tomography (CT) based on a neural network-based algorithm as described above, the following effects are obtained.

First, it is possible to reduce the time required for thyroid ophthalmopathy diagnosis while maintaining the diagnostic accuracy (AUC) through CT images.

Second, by designing a separate information processing pipeline for each CT image of a patient with thyroid ophthalmopathy, the amount of computation required for learning can be flexibly adjusted.

Third, by judging the severity of thyroid eye disease patients through an automated reading process through CT images, it can be applied to treatment method selection and prognosis prediction in the treatment process.

Fourth, by adding a Half Depthwise Convolution structure to determine a patient with unilateral or asymmetric exophthalmos, the diagnostic accuracy (AUC) can be guaranteed.

1 is a step-by-step flowchart of the present invention.
2 and 3 are flowcharts of CT image preprocessing according to the present invention.
4 is a table showing performance results according to CT image input values to be input to a neural network (NN) according to the present invention.
5 is a graph showing an ROC curve for comparing the diagnostic accuracy (AUC) between the CT results derived by the present invention and professional clinical CT.

A method for diagnosing thyroid eye disease using orbital computed tomography (CT) based on a neural network-based algorithm according to embodiments of the present invention will be described in detail with reference to the accompanying drawings. Since the present invention can have various changes and can have various forms, specific embodiments are illustrated in the drawings and described in detail in the text. However, this is not intended to limit the present invention to the specific disclosed form, it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention. In describing each figure, like reference numerals have been used for like elements. In the accompanying drawings, the dimensions of the structures are enlarged than actual for clarity of the present invention, or reduced than actual in order to understand the schematic configuration. Also, terms such as first and second may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. Meanwhile, unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by those of ordinary skill in the art to which the present invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

1 is a step-by-step flowchart of the present invention.

Referring to FIG. 1 , a method for diagnosing thyroid eye disease using orbital computed tomography (CT) based on a neural network-based algorithm includes collecting CT image images of a patient suffering from thyroid eye disease; and each of the collected CT images. preprocessing the CT image by creating an information processing pipeline for learning NN: Neural Network; and repeating NN: Neural Network learning n times, averaging n diagnostic accuracy values, to obtain optimal diagnostic performance of the artificial neural network (NN: Neural Network) paying off; and diagnosing thyroid ophthalmopathy using the CT image through an algorithm of a neural network (NN) that produces the optimal diagnostic performance.

1 is a step-by-step flowchart of the present invention.

2 and 3 are flowcharts of CT image preprocessing according to the present invention.

2 and 3, the pre-processing of the CT image of the thyroid eye disease patient is based on the collected CT image image dataset of the thyroid eye disease patient, ), obtaining image information data of coronal, sagittal parts; and pre-processing by creating an image information processing pipeline based on the image information of the axial, coronal, and sagittal portions of the CT image image. Therefore, the pre-processing of the CT image includes normalizing the size by using interpolation for the initially input axial, coronal, and sagittal CT images; and the CT image image. extracting a part related to the eye in the ; and displaying an unnecessary black area in the vicinity other than the CT image image from which only the part related to the eye is extracted; and deleting the area marked as a black area in the CT image image and normalizing the size of the image through interpolation with respect to the CT image from which images around the eyeball have been removed; and normalizing the pixel values of the image to between 0 and 1 after removing the remaining parts except for fat and muscle through the Hounsfield unit value. In more detail, a total of 288 CT image sets were collected, including 99 cases of purity GO, 94 cases of medium to deep GO, and 95 cases of normal control. For CT images acquired in the axial, coronal and sagittal planes, the number of images in each plane is fixed to 32 using spline interpolation to overcome the fluctuations due to differences in CT equipment. After that, the region of interest (ROI) is manually cut out and the remaining black margin is removed. To meet the fixed-size input requirements for neural networks (NNs), CT images are scaled down to 128 × 128 for axial and sagittal planes and 64 × 128 for coronal planes. The CT image was scaled with Hounsfield unit (HU) values, and fat and UM were selected in the range of -110-10 and 0-40 HU, respectively, to remove unnecessary pixels. 17 Finally, normalize all images by scaling between 0 and 1.

Here, the CT image normalization step using the Hounsfield unit value is based on the fact that a person has two eyes, and does not cover and process a large number of information obtainable from each eye into a single piece of information without a standard, but a Half Depthwise Convolution structure. and further enhancing the limits of diagnostic performance. In addition, the step of creating the combined data set comprises the steps of constructing data based on the pre-processed CT image, assuming four diagnostic situations; and the axial, central, and sagittal CT images Creating a data set of 7 combinations through It includes a step of flexibly adjusting the amount of calculation required for learning by designing it to be placed separately.

In addition, the step of learning the neural network (NN) uses 80% of the CT images of each of the CT image data to learn the neural network (NN), and the remaining 20% measuring the diagnostic accuracy (AUC) by using the CT images; in particular, the algorithm of the artificial neural network (NN) diagnoses whether the thyroid ophthalmopathy is present, and the severity of the thyroid ophthalmopathy. Including; generating a formula for diagnosing the role. Thereafter, the learning of the neural network (NN) is repeated n times, an accuracy value is derived based on the n diagnosis results, and the average of the derived n accuracy values is obtained and the artificial neural network ( Including; deriving the optimal diagnostic performance specification (SPEC) and performance of NN: Neural Network, wherein the diagnosis of thyroid ophthalmopathy is based on an algorithm of an artificial neural network (NN: Neural Network) that has undergone the repeated learning, and diagnosing thyroid ophthalmopathy with the CT image. According to the method for diagnosing thyroid ophthalmopathy using orbital computed tomography (CT) based on neural network-based algorithm as described above, the following effects are shown. The time required for diagnosis can be reduced, but by designing a separate information processing pipeline for each CT image of a patient with thyroid eye disease, the amount of computation required for learning can be flexibly adjusted. In addition, by judging the severity of thyroid eye disease patients through an automated reading process through CT images, it can be applied to treatment method selection and prognosis prediction in the treatment process. Diagnosis accuracy (AUC) can be ensured by judging patients with exophthalmos.

4 is a table showing the performance results according to the CT image input value to be input to the NN (Neural Network) of the present invention, and FIG. 5 is the diagnostic accuracy (AUC) between the CT result derived by the present invention and the specialized clinical CT It is a graph showing the ROC curve for comparison.

Referring to Figure 4, (1) moderate to severe GO versus moderate control, (2) mild GO versus moderate control, (3) moderate to severe GO versus moderate control, and (4) moderate to severe GO vs. Normal Control. Next, each experimental group was represented as a separate data set. To mitigate the effects of gender and age-dependent selection bias, we used holdout cross-validation with randomized partitioning regardless of the clinical or demographic characteristics of the participants, and for each data set, 80% used a Neural Network (NN). As a training set to train, the remaining 20% is used as a test set to evaluate the trained Neural Network (NN). The final performance of the proposed Neural Network (NN) and the existing Neural Network (NN) is measured by averaging the results of 30 replicates.

The proposed NN (Neural Network) consists of three input layers consisting of 32-bit single-precision floating devices that take axial (128 × 128 × 32), sagittal (128 × 128 × 32), and coronal images (64 × 128 × 32). constructed and processed independently before the final fully connected layer. In the proposed Neural Network (NN), first, shapes are extracted from the convulsive and deep convulsive layers based on input CT images of up to three different planes. The max pooling layer follows each convolution operation. After the first step, the dimensions of the axial and coronal images are reduced to 32 × 32 × 16 and 16 × 32 × 16, respectively. Next, feature maps are extracted from the left and right trajectories of each image separately to compare the trajectories, and asymmetric or unilateral GOs are detected. To do this, we use a depth-wide convolution with the filter size set to half the image size. Each spasm filter extracts the actual value from a separate left or right orbital image. Specifically, 16 convolutional filters are used for each trajectory, yielding input values for subsequent 16 × 2 nodes. Third, each group of 16 × 2 nodes is flattened to 32 × 1 nodes, which are fully connected to subsequent 4 × 1 nodes. Finally, the output value of the 4 × 1 node is passed to the output layer. The output layer contains one sigmoidal node for calculating significance values if the data set consists of two classes: mild GO patients versus normal controls.

Referring to Figure 5, Figure 5 shows the ROC curve to compare the proposed NN (Neural Network) and the diagnostic accuracy (AUC) with a professional clinician. The blue color represents the suggested NN (Neural Network) diagnostic accuracy, and the red color represents the diagnostic accuracy of professional clinicians. It can be seen that the diagnostic accuracy of NN (Neural Network) proposed in all four comparative experiments is higher.

For comparison, four independent experiments were performed comparing three experimental groups without clinical information, such as the proposed Neural Network (NN) and the conventional Neural Network (NN), and the final grade was determined by a majority vote, and the receiver operating characteristics (ROC) Compare the diagnostic performance of the area under the curve with the proposed Neural Network (NN). We developed a novel convulsive neural network (NN) specialized in diagnosing thyroid ophthalmopathy (GO). Compared with the normal control group, the AUC for diagnosing moderate to severe GO patients was 0.979, which was the highest, with moderate to severe GO The case of discriminating between patients and mild GO patients was found to be 0.941. In addition, the proposed Neural Network (NN) shows much higher performance than the existing Neural Networks (NNs) such as GugLeNet, ResNet-50, and VGG-16. The neural network (NN) of the present invention includes three convolutional layers with three input channels, followed by one fully connected layer for binomial classification and a final softmax layer for multiple classification. Contains the sigmoid classification hierarchy. Considering that more than 100 CT images with different planes need to be analyzed for each patient, the configuration of multiple input channels allows all relevant functions to be used without loss. In addition, the proposed Neural Network (NN) can improve diagnostic performance by including an information processing pipeline for CT images of each plane.

In the detailed description of the present invention described above, although it has been described with reference to preferred embodiments of the present invention, those skilled in the art or those having ordinary knowledge in the technical field will And it will be understood that various modifications and variations of the present invention can be made without departing from the technical scope.

Claims (10)

Collecting a CT image image of a patient suffering from thyroid ophthalmopathy;
pre-processing the CT image by creating an information processing pipeline for each of the collected CT images;
constructing data assuming a diagnosis situation based on the pre-processed CT image;
training an artificial neural network (NN) by creating a combined data set of the collected images;
repeating the neural network (NN) learning n times, averaging n diagnostic accuracy values, to obtain optimal diagnostic performance of the neural network (NN);
Orbital computed tomography (CT) based on a neural network-based algorithm, comprising: diagnosing thyroid ophthalmopathy with the CT image through an algorithm of an artificial neural network (NN) that produces the optimal diagnostic performance A method for diagnosing thyroid ophthalmopathy using According to claim 1,
The step of pre-processing the CT image of the thyroid eye disease patient,
Based on the collected CT image image dataset of patients with thyroid ophthalmopathy,
obtaining image information data of axial, coronal, and sagittal portions of the thyroid eye disease CT image image;
Based on the image information of the axial, coronal, and sagittal parts of the CT image image, pre-processing by creating an image information processing pipeline; A method for diagnosing thyroid ophthalmopathy using orbital computed tomography (CT). 3. The method of claim 2,
The pre-processing of the CT image comprises:
normalizing the size of the initially input axial, coronal, and sagittal CT images by using interpolation;
extracting an eye-related part from the CT image;
displaying an unnecessary black area in the vicinity other than the CT image image from which only the part related to the eye is extracted;
deleting unnecessary black areas from the CT image;
normalizing the size of the image through interpolation with respect to the CT image from which images around the eyeball have been deleted;
Orbital computed tomography (CT) based on a neural network-based algorithm comprising: removing the remaining parts except fat and muscle through the Housefield Unit value, and then normalizing the pixel values of the image to between 0 and 1 A method for diagnosing thyroid ophthalmopathy. 4. The method of claim 3,
The CT image normalization step using the Hounsfield unit value,
Focusing on the fact that a person has two eyes, it does not encompass and process a large number of information obtainable from each eye into one information without a standard, but includes the step of improving the limit of diagnostic performance by adding a Half Depthwise Convolution structure A method for diagnosing thyroid ophthalmopathy using orbital computed tomography (CT) based on a neural network-based algorithm. According to claim 1,
The step of creating the combination data set is,
constructing data based on the pre-processed CT image, assuming four diagnostic situations;
Using orbital computed tomography (CT) based on a neural network-based algorithm comprising a; How to diagnose thyroid eye disease. 3. The method of claim 2,
The image information processing five lines of the axial, coronal, and sagittal parts of the CT image image,
Diagnosis of thyroid ophthalmopathy using orbital computed tomography (CT) based on a neural network-based algorithm, characterized in that it comprises; Way. According to claim 1,
The step of learning the artificial neural network (NN) is,
By using 80% of the CT images of each CT image data, the artificial neural network (NN) is learned,
Measuring diagnostic accuracy (AUC) using the remaining 20% of the CT images. A method for diagnosing thyroid ophthalmopathy using orbital computed tomography (CT) based on a neural network-based algorithm, comprising: 8. The method of claim 7,
The algorithm of the artificial neural network (NN) is,
Diagnosing the presence of the thyroid eye disease,
Generating a calculation formula for diagnosing the severity of the thyroid eye disease. A method for diagnosing thyroid eye disease using orbital computed tomography (CT) based on a neural network-based algorithm, comprising: a. 9. The method of claim 8,
By repeating the learning of the artificial neural network (NN) n times,
An accuracy value is derived based on the n diagnostic results,
Orbital computed tomography based on a neural network-based algorithm, comprising: averaging the derived n accuracy values to derive an optimal diagnostic performance specification (SPEC) and performance of the neural network (NN) A method for diagnosing thyroid ophthalmopathy using radiography (CT). According to claim 1,
The diagnosis of thyroid ophthalmopathy is
Orbital computed tomography (CT) based on a neural network-based algorithm, comprising: diagnosing thyroid ophthalmopathy with the CT image based on an algorithm of an artificial neural network (NN) that has undergone the repeated learning ) to diagnose thyroid ophthalmopathy.

Images