KR102177296B1 - System for estimating aspect score based on images of stroke patients - Google Patents

Abstract

The ASPECT score estimation system according to the present invention includes a preprocessing unit that normalizes and standardizes the features of an image dataset, an image processing unit that separates each lesion within a CT image classified into a supra ganglion level and a ganglion level, and a positive/negative for each lesion. It characterized in that it comprises a determination unit to determine whether the lesion has a stroke by independently building a neural network for learning the voice image.

Description

System for estimating ASPECT score based on images of stroke patients {SYSTEM FOR ESTIMATING ASPECT SCORE BASED ON IMAGES OF STROKE PATIENTS}

The present invention relates to a system for estimating an ASPECT score based on an image of a stroke patient, and more particularly, an ASPECT score from a brain CT image that estimates the ASPECT score from a brain CT image of a stroke patient based on image processing and neural networks. It relates to a system that does.

Stroke is a disease that causes brain damage due to blockage or rupture of blood vessels supplying the brain, resulting in physical disability, and is the most important cause of death worldwide, and causes permanent disability even if it does not lead to death. Classified as a dangerous disease.

Conventionally, stroke has been mainly recognized as a disease for the elderly, but recently, as strokes are common even in their 30s and 40s, it is recognized as a very dangerous disease that occurs not only in old age but also in young adults (Non-Patent Document 1 and See Patent Document 2).

These strokes can be divided into two types:'ischemic stroke', which occurs when blood vessels supplying blood to the brain are blocked, and'cerebral hemorrhage,' where blood vessels to the brain burst and bleeding occurs. .

The ischemic stroke accounts for about 80% of all strokes, and most of the ischemic strokes are caused by clotting blood clots, thrombosis, blocking blood vessels that supply oxygen and nutrients to the brain.

Various tests have been developed to diagnose stroke, and among them, the method using computed tomography (CT) can be performed in a relatively short time.

Accordingly, the examination method using the CT has the advantage of being able to perform a quick examination, and thus it is considered to be an examination method suitable for the characteristics of a stroke disease in which rapid response is essential.

In addition, in the case of hemorrhagic stroke, since the bleeding is immediately observed on CT after the occurrence of bleeding, the CT is useful as a tool for discriminating cerebral hemorrhage before using a thrombolytic agent that dissolves blood clots and opens up clogged blood vessels in order to treat ischemic stroke. Is used.

In addition, the CT is also important to observe the progress of cerebral hemorrhage after the use of thrombolytic agents.

On the other hand, ASPECT score (Alberta Stroke Program Early CT Score) is used as a representative index that can discriminate stroke.

The ASPECT score divides the MCA (middle cerebral artery) region into 10 predefined anatomical regions, and evaluates the presence of an early infarct marker with substantially low pollution to NCCT (see Non-Patent Document 3 below).

This ASPECT score proved to be a strong predictor for diagnosing the condition of stroke patients.

Global atlas on cardiovascular disease prevention and control. Geneva, World Health Organization, 2011. WHO Cardiovascular Diseases Fact Sheet No. 317. Updated March 2013, http/www.who.int/mediacentre/factsheets/fs317/en/ Barber PA, Demchuk AM, Zhang J, et al. 'Validity and reliability of a quantitative computed tomography score in predicting outcome of hyperacute stroke before thrombolytic therapy: ASPECTS Study Group―Alberta Stroke Program Early CT Score.' Lancet 2000; 355:1670-74 CrossRef Medline PFAFF, J., et al. 'e-ASPECTS Correlates with and Is Predictive of Outcome after Mechanical Thrombectomy.' American Journal of Neuroradiology, 2017.

However, the determination of the initial signs of ischemia and conversion to the ASPECT score have significant interrater variability, which is influenced by post-experience, among other factors (see Non-Patent Document 4).

That is, the ASPECT score evaluates the disease by calculating the degree of stroke progression of the user as a score of 0 to 10, and through this, the medical staff uses it as a major index used to determine the user's treatment method and predict the prognosis.

However, in the process of estimating the ASPECT score by a specialist, the estimate may vary depending on the experience and experience of each specialist due to the user's initial signs and image complexity.

Such scoring variability has a problem that negatively affects the decision-making process for patient clinical outcomes.

Therefore, there is a need to develop a technology that can objectify and automate ASPECT scores based on image processing and deep learning technologies.

An object of the present invention is to solve the above-described problems, and to provide an ASPECT score estimation system and method for estimating an ASPECT score, which is a factor that discriminates stroke disease from a brain CT image.

Another object of the present invention is to provide an ASPECT score estimation system and method capable of reducing score variability when estimating ASPECT scores based on image processing and deep learning techniques.

Another object of the present invention is to provide an ASPECT score estimation system and method capable of improving the reliability of the estimated ASPECT score.

In order to achieve the above object, the ASPECT score estimation system according to the present invention is a preprocessor that normalizes and standardizes the features of an image dataset, and separates each lesion within a CT image classified into a Supra ganglion level and a ganglion level. It characterized in that it comprises an image processing unit to divide and a determination unit to determine whether the lesion has a stroke by independently constructing a neural network for learning positive/negative images for each lesion.

Specifically, in the system for estimating an ASPECT score based on a brain CT image of a stroke patient according to the present invention, the system includes: a preprocessor for normalizing and standardizing features of an image dataset; An image processing unit for separating each lesion within the CT image classified into a supra ganglion level and a ganglion level; And a determination unit that independently builds a neural network for learning positive/negative images for each lesion to determine whether the lesion has a stroke; but, the preprocessor includes a Gaussian blur convoluted over the entire patient's brain CT image and included in the image. Remove the generated noise, search the skull ellipse to find the skull in the brain CT image, align the position of the image based on the center point of the searched skull and rotate the image to uniformly align the position and rotation angle of the dataset. , Horizontal Invert according to the lesion location (Lesion-Side), and the preprocessor searches for an adaptive threshold value by considering the pixel value distribution of the segmented image in the operation of searching for the skull ellipse. And, it induces to leave only pixel information corresponding to the skull, detects the edge based on the image on which the threshold has been completed, searches for the skull ellipse from the image where only the edge information remains, and acquires ellipse information of the inner and outer edges of the skull. , In accordance with the operation of the preprocessor, the CT image is normalized and normalized to an image aligned in the center so that the left brain is a lesion, and the center vertical line of the image is a symmetry line of the brain, and the The determination unit cuts out each lesion image from the entire image according to the result of separating each lesion from the image processing unit, and performs supervised learning with a neural network configured for each lesion including information on the positive/negative information on the cut image. , For each lesion, learning and classification are performed in an independent neural network, and a final ASPECT score is calculated based on the number of neural networks that determine the lesion as positive, and the determination unit is independent of the Supra ganglion level and the ganglion level. A neural network is constructed, and each of the independently constructed neural networks learns based on CNN, is composed of 6 hidden layers and 2 dropout layers to prevent overfitting, and ReLU can be used as an activation function.

As described above, according to the ASPECT score estimation system and method according to the present invention, it is possible to estimate the ASPECT score, which is an objective index for diagnosing the condition of a stroke patient, using the patient's brain CT image.

And according to the present invention, due to the nature of stroke disease that requires rapid prescription, it can be used as a reliable index that can prevent problems due to score fluctuations among experts and facilitate patient treatment decisions in the medical field. The effect is obtained.

In addition, according to the present invention, it is possible to overcome the inaccuracy of the score value according to the experience of a specialist caused by the complexity of the stroke ASPECT score estimation method through brain CT image analysis and the demand for expertise.

In addition, according to the present invention, it is possible to significantly reduce manpower, time, and economic costs required for the analysis process by automating the entire process of stroke analysis based on CT images.

1 is a block diagram of an ASPECT score estimation system according to a preferred embodiment of the present invention,
2 is a flow chart explaining step by step an ASPEC score estimation method according to a preferred embodiment of the present invention,
3 is an exemplary diagram illustrating a pre-processing step;
4 is a diagram illustrating a center point alignment process;
5 is a diagram for explaining rotational alignment;
6 is a diagram illustrating an image before and after fading processing;
7 is an exemplary diagram for explaining ganglion level division and Supra ganglion level division,
8 is a diagram illustrating a result of segmentation of a ganglion level;
9 is a diagram illustrating a neural network structure of each lesion;
10 is a performance evaluation graph.

Hereinafter, an ASPECT score estimation system and method according to an exemplary embodiment of the present invention will be described in detail with reference to the accompanying drawings.

In the following, the dataset (DataSeT) used in the algorithm for implementing the ASPECT score estimation method according to the present invention used the brain CT images of 287 stroke patients whose ASPECT scores were calculated according to the judgment of a medical professional, and each patient's brain For CT images, a total of two images are used, including one ganglionic level and one Supra ganglionic level.

For the evaluation of the ASPECT score, positive/negative judgments were made for 7 areas at the ganglion level and 3 areas at the Supra ganglion level.

The accuracy of the algorithm is judged according to whether or not the positive/negative ground truth values for each patient's 10 lesions prepared by a medical expert match the positive/negative values for each lesion calculated through the algorithm.

1 is a block diagram of an ASPECT score estimation system according to an exemplary embodiment of the present invention.

As shown in FIG. 1, the ASPECT score estimation system 10 according to a preferred embodiment of the present invention normalizes and standardizes features of an image dataset to calculate an ASPECT score based on a brain CT image of a stroke patient. Pre-Processing unit 20, Segmentation image processing unit 30 that separates each lesion within the CT image classified into Supra ganglion level and ganglion level, and Deep-Learning based As a result, a neural network for learning positive/negative images for each lesion is independently constructed, and a determination unit 40 determines whether the lesion has a stroke.

The pre-processing unit 20 is communicatively connected to a database that stores and manages CT images captured by CT equipment or CT equipment to obtain and pre-process CT images of a patient for which an ASPECT score is to be estimated.

And the ASPECT score estimation method according to a preferred embodiment of the present invention is based on the CT image of a stroke patient, a preprocessing step of normalizing and standardizing the features of the image dataset to calculate the ASPECT score, classifying it into a Supra ganglion level and a ganglion level. It includes a segmentation step of separating each lesion within the CT image and a determination step of determining whether the lesion has a stroke by independently constructing a neural network for learning positive/negative images for each lesion based on deep learning.

The patient's ASPECT score is calculated by summing up the stroke status of each lesion.

2 is a flowchart illustrating step-by-step a method for estimating an ASPEC score according to an exemplary embodiment of the present invention.

In this embodiment, along with a step-by-step description of the entire algorithm, the test software result of the deep learning-based lesion diagnosis step is examined, and the entire algorithm is verified.

(1) pretreatment step

In the pre-processing step, the pre-processing unit 20 functions to ensure more accurate results in the subsequent segmentation and deep learning steps through normalization and standardization of the dataset.

To this end, in the pre-processing step, the pre-processing unit 20 detects the position of the skull in the brain CT image based on image processing, and aligns it based on a rotation degree and a center point. An alignment step and a horizontal invert step according to a lesion position (Lesion-Side) may be performed.

The image that has gone through the pre-processing step is normalized to the image aligned in the center so that the left brain is the lesion and the center vertical line of the image is the symmetry line of the brain.

In this pre-processing step, in the deep learning-based decision step, the main features that can discriminate whether or not a stroke are identified are optimized into an image that is good for specifying.

In FIG. 2, the pre-processing step includes a noise removal step (S10) for removing noise in an image, a search step (S12) for searching for a skull in a patient's brain CT image, and an image based on the center point of the searched skull. And an alignment step (S14) of aligning the position of and rotating the image to uniformly align the position and rotation angle of the dataset.

3 is an exemplary diagram illustrating a pre-processing step.

3(a) illustrates an image convoluted with Gaussian blur, FIG. 3(b) illustrates a threshold image, and FIG. 3(c) illustrates an outline search image, 3(d) illustrates the inner and outer ellipses of the skull.

Noise included in the CT image is a factor that degrades the accuracy of each step included in the preprocessing step for acquiring the main features of the brain, so it needs to be removed.

Accordingly, the noise removal step S10 removes noise included in the image by convolving a Gaussian Blur on the entire image, as shown in FIG. 2A.

The search step S12 performs automatic thresholding, contour finding, and skull ellipse search functions.

That is, in the search step S12, each function may be separated to find the inner and outer ellipses of the skull from the image from which noise has been removed.

The automatic thresholding function searches for an adaptive threshold value in consideration of the pixel value distribution of the divided image as shown in FIG. 3(b), and pixel information corresponding to the skull Induce them to leave only.

The contour search function detects an edge based on the image on which the threshold has been completed, as shown in FIG. 3(c), and the skull ellipse search function is shown in FIG. 3(d). As described above, by searching for a skull ellipse from an image where only edge information remains, ellipse information on the inner and outer edges of the skull is obtained.

The alignment step (S14) is a pre-processing step for maintaining the consistency of the dataset, and the consistency of the dataset is essential to increase the learning and classification accuracy in deep learning-based learning and judgment to determine a stroke afterwards.

4 is a diagram illustrating a center point alignment process, and FIG. 5 is a diagram illustrating rotational alignment.

That is, since the position of the skull of the patient is different when a specialist takes a CT image of the brain of a stroke patient, the alignment step includes Center-Point Alignment and Rotation Alignment of the skull shown in FIGS. 4 and 5. ).

Accordingly, the present invention can obtain the consistency of the position and rotation of the dataset.

Here, the position and rotational alignment of the dataset may be calculated based on the skull ellipse information obtained in the preprocessing step.

Meanwhile, the pre-processing step further includes a fading step (S16) of removing a skull region, which is an unnecessary feature in determining whether or not a stroke, so as to increase the learning and classification accuracy of the deep learning-based learning and determination step. I can.

6 is a diagram illustrating images before and after fading processing.

6A and 6B illustrate an image before fading and an image after fading.

That is, the fading step sets a region of interest (ROI) for an area of a certain width from the skull ellipse based on the skull ellipse information acquired in the preprocessing step as shown in FIG. 6(a), The threshold value corresponding to the skull in the CT image is adaptively calculated, and a fading operation is performed based on the calculated threshold value.

(2) Division step

The segmentation step is a step for separating a total of 10 regions for estimating the ASPECT score from the brain CT image, and extracts 7 regions from the ganglion level and 3 regions from the Supra ganglion level (S18).

7 is an exemplary diagram illustrating ganglion level division and Supra ganglion level division.

In the ganglion level and Supra ganglion level images, feature points that must appear at the onset of a stroke exist, and each lesion is segmented based on image processing technology using the skull ellipse information acquired in the preprocessing step and the feature points. .

In FIG. 7, the standard using the feature points is shown, and in FIGS. 7A and 7B, the feature points used at the ganglion level and the supra ganglion level are shown as red dots.

The image processing unit 30 of the ASPECT score estimation system 10 according to the present invention searches for a feature point based on a preset segmentation criterion, and then geometrically connects the feature points to segment the lesion.

8 is a diagram illustrating a result of segmentation of a ganglion level.

8A and 8B illustrate images before and after division.

In FIG. 8, it can be seen that the 7 lesions extracted at the ganglion level were divided.

Since each lesion has a shape that is not clearly distinguished by a pixel value, it is optimized to approximate the segmentation criteria established according to the opinion of a specialist.

(3) Diagnosis of lesions based on deep learning

In step S22, the determination unit 40 cuts each lesion image from the entire image according to the segmentation result (image crop), and generates a neural network configured for each lesion, including positive/negative information, for the cropped image. Conduct supervised learning.

Each neural network learns based on CNN (Convolutional Neural Network), consists of 6 hidden layers and 2 dropout layers to prevent overfitting, and ReLU (Rectified Linear Network) Unit) as an activation function.

Therefore, the determination unit 40 performs learning and classification in an independent neural network for each lesion, and the number of neural networks that determine the lesion as positive means the number of lesions with stroke disease, based on the following equation The final ASPECT score is calculated according to 1 (S24).

[Equation 1]

(4) Verification

In this embodiment, the ASPECT score estimated through the above-described process is a test result for M4 lesions at the level of the Supra ganglion, and a prototype for predicting the accuracy of the neural network for each lesion was developed and verified.

9 is a diagram illustrating a neural network structure of each lesion.

As shown in FIG. 9, in the verification process, a data set for 187 patients for M4 lesions was used, 80% was used as a learning dataset, and 20% of images were used as a test dataset.

The prototype test accuracy for the M4 area was about 70.9%, and the lack of data set and lack of segmentation accuracy were judged to be the main causes.

Through the verification results of the above-described prototype, it can be seen that the segmentation accuracy for each lesion greatly affects the ASPECT score estimation accuracy.

Accordingly, the system was changed so as not to take into account the effect of the division and additional tests were conducted.

The input data and the neural network structure were changed to estimate the ASPECT score for the whole image without dividing the lesion for the ganglion level and the supra ganglion level among the brain CT images.

That is, the ASPECT score estimation system has an independent neural network for the ganglion level and the supra ganglion level, and learns the ASPECT score by receiving a not-segmented brain CT image of the corresponding level that has undergone a preprocessing step.

CT images of 278 stroke patients were used as the training dataset, and ResNet101 was used for neural network training.

The Bland-Altman plot was used as the performance evaluation algorithm, and the difference between the ground-truth calculated by the specialist and the result obtained from the neural network was used to evaluate how close the ASPECT score to the specialist was calculated. I did.

[Equation 2]

에 대하여 수프라 신경절 레벨과 신경절 레벨에 대해 독립적으로 실측 자료 값(

)과 결과값(

)의 오차를 계산하고, 각 레벨의 오차를 더한 값을 최종 에러값(

)으로 결정하였다.That is, the patient according to Equation 2

For Supra ganglion level and ganglion level independently measured data values (

) And result value (

Calculate the error of ), and add the error of each level to the final error value (

).

Table 1 is a performance evaluation result table, and a performance evaluation graph obtained by the Bland-Altman plot of FIG. 10.

Average Standard Deviation 95% confidence interval 0.1116 2.5080 -0.5453 ~ 0.7684

In Tables 1 and 10, the error mean shows a result that is close to '0', but the standard deviation is '2.5080' and shows a high scatter, so the 0.95% confidence interval (CI) result of'-0.5453 to 0.7684' Showed.

As described above, the present invention can estimate the ASPECT score, which is an objective index for diagnosing the condition of a stroke patient, using only CT images.

In addition, the present invention is a program that automatically calculates ASPECT scores for stroke diseases, which prevents problems due to score variability among experts due to the characteristics of stroke diseases that require rapid prescription, and facilitates treatment decisions of patients in the medical field. It can be used as a reliable indicator.

According to the verification results, the system described in the present embodiment showed somewhat low accuracy due to reasons such as insufficient volume of the dataset of stroke CT images, reliability problems of measured data, and ambiguity of features in the image.

However, since neural networks have a characteristic that the results are greatly affected by the quality of the dataset, a highly reliable measurement result is applied through the increase of the dataset and the scoring of multiple specialists, and the opinions of the specialists are obtained in the preprocessing stage for the image. If the neural network is modified to receive only features specialized for stroke detection through emphasis and selection of reflected features, high accuracy results can be obtained.

Although the invention made by the present inventor has been described in detail according to the above embodiment, the invention is not limited to the above embodiment, and it goes without saying that the invention can be changed in various ways without departing from the gist.

10: ASPECT score estimation system
20: pretreatment unit
30: image processing unit
40: judgment unit

Claims (1)

In the system for estimating the ASPECT score based on the brain CT image of a stroke patient,
A preprocessor for normalizing and standardizing features of an image dataset;
An image processing unit for separating each lesion within the CT image classified into a supra ganglion level and a ganglion level; And
Including; a judging unit for determining whether the lesion has a stroke by independently constructing a neural network for learning positive/negative images for each lesion,

The pretreatment unit,
Gaussian blur is convolved over the patient's brain CT image to remove noise included in the image,
Search the skull ellipse to find the skull in the brain CT image,
Align the position of the image based on the center point of the searched skull and rotate the image to uniformly align the position and rotation angle of the dataset,
Horizontal Invert according to the lesion location (Lesion-Side) is performed,

The preprocessor, in the operation of searching for the skull oval,
Considering the pixel value distribution of the segmented image, an adaptive threshold value is searched, and only pixel information corresponding to the skull is left,
Edge is detected based on the image on which the threshold is completed,
From the image remaining only edge information, the skull ellipse is searched to obtain the inner and outer ellipse information of the skull,

According to the operation of the preprocessor, the CT image is normalized and normalized to an image aligned in the center so that the left brain is a lesion and the center vertical line of the image is a symmetry line of the brain,

The determination unit,
According to the result of separating each lesion of the image processing unit, each lesion image is cut out from the entire image,
Supervised learning is performed with a neural network configured for each lesion, including information about positive/negative, the cut-out image,
For each lesion, learning and classification are performed in an independent neural network, and a final ASPECT score is calculated based on the number of neural networks that judged the lesion as positive,

The determination unit,
Constructing an independent neural network for the Supra ganglion level and the ganglion level,

Each of the independently constructed neural networks,
Learning based on CNN,
It consists of 6 hidden layers and 2 dropout layers to prevent overfitting,
ReLU is used as an activation function,

The pretreatment unit,
In order to improve the learning and classification accuracy of deep learning-based learning and judgment, a fading process is performed to remove the skull area unnecessary for determining whether a stroke has occurred, and

In the fading process, based on the acquired skull ellipse information, an ROI is set for an area of a certain width from the skull ellipse, and a threshold value corresponding to the skull in the CT image is adaptively calculated, and the calculated threshold It is performed by performing a fading operation based on the value,

The image processing unit,
In the brain CT image, 7 regions were extracted from the ganglion level, 3 regions were extracted from the Supra ganglion level region, and a total of 10 regions were extracted,
Segmentation for each lesion is performed based on image processing technology using the skull ellipse information acquired by the preprocessor and the feature points included in the extracted 10 regions,
An ASPECT score estimation system, characterized in that after searching for a feature point based on a preset segmentation criterion, geometrically connecting each feature point to segment a lesion.

Images