WO2020116878A1 - Intracranial aneurysm prediction device using fundus photo, and method for providing intracranial aneurysm prediction result - Google Patents

Abstract

Embodiments relate to an intracranial aneurysm prediction device and a method by which the intracranial aneurysm prediction device provides an intracranial aneurysm prediction result, the device comprising: a storage unit for storing an intracranial aneurysm prediction model learned in advance; a data acquisition unit for acquiring a fundus photo of a subject; and a prediction unit applying fundus photo data of the subject to the intracranial aneurysm prediction model so as to determine whether the subject has an intracranial aneurysm.

Description

Apparatus for predicting cerebral aneurysm using fundus photography and method for providing results of predicting cerebral aneurysm

The present invention relates to a technique for predicting intracranial aneurysm disease using fundus photography, and more specifically, by applying a patient's fundus photography to a predicted model of cerebral aneurysm based on sample data associated with fundus photography The present invention relates to a device for predicting cerebral aneurysm and a method for providing results of predicting cerebral aneurysm to determine whether the patient has a cerebral aneurysm.

[National research and development project supporting this invention]

This study was conducted under the supervision of Seoul National University Bundang Hospital, and the research title is "Securing and applying the original technology for reading fundus photo using Deep Learning technology" (Research Period: 2018.11.01 .~2021.10.31.Project unique number: 2018R1D1A1A09083241).

As the life span of humans increases due to the development of technology, the population structure is trending toward an aging society and even an aging society. Due to this, the number of patients with degenerative disease is increasing, and the socio-economic burden is also rapidly increasing. These degenerative diseases include vascular diseases, joint diseases, and the like, and vascular diseases belong to diseases with a high risk of death due to the physical structure characteristics of blood vessels, difficulty in surgery, and high probability of sequelae. Indeed, according to data from the Korea Statistical Office in 2016, heart disease and cerebrovascular disease are the second and third leading causes of death.

Cerebral aneurysm is one of the cerebrovascular diseases, which is a disease in which a part of the brain artery in the head is formed due to a defect. When a patient has a cerebral aneurysm, when a part of the patient's cerebral blood vessels are weak, the blood vessel walls are stretched and flared out. Because the blood vessel wall of the cerebral artery is thin and structurally different from the normal like a tummy shape, the cerebral blood vessels of a cerebral aneurysm disease can easily burst.

The most common disease of cerebral aneurysms is subarachnoid hemorrhage, which has the highest mortality and disability rates. Most of the subarachnoid hemorrhages have a symptom of swelling of a part of the cerebral blood vessels, which eventually leads to the occurrence of intracranial aneurysm.

In general, an unruptured cerebral aneurysm usually does not cause symptoms before rupture, and is a disease unpredictable on a simple CT.

Screening tests currently commonly performed include, for example, invasive tests such as percutaneous intracranial angiography (TFCA), or high radiation such as brain computed tomography angiography (CT Angio), magnetic resonance angiography (MRA). This may include inspection or expensive imaging.

This current screening test is an invasive test and/or a test using a contrast agent, and there is a possibility of adversely affecting the health of the test subject due to the risk of the invasion and contrast agent.

In addition, screening tests must be performed on a large number of subjects, and the cost consumed for this is also significant.

Brain aneurysms, such as subarachnoid hemorrhage, can be preventively treated if predicted before rupture. However, since cerebral aneurysms rarely develop symptoms, it is very difficult to detect unruptured cerebral aneurysms in an unexploded state. Because of this, it is also difficult to determine who will perform the above-mentioned invasive or expensive imaging tests.

Therefore, there is a need for a non-invasive and easy examination technique that can screen people who are likely to have cerebral aneurysms.

[Advanced technical literature]

[Non-patent literature]

1. Korean Stroke Society Guidelines for Stroke Treatment Non-ruptured cerebral aneurysm screening (2013.01.)

According to an aspect of the present invention, in detail, a cerebral aneurysm capable of determining with high accuracy whether a patient has a cerebral aneurysm by applying a patient's fundus photograph to a predicted cerebral aneurysm model based on sample data associated with the fundus photograph. It is possible to provide a prediction device and a method for providing a predicted result of a cerebral aneurysm.

A method for providing a result of predicting cerebral aneurysm performed by a computing device including a processor according to an aspect of the present invention or a method for predicting a brain aneurysm (determination) includes: obtaining a fundus photograph of a subject; And applying the fundus photo data of the subject to a cerebral aneurysm prediction model to determine whether the subject has a cerebral aneurysm. Here, the cerebral aneurysm prediction model is a model in which the ability to classify an input fundus photograph into a cerebral aneurysm group using a plurality of training samples including a learning fundus photograph is learned.

In one embodiment, the cerebral aneurysm prediction model comprises: an input layer into which a fundus photograph is input; A feature extraction layer including a plurality of convolutional layers and a pooling layer and extracting features from the fundus photo of the input layer; And a complete connection layer including a plurality of nodes, and may include a classification layer that classifies the fundus picture input based on the feature into a cerebral aneurysm group or a non-disease group.

In one embodiment, at least one of the plurality of convolutional layers may include a residual bottleneck block configured such that input data proceeds through a plurality of paths and then recombines.

In one embodiment, the residual bottleneck block includes a plurality of subconvolutional layers, wherein each subconvolutional layer is configured such that a convolution filter is applied, and the residual bottleneck block includes input data having a first path and a second path. It may be configured to proceed through, and the data proceeding through the first path and the data proceeding through the second path are combined.

In one embodiment, input data traveling through the first path may pass through a plurality of sub-convolutional layers. Here, at least one of the plurality of sub-convolution layers through which the input data through the first path passes may output data having a lower dimension than other convolution filters.

In one embodiment, the residual bottleneck block is output from a first subconvolutional layer and a first subconvolutional layer that output a first featuremap by applying a plurality of 1×1 filters to the input featuremap. A second sub-convolution layer that outputs a second feature map by applying a plurality of 3×3 sized filters to the first feature map, and a plurality of 1×1 to a second feature map output from the second sub-convolution layer A third sub-convolution layer that outputs a third feature map by applying a size filter may be included.

In one embodiment, when at least one of a plurality of sub-convolution layers through which the input data through the first path passes is configured to output sampling data, the input data proceeding through the second path is the Sampling may be performed to correspond to the sampling data.

In one embodiment, data combining in the residual bottleneck block may be performed by element-wise addition processing.

In one embodiment, the feature extraction layer may include a first convolution layer. Here, the first convolution layer is configured to output a first feature map using 64 7×7 filters on the fundus image output from the input layer.

In one embodiment, the feature extraction layer may further include a max pooling layer configured to perform max pooling of the first feature map.

In one embodiment, the feature extraction layer may further include second to fourth convolution layers. Here, at least one of the second convolution layer to the fourth convolution layer includes the residual bottleneck block.

In one embodiment, at least one of the second convolution layer to the fourth convolution layer may include a plurality of residual bottleneck blocks.

In one embodiment, when there are multiple residual bottleneck blocks, at least one residual bottleneck block is configured to sample and output input data.

In one embodiment, the cerebral aneurysm prediction model may be learned by updating the parameters included in the cerebral aneurysm prediction model in a direction that minimizes the cost function of the cerebral aneurysm prediction model. Here, the loss function represents the difference between the actual value and the result obtained by applying the fundus photograph to the cerebral aneurysm prediction model.

In one embodiment, the cerebral aneurysm prediction model may be trained using a learning sample including a monocular fundus photograph.

In one embodiment, the step of acquiring a fundus photograph of the subject further includes filtering the binocular fundus photograph so that the monocular fundus photograph is applied to the model for predicting cerebral aneurysm when the binocular fundus photograph of the subject is obtained. can do.

In one embodiment, the cerebral aneurysm prediction model may be trained using a learning sample including a binocular fundus photograph.

In one embodiment, the cerebral aneurysm prediction model includes: a first input layer into which a first fundus photograph is input; A first feature extraction layer for extracting features of the first fundus picture; A second input layer into which a second fundus photograph is input; A second feature extraction layer for extracting features of the second fundus picture; A combining layer configured to combine the feature values of the first feature extraction layer and the feature values of the second feature extraction layer; And a classification layer for classifying a ocular fundus photograph into a cerebral aneurysm group or a non-cerebral aneurysm group based on the combined feature values.

In one embodiment, the combining layer may be configured to handle vector concatenation.

The computer-readable recording medium according to another aspect of the present invention may store program instructions readable by a computing device and operable by the computing device. Here, when the program instruction is executed by a processor of the computing device, the processor may perform a method for predicting cerebral aneurysm according to the above-described embodiments.

A brain aneurysm prediction apparatus for predicting a cerebral aneurysm using a fundus photograph according to another aspect of the present invention includes a storage unit for storing a pre-trained brain aneurysm prediction model; A data acquisition unit that acquires a fundus photograph of the subject; And a predictor configured to determine whether the subject has a cerebral aneurysm by applying the fundus photo data of the subject to the cerebral aneurysm prediction model.

In one embodiment, the cerebral aneurysm prediction model comprises: an input layer into which a monocular fundus photograph is input; A feature extraction layer that extracts features from the fundus picture of the input layer; And a classification layer for classifying the fundus picture input based on the feature into a cerebral aneurysm group or a non-disease group.

In one embodiment, the cerebral aneurysm prediction model includes: a first input layer into which a first fundus photograph is input; A first feature extraction layer for extracting features of the first fundus picture; A second input layer into which a second fundus photograph is input; A second feature extraction layer for extracting features of the second fundus picture; A combining layer configured to combine the feature values of the first feature extraction layer and the feature values of the second feature extraction layer; And a classification layer for classifying a ocular fundus photograph into a cerebral aneurysm group or a non-disease group based on the combined feature values.

The apparatus for predicting cerebral aneurysm according to an aspect of the present invention is a non-invasive method and can predict whether or not a cerebral aneurysm is performed with high accuracy through fundus photography with little risk to a subject.

A fundus camera capable of taking fundus photographs is widely used in most ophthalmology clinics, and thus has high accessibility, and does not need to have a separate medical device for measuring a patient's body condition for predicting cerebral aneurysms. In addition, fundus photography is relatively inexpensive compared to conventional cerebral aneurysm examination. That is, the accessibility and stability of the inspection are very high, and economically low.

In addition, the apparatus for predicting cerebral aneurysm according to an aspect of the present invention is capable of predicting an unruptured cerebral aneurysm that is difficult to test in the related art.

As a result, the present invention can be utilized as a non-invasive and easy examination technique that can screen people who are likely to have cerebral aneurysms. For example, it can be used to filter the high-risk group of cerebral aneurysms prior to conducting current invasive or imaging tests. Therefore, it is possible to reduce the social and economic costs consumed by a person who does not need a brain aneurysm test.

Furthermore, the apparatus for predicting cerebral aneurysm according to the present invention may be used as a device for diagnosing a person with cerebral aneurysm disease.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

BRIEF DESCRIPTION OF DRAWINGS To describe the technical solutions in the embodiments of the present invention or in the prior art more clearly, the drawings required in the description of the embodiments are briefly introduced below. It should be understood that the drawings below are for the purpose of describing the embodiments of the present specification and not for the purpose of limitation. In addition, some elements to which various modifications such as exaggeration and omission are applied may be illustrated in the drawings below for clarity.

1 is a schematic block diagram of an apparatus for predicting cerebral aneurysm, according to an embodiment of the present invention.

2 is a diagram for explaining residual learning.

3 is a view for explaining the structure of a brain aneurysm prediction model according to an embodiment of the present invention.

FIG. 4A is a diagram for explaining a detailed structure of a model for predicting cerebral aneurysm of FIG. 3 according to an embodiment of the present invention.

4B is a view for explaining some convolutional layers of FIG. 4A according to an embodiment of the present invention.

5 is a flowchart illustrating a process of determining a cerebral aneurysm using the model of FIG. 3 according to an embodiment of the present invention.

6 is a diagram for explaining the structure of a brain aneurysm prediction model according to a second embodiment of the present invention.

7 is a view for explaining the performance of the brain aneurysm prediction model according to an experimental example of the present invention.

8 is a diagram illustrating prediction results of a fundus photograph according to an experimental example of the present invention.

The terminology used herein is only to refer to a specific embodiment and is not intended to limit the invention. The singular forms used herein also include plural forms unless the phrases clearly indicate the opposite. As used herein, the meaning of “comprising” embodies a particular property, region, integer, step, action, element, and/or component, and the presence or presence of other properties, regions, integers, steps, action, element, and/or component. It does not exclude addition.

Although not defined differently, all terms including technical terms and scientific terms used herein have the same meaning as those generally understood by those skilled in the art to which the present invention pertains. Commonly used dictionary-defined terms are further interpreted as having meanings consistent with related technical documents and currently disclosed contents, and are not interpreted as ideal or very formal meanings unless defined.

In the present specification, cerebral aneurysm refers to a disease caused by an abnormality in cerebral arteries, such as subarachnoid hemorrhage, which occurs due to swelling of a part of the cerebral blood vessels.

In this specification, a fundus image may be referred to as a retinal fundus image.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

1 is a schematic block diagram of an apparatus for predicting cerebral aneurysm, according to an embodiment of the present invention.

Referring to FIG. 1, the apparatus for predicting cerebral aneurysm 1000 includes a data acquisition unit 10 that acquires a fundus photograph of a subject; And a prediction unit 50 that determines whether the subject has a cerebral aneurysm by applying the fundus photo data of the subject to the brain aneurysm prediction model. In some embodiments, the cerebral aneurysm prediction apparatus 1000 may further include a learning unit 30 for modeling a brain aneurysm prediction model.

The optic nerve papilla and the blood vessels of the retina are genetically differentiated parts of the brain and have characteristics similar to those of the brain, and there is a possibility that the retinal blood vessel abnormality and the cerebral blood vessel abnormality are related. That is, an abnormality of the cerebral blood vessel may be realized as an abnormality of the retinal blood vessel. Based on this, the apparatus for predicting cerebral aneurysm 1000 may determine a subject with cerebral aneurysm using a fundus photograph of the subject.

The cerebral aneurysm prediction apparatus 1000 according to the embodiments may have aspects that are entirely hardware, entirely software, or partially hardware and partially software. For example, the cerebral aneurysm prediction apparatus 1000 may collectively refer to hardware equipped with data processing capability and operating software for driving the hardware. The terms "unit", "module", "device", or "system" in this specification are intended to refer to a combination of hardware and software driven by the hardware. For example, the hardware may be a data processing device including a central processing unit (CPU), a graphics processing unit (GPU), or other processor. Also, the software may refer to a running process, an object, an executable, a thread of execution, a program, or the like.

The data acquisition unit 10 receives a fundus photograph of a subject photographed by the fundus camera. The fundus camera includes various fundus cameras capable of obtaining fundus photos. For example, the fundus camera may include a Shandong fundus camera, a Musandong fundus camera, an OCT type fundus camera, and the like.

In general, fundus photographs are obtained in the form of binocular fundus photographs. However, the present invention is not limited to this, and the data acquisition unit 10 may acquire a monocular fundus photograph. In this case, the labeling data indicating whether the obtained monocular fundus photograph is the fundus photograph of the left eye or the fundus photograph of the right eye may be further obtained.

Also, the data acquisition unit 10 may acquire data related to the fundus photograph. For example, the data related to the fundus photograph may include subject identification information (eg, name, identity information, identifier, etc.) that can identify the subject of the fundus photograph.

In addition, the data acquisition unit 10 provides the fundus photograph and data related to the fundus photograph of the subject to the prediction unit 50. The fundus photograph provided to the prediction unit 50 depends on the structure of the brain aneurysm prediction model used by the prediction unit 50.

In one embodiment, when the brain aneurysm prediction model is trained using a binocular fundus photograph, the data acquisition unit 10 provides the subject's binocular fundus photograph to the predictor 50.

In another embodiment, when the brain aneurysm prediction model is trained using a monocular fundus photograph, the data acquisition unit 10 may be further configured to provide a corresponding monocular photograph. For example, when the brain aneurysm prediction model is trained using a left-eye photograph, the data acquisition unit 10 that acquires the binocular photograph may filter the binocular photograph and provide only the left-eye photograph to the predictor 50.

The predicting unit 50 determines whether the subject has a cerebral aneurysm by applying a fundus photograph of the subject to a brain aneurysm prediction model. In one embodiment, the prediction unit 50 may determine whether the subject is a patient by using a situation determination model learned by the learning unit 30.

In another embodiment, the prediction unit 50 may determine whether the subject is a patient by using a situation determination model previously learned in the learning unit 30. In this case, the previously learned situation determination model may be stored in a storage device (not shown) of the device 1000.

The learning unit 30 learns a brain aneurysm prediction model for predicting whether or not a cerebral aneurysm exists using a plurality of samples for learning. Each learning sample may include a binocular fundus photograph or a monocular fundus photograph, disease data indicating whether or not a cerebral aneurysm exists. The disease data may be expressed as binary labels representing true and false cerebral aneurysms.

In one embodiment, the learning unit 30 may be a component included in the brain aneurysm prediction apparatus 1000 as shown in FIG. 1. However, the present invention is not limited to this. In another embodiment, the learning unit 30 may be a component remotely located in the brain aneurysm prediction apparatus 1000. In this case, the apparatus 1000 may receive and store the brain aneurysm prediction model previously generated by the remotely located learning unit 30 prior to the brain aneurysm test on the subject, and use the brain aneurysm test on the subject. In this case, the cerebral aneurysm prediction apparatus 1000 may further include a storage device (not shown) for storing a pre-generated brain aneurysm prediction model. For example, the storage device may include a read-only memory (ROM), a flash memory, and a hard disk drive (HDD). Alternatively, the cerebral aneurysm prediction model may be stored in a cloud server, and the device 1000 may be configured to use the cerebral aneurysm prediction model by communicating with the cloud server.

The cerebral aneurysm prediction model is composed of a convolutional neural network (CNN) structure including at least one convolution layer, a pooling layer, and a full-connected layer responsible for final determination. Can be.

In one embodiment, at least a portion of the cerebral aneurysm prediction model may be configured to include a residual bottleneck block.

2 is a diagram for explaining residual learning.

A machine learning model with a deeper network can well extract the representative concepts present in the training data, so the deeper the network included in the machine learning model, the better the learning result.

In the case of a conventional CNN learning model composed of only weight layers, if the input is x, the stacked ouput F(x) through one or more weight layers is data to be finally learned.

However, if the network of the CNN structure is simply deep, the effect of learning disappears or the learning speed becomes very slow (Vanishing Gradient Problem), and the number of parameters increases proportionally by having a deep network structure, which is optimized only for the learned data. There is a problem in that inferencing performance for data that is not performed (ie, test data) is deteriorated (Overfitting problem).

The residual bottleneck block has a structure in which a type of skip connection (or referred to as a shortcut connection) is added to an existing stacked structure in a CNN-based model having a deep structure. .

In the residual bottleneck block, if the input is x, the data to be finally learned is H(x). The H(x) is F(x) + x, and can be expressed as F(x) = H(x)-x. Accordingly, the residual bottleneck block can learn the residual between the output H(x) and the input stacked through the weighting layer, and as a result, the original learning intended result F(x) can be obtained. As described above, learning of a CNN model having a residual learning block structure that only needs to learn residuals may be referred to as residual learning.

If a CNN model including such a residual bottleneck block is used, it can be easily optimized even in a deep network structure. Therefore, the model for predicting cerebral aneurysm based on the residual bottleneck block structure can be optimized with a deep network structure and has high accuracy. The structure of the brain aneurysm prediction model based on the residual bottleneck block structure will be described in more detail with reference to FIGS. 3 and 7 below.

Embodiment 1

3 is a diagram for explaining the structure of a model for predicting a brain aneurysm to which a monocular fundus photograph is applied according to a first embodiment of the present invention.

Referring to FIG. 3, the brain aneurysm prediction model includes an input layer 310, a feature extraction layer 330, and a classification layer 350.

In one embodiment, the input layer 310 receives a training sample for learning and outputs a fundus image. The fundus image is transferred to the feature extraction layer 330. For example, the input layer 310 may be configured to output as a retinal fundus image composed of 512×512 RGB (ie, 512×512×3) channels.

In some embodiments, the input layer 310 may be further configured to perform augmentation processing. For example, to derive stronger learning results for fundus photos, random filp, random rotation (e.g., between -30° and 30°), random brightness adjustment ), random contrast adjustment, etc. may be performed.

The feature extraction layer 330 extracts features that can be used to predict cerebral aneurysms from the output image output from the input layer 310. The above features are extracted through a convolution filter.

The feature extraction layer 330 includes a plurality of convolutional layers and a pooling layer. At least one of the plurality of convolutional layers may include a residual bottleneck block B configured such that input data is processed through a stack path, which is a path through which the input data passes through the stack structure, and a shortcut path that does not pass through the stack structure. have.

In one embodiment, the residual bottleneck block (B) is input data going through the stack path to extract features by passing at least one sub-convolution layer modeled so that a convolution filter is applied (that is, output a feature map) ) Is composed. The progress of the data refers to a series of data processing processes performed by proceeding along a path. The output data processed through the stack path is combined with data processed through the shortcut path.

In some embodiments, the residual bottleneck block may be further configured to apply a sampling filter to the data input to the shortcut path when the size of the data input to the stack path is reduced in the feature extraction process. Due to the application of the sampling filter, data corresponding to data output through the stack path may be output through the shortcut path. For example, when the feature map output through the stack path is 1/2 size compared to the input feature map, a sampling filter may be applied to the shortcut path to output a feature map of the same size as the output feature map.

The sampling filter includes a filter configured to perform only the stride operation or a convolution filter configured to extract features simultaneously with the stride operation. In the above example, when the feature map output through the stack path is sampled in a size of 1/2, the sampling filter may be a 1×1 sized sampling filter that moves at 2 pixel intervals.

FIG. 4A is a diagram for explaining a detailed structure of a model for predicting cerebral aneurysm of FIG. 3 according to an embodiment of the present invention.

Referring to FIG. 4A, the feature extraction layer 330 of the cerebral aneurysm prediction model of FIG. 3 includes 5 convolutional layers 331, 333, 334, 335, 336, 1 max pooling layer 332, and 1 average A pulling layer 337 may be included.

The convolution layer 331 applies a convolution filter to the data input through the input layer 310 (eg, a fundus image of 512×512×3) and outputs a feature map. In one example, the convolutional layer 331 outputs a feature map of the convolutional layer 331 by moving 64 7×7 sized filters at 2 pixel intervals and applying them.

The max pooling layer 332 is configured to perform max pooling on the feature map output from the convolution layer 331. In one example, the max pooling layer 332 outputs a feature map sampled in size 1/2 by moving and applying a 3×3 filter at 2 pixel intervals.

The convolution layer 333 receives the sampled feature map output from the max pooling layer 332, and applies a convolution filter to it to output the feature map.

In one embodiment, the convolution layer 333 includes a residual bottleneck block (B). Referring to FIG. 4B, the residual bottleneck block B of the convolution layer 333 includes a stack path for outputting a stacking output as input data sequentially passes through the plurality of sub-layers, and input data is the It is configured to proceed with a shortcut path that combines with the output data of the stack path. The residual bottleneck block is a plurality of sub-layers located on a stack path, and includes a plurality of sub-layers configured to apply a convolution filter to input data. Within the plurality of sub-layers, at least one sub-layer is configured to apply a convolution filter that outputs data of a lower dimension than the convolution filter included in the other sub-layer.

4B is a diagram for explaining the structure of a residual bottleneck block included in a convolution layer, according to an embodiment of the present invention.

Specifically, as illustrated in FIG. 4B, the residual bottleneck block B of the convolution layer 333 is applied to the input feature map by applying a plurality of (eg, 64) 1×1 size convolution filters. A second feature map by applying a plurality of (e.g., 64) 3×3 size filters to the first feature map output from the sub-convolution layer 3331 and the sub-convolution layer 3331 that output the 1 feature map A third feature map by applying a plurality of (for example, 128) 1×1 convolution filters to the sub-convolution layer 3333 outputting and the second feature map output from the sub-convolution layer 3333 It includes a sub-convolution layer (3335) for outputting. The subconvolution layers 3331, 3333, and 3335 are sequentially applied to the input data of the stack path.

The convolution layer 333 having the structure of FIG. 4B reduces the dimension of data by using a subconvolution layer 3331 having a 1×1 filter, and then a subconvolution layer 3333 having a 3×3 filter Use to increase the data dimension. As such, after the dimension of the reduced data is increased based on the subconvolution layer 3333 having a 3×3 filter, the phenomenon of decreasing again appears, so the subconvolution layer 3333 is referred to as a bottleneck layer. Can be.

The convolution layer 333 is further configured to combine input data (ie, a sampled feature map) with a third feature map output from the stack path. The combining may be performed by element-wise addition processing.

Also, the convolution layer 333 may be further configured to replace data (feature map) output from the subconvolution layer. In one embodiment, the convolution layer 333 may replace the feature value of the feature map using the activation function ReLU. ReLU is a function that outputs 0 for inputs below 0 and linearly proportional values for inputs above 0. In this case, the convolution layer 333 may output an alternative value by applying an activation function ReLU to an output value in which the feature map output by the stack path and the feature map output by the shortcut path are combined. In some embodiments, the convolution layer 333 may be further configured to apply an activation function ReLU to the output values of the subconvolution layers 3331 and 3333.

The convolution layer 333 may be further configured such that the feature extraction process is repeated multiple times. In some embodiments, the convolution layer 333 may include a plurality of residual bottleneck blocks B, such that the feature extraction process is performed multiple times. For example, the convolution layer 333 may include three residual bottleneck blocks B to iterate the aforementioned feature extraction process three times.

Other convolutional layers 334, 335, and 336 also include a residual bottleneck block (B). The residual bottleneck block also includes a plurality of sub-layers including a convolution filter, a stack path for outputting a stacking output as input data sequentially passes through the plurality of sub-layers, and the input data is the stack path It is configured to proceed with a shortcut path to combine with the output data of. Other convolutional layers 334, 335, and 336 will be described based on differences from the convolutional layer 333.

Referring back to FIG. 4A, in one embodiment, the residual bottleneck block B of the convolutional layer 334 includes a plurality of (e.g., featuremaps output from the convolutional layer 333). , 128) Subconvolution layer 341 configured to output a first feature map by applying a 1×1 convolution filter; A sub-convolution layer 3343 configured to output a second feature map by applying a plurality of (eg, 128) 3×3 convolution filters to the first feature map output from the sub-convolution layer 3331; And a subconvolution layer 3345 configured to output a third feature map by applying a plurality of (eg, 512) 1×1 convolution filters to the second feature map output from the subconvolution layer 3333. ).

The subconvolution layers 3331, 3343, and 3345 are sequentially applied to the input data of the stack path.

In one embodiment, one or more of the sub-convolution layers may be further configured to reduce the size of the feature map. Therefore, the data output through the stack path may be sampled data.

For example, as illustrated in FIG. 4A, among the sub-convolution layers 3331, 3343, and 3345, the sub-convolution layer 3331 is configured to move the convolution filter at 2-pixel intervals, and other sub-convolution layers (3343, 3345) is configured so that the convolution filter is moved at intervals of 1 pixel. In this case, the output feature map (64x64 in FIG. 4A) of the sub-convolution layer 3331 is sampled in half the size of the input feature map (128x128 in FIG. 4A), thereby stacking the stack path of the convolution layer 334. The sampled feature map may be output through.

The convolution layer 334 is configured to combine input data traveling through the shortcut path with a feature map output from the stack path (eg, a third feature map of the subconvolution layer 3345 ). The combining may be performed by element-wise addition processing.

In one embodiment, when the data output through the stack path (ie, feature map) is sampled, the residual bottleneck block B of the convolution layer 334 applies a sampling filter to the data going through the shortcut path. It is composed more. The sampling filter causes data corresponding to data output through the stack path to be output through the shortcut path. For example, in the sampling filter, as shown in FIG. 4A, the subconvolution layer 3331 among the subconvolution layers 3331, 3343, and 3345 is configured such that the convolution filter moves at 2-pixel intervals, thereby stacking paths. When a feature map sampled at a size of 1/2 is output through, it may be a sampling filter having a size of 1×1 moving at 2 pixel intervals. As a result, element-wise addition processing in the convolution layer 334 is possible.

Also, the convolution layer 333 may be further configured to replace data (feature map) output from the subconvolution layer. In one embodiment, the convolution layer 334 applies the activation function ReLU to the output values of the subconvolution layers 3331 and 3343, and the activation function ReLU to the combined output values of the subconvolution layers 3345 and 3347. Can be applied to print the replacement value.

The convolution layer 334 may be further configured such that the feature extraction process is repeated multiple times. For example, the convolution layer 334 may include 8 residual bottleneck blocks to iterate through the feature extraction process 8 times.

In some embodiments, when the convolution layer 334 includes a plurality of residual bottleneck blocks, one or more of the plurality of residual bottleneck blocks may be configured not to sample the input feature map.

For example, as shown in FIG. 4A, among the plurality of residual bottleneck blocks included in the convolution layer 334, in the residual bottleneck block in which the feature extraction process is first performed (eg, a filter having a size of 1×1 is 2 The feature map sampled through the stack path and the shortcut path is output by moving in pixel intervals, and in the remaining residual bottleneck block (e.g., by moving a filter of size 1×1 in pixel intervals) through the stack path and shortcut path A feature map that is not sampled may be output.

The residual bottleneck block B of the convolution layer 335 uses a plurality of (eg, 256) convolution filters of size 1×1 on the input feature map (eg, feature map output from the convolution layer 334). A sub-convolution layer 3351 configured to apply and output a first feature map; A subconvolution layer 3353 configured to output a second feature map by applying a plurality of (eg, 256) 3×3 convolution filters to the first feature map output from the subconvolution layer 3351; And a sub-convolution layer 3355 configured to output a third feature map by applying a plurality of (eg, 1024) 1×1 convolution filters to the second feature map output from the sub-convolution layer 3335. It may include.

The residual bottleneck block of the convolutional layer 335 is configured to combine input data traveling through the shortcut path with a feature map output from the stack path (eg, a third feature map of the subconvolution layer 3355).

In addition, the convolution layer 335 may be further configured to replace data (feature map) output from the subconvolution layer.

In one embodiment, at least one of the sub-convolution layers of the residual bottleneck block B included in the convolution layer 335 may be further configured to sample input data (ie, input feature map). In this case, the residual bottleneck block included in the convolution layer 335 is further configured such that data sampled through the stack path is output and data sampled through the shortcut path is output.

The convolution layer 335 may be further configured such that the feature extraction process is repeated multiple times. For example, the convolution layer 335 may include 64 residual bottleneck blocks to iterate the feature extraction process 64 times.

Since the convolution layer 335 is similar to the configuration of the convolution layer 334, detailed description is omitted.

The residual bottleneck block B of the convolution layer 336 includes a plurality of (eg, 512) convolution filters of size 1×1 on the input feature map (eg, feature map output from the convolution layer 335). A sub-convolution layer 3401 configured to apply and output a first feature map; A sub-convolution layer 3403 configured to output a second feature map by applying a plurality of (eg, 512) 3×3 convolution filters to the first feature map output from the sub-convolution layer 3161; And a subconvolution layer 3365 configured to output a third feature map by applying a plurality of (eg, 2048) 1×1 convolution filters to the second feature map output from the subconvolution layer 3336. It may include.

The residual bottleneck block (B) of the convolution layer 336 is such that the input data going through the shortcut path is combined with the feature map output from the stack path (eg, the third feature map of the subconvolution layer 3356). It is composed.

In addition, the convolution layer 336 may be further configured to replace data (feature map) output from the sub-convolution layer.

In one embodiment, at least one of the sub-convolution layers of the residual bottleneck block B included in the convolution layer 336 may be further configured to sample the input data (ie, input feature map). In this case, the residual bottleneck block B included in the convolution layer 336 is further configured such that data sampled through the stack path is output and data sampled through the shortcut path is output.

The convolution layer 336 may be further configured such that the feature extraction process is repeated multiple times. For example, the convolution layer 336 may include three residual bottleneck blocks B to iterate the feature extraction process three times.

Since the convolution layer 336 is similar to the configuration of the convolution layer 334, detailed description is omitted.

When the fundus photograph is processed through the aforementioned convolution layers 331, 333, 334, 335, and 336, features capable of predicting whether or not a cerebral aneurysm is extracted are extracted. For example, in the feature extraction layer 330, features related to discs and blood vessels in the eye may be output as an activation map by applying an activation function.

The average pooling layer 337 samples the feature map by averaging the feature map output through the convolution layer 336. In one example, when the feature map having a size of 16x16 is output from the convolution layer 336, the average pooling layer 337 is averaged using a filter of size 16×16 and has a size of 1×1×2048. The brain aneurysm feature map is output.

The classification layer 350 may include a full connected layer 351 and/or a probability layer 355.

The fully connected layer 351 is composed of a plurality of fully connected nodes, and each node converts input data to output data based on the weight for each node. The fully connected layer 351 receives the cerebral aneurysm feature map output from the feature extraction layer 330 and outputs feature values based on weights of a plurality of nodes.

In one embodiment, feature values output from the fully connected layer 351 are classified into two groups. For example, a feature value having a size of 1×1×2 is output from the complete connection layer 351 and is classified into a cerebral aneurysm group or a non-cerebral aneurysm group.

The probability layer 355 calculates a probability that the fundus photograph belongs to a group of cerebral aneurysms based on the output value output from the complete connection layer 351. In one example, the probability layer 355 may calculate the probability by a softmax equation that can be expressed as follows.

[Equation 1]

Here, yk represents the probability of belonging to the k-th group. And, n is the number of nodes of the output layer in the fully connected layer 351, e ^ak represents the exponential function of the input signal a _k of the fully connected layer 351.

The softmax layer 355 may calculate a probability value that has a value between 0 and 1 by dividing the value of the k-th element by the sum of all elements.

The classification layer 350 determines whether the fundus picture is a fundus picture of a person with a cerebral aneurysm based on the probability value calculated by the softmax layer 355. In one example, when the probability P1 of belonging to the cerebral aneurysm group is 0.5 or more, the fundus photograph may be classified as a cerebral aneurysm group. The classification method based on the probability is a simple example and is not limited thereto, and a probability threshold value by various methods may be used for classification. In another example, an optimal value obtained using the ROC curve may be set as a probability threshold.

Equation 1 is a simple example, and the probability P1 may be calculated by another Softmax equation.

The learning unit 30 updates parameters of the cerebral aneurysm prediction model of FIG. 3 according to learning. Specifically, the fundus photographs included in the plurality of training samples are input to the cerebral aneurysm prediction model to classify the fundus photographs, and the classification results (ie, model application results) and disease data included in the training samples (ie, actual results) ) To update the parameter to reduce the error. Through this process, the feature extraction layer 330 (eg, a convolutional layer) and/or the classification layer 350 (eg, a fully connected layer) are learned.

The learning unit 30 models the brain aneurysm prediction model of FIG. 3, which determines whether a cerebral aneurysm is based on a monocular fundus photograph using a learning sample. Here, the learning sample is a monocular fundus photograph, and includes a left-eye fundus photograph or a right-eye fundus photograph.

In some embodiments, when a left eye and a right eye fundus photograph are acquired from a specific photographer for a sample, each fundus photograph of a particular photographer is not used as a learning sample. For example, when a model for predicting a brain aneurysm based on a left eye fundus photograph for a specific photographer is trained, a model for predicting a brain aneurysm based on a right eye fundus photograph for a specific photographer is not further trained. However, in the above example, in learning the brain aneurysm prediction model of FIG. 3, it is not limited to a particular type of monocular fundus photograph. If the brain aneurysm prediction model is learned based on the left eye fundus photograph for the first photographer, the brain aneurysm prediction model may be further trained based on the right eye fundus photograph for the second photographer.

The parameters of the cerebral aneurysm prediction model using the fundus picture as input are updated in a direction in which the cost function of the brain aneurysm prediction disease model is minimized. Here, the cost function represents the difference between the result value from the model and the actual result value. Updates of these parameters are commonly referred to as optimization. In one example, parameter optimization may be performed through adaptive moment estimation (ADAM), but is not limited thereto, and various gradient discents such as Momentum, NAG (Nesterov Accelerated Gradient), Adagrad (Adaptive Gradient), RMSProp, etc. It can also be carried out in a manner.

The learning process for the convolution layer and the fully connected layer consists of three steps: bias calucation, error back propagation, and weight update. On the other hand, since the pooling layer has no weight assigned to the connecting line, it consists of only two steps: forward calculation and error back propagation. This is a learning process that is commonly used for the learning model of the CNN structure, so a detailed description is omitted.

Eventually, the brain aneurysm prediction model of FIG. 3 is learned to enhance the brain aneurysm prediction ability. Specifically, in the cerebral aneurysm prediction model of FIG. 3, a fundus picture included in a plurality of training samples is input, and features for predicting cerebral aneurysm are extracted from the fundus picture of the sample, and then the fundus picture is input based on the characteristic. Learning to classify into a group of cerebral aneurysms or a group of non-brain aneurysms. The apparatus for predicting cerebral aneurysm 1000 may determine whether the subject is a patient with cerebral aneurysm by applying a monocular fundus photograph of the subject to the learned brain aneurysm prediction model 1.

5 is a flowchart illustrating a process of determining a cerebral aneurysm using the model of FIG. 3 according to an embodiment of the present invention.

In step S610, a fundus photograph of a subject is acquired. The fundus photograph of the subject may be taken by a fundus camera. The fundus camera may be electrically connected to the brain aneurysm prediction apparatus 1000 to enable wired/wireless communication.

In one embodiment, when the fundus camera is externally coupled or configured to be electrically communicable by remote location, the data acquisition unit 10 of the cerebral aneurysm prediction apparatus 1000 acquires fundus photo data taken by the fundus camera do.

The data acquisition unit 10 provides a monocular fundus photograph of the subject to the prediction unit 50. In the above process, provision of a monocular fundus photograph may be performed in various ways. In one example, the data acquisition unit 10 may acquire a monocular fundus photograph. In another example, when a binocular fundus photograph is taken by the fundus camera and a binocular fundus photograph is received, the data acquisition unit 10 filters the binocular fundus photograph to provide a monocular fundus photograph to the predictor 50.

In some embodiments, the data acquisition unit 10 may further provide the prediction unit 50 with data related to fundus photos, such as a subject's identity (eg, including a name, age, identifier, etc.).

Then, in step S650, the prediction unit 50 applies a monocular fundus photograph of the subject to the cerebral aneurysm prediction model 1. The cerebral aneurysm prediction model 1 classifies the fundus image into a cerebral aneurysm group or a non-disease group. If the fundus photograph is classified as a group of cerebral aneurysms, it is determined that the subject of the fundus photograph has a cerebral aneurysm (ie, the subject of the fundus photograph is a cerebral aneurysm disease).

Additionally, the cerebral aneurysm prediction apparatus 1000 may provide a prediction result of the prediction unit 50 to the user.

Using the brain aneurysm prediction model 1, it is also possible to predict whether the subject has an unruptured cerebral aneurysm. In particular, since the invasive test is not performed in the prediction process, the subject's body may be safer.

It will be apparent to those skilled in the art that the cerebral aneurysm prediction apparatus 1000 may include other components not described herein. For example, the cerebral aneurysm prediction device 1000 includes a network interface, an input device for data entry, and an output device for display, printing, or other data display, and other hardware elements required for the operations described herein. It may include.

Example 2

6 is a diagram for explaining the structure of a brain aneurysm prediction model according to a second embodiment of the present invention.

Since the structure of the brain aneurysm prediction model 2 according to the second embodiment of the present invention is substantially similar to the structure of the brain aneurysm prediction model according to the first embodiment of FIG. 3, differences will be mainly described.

Referring to FIG. 6, the brain aneurysm prediction model 2 according to the second embodiment includes a first input layer 310A, a first feature extraction layer 330A, a second input layer 310B, and a second feature extraction layer 330B and classification layer 350. Also, a combination layer 340 that combines the output value of the first feature extraction layer 330A with the second feature extraction layer 330B may be included.

The cerebral aneurysm prediction model (2) is trained by receiving both fundus photos. In the fundus photograph of the first eye (eg, the left eye) of both eyes, features are extracted through the first input layer 310A and the first feature extraction layer 330A, and the second eye of the binocular eye (eg, the right eye) The fundus picture for the feature is output through the second input layer 310B and the second feature extraction layer 330B.

In one embodiment, the first feature extraction layer 330A and the second feature extraction layer 330B may have the same structure as in FIG. 4A described above. In this case, the feature value for the first one output from the first feature extraction layer 330A is a feature map having a size of 1x1x2048, that is, a feature vector is output, and the feature for the second feature output from the second feature extraction layer 330B. The value is a feature map of size 1x1x2048, that is, a feature vector is output.

The combining layer 340 combines the feature values for the first draft and the feature values for the second draft before inputting the extracted features into the classification layer 350. In one example, the combining of the combining layer 340 is a combination different from the element-by-element combination of the feature map performed in the residual bottleneck block, and the feature vector output from the first feature extraction layer 330A as shown in FIG. 6. The feature vector output from the second feature extraction layer 330B is performed by vector-concatenation processing.

The classification layer 350 classifies the ocular fundus photograph into a cerebral aneurysm group and a non-cerebral aneurysm group by receiving the combined feature values from the combining layer 340.

The cerebral aneurysm prediction model 2 machine-learns the parameters of the feature extraction layers 330A, 330B and/or the classification air 350 using a training sample including a fundus photograph of both eyes. Learning of the brain aneurysm prediction model 2 having the structure of FIG. 7 is similar to learning of the brain aneurysm prediction model 1 of the first embodiment, so a detailed description thereof will be omitted. The cerebral aneurysm prediction apparatus 1000 determines whether the subject is a patient with a cerebral aneurysm by applying a photograph of the subject's binocular fundus to the previously learned brain aneurysm prediction model 2 of FIG. 7.

The process of predicting a cerebral aneurysm using the model of FIG. 6 is similar to that of FIG. 5.

First, in step S610, the data acquisition unit 10 acquires a binocular fundus photograph of the subject. Alternatively, the subject identification information may be further obtained.

Then, in step S630, the prediction unit 50 applies a photo of the both eyes of the subject to the brain aneurysm prediction model 2 of FIG. The prediction unit 50 inputs a fundus picture for the first eye among the subject's binocular fundus photos as the first input layer 310A, and inputs a fundus photo for the second eye into the second input layer 310B. The cerebral aneurysm prediction model 2 determines whether a subject's binocular fundus photograph is classified into a cerebral aneurysm group based on feature values extracted from the binocular fundus photograph.

Then, in step S650, if it is determined that the binocular fundus photograph is classified into the brain aneurysm prediction group, the prediction unit 50 determines that the subject of the binocular fundus photograph has a cerebral aneurysm.

7 is a diagram illustrating prediction results using a binocular fundus photograph according to an embodiment of the present invention.

As shown in FIG. 7, when the cerebral aneurysm prediction apparatus 1000 applies a binocular fundus photograph of the subject X, which is an actual cerebral aneurysm disease, to the cerebral aneurysm prediction model 2, the subject X becomes a cerebro aneurysm disease The determined result can be obtained. In addition, when the cerebral aneurysm prediction apparatus 1000 applies a binocular fundus photograph of the subject Y, who is a non-encephalopathic patient, to the cerebral aneurysm prediction model 2, it is possible to obtain a result of determining the subject Y as normal. have.

Experimental Example

8 is a diagram for explaining the performance of the models of FIGS. 3 and 6 according to experimental examples of the present invention.

In the first experimental example, the ratio of the fundus photo and the normal fundus photo of a cerebral aneurysm disease was set to 1:1, and then learning was performed using a monocular fundus photo. After setting the ratio of 1:1, the study was performed using a binocular fundus photograph, and in the third experimental example, after setting the ratio of the fundus photograph and normal fundus photograph of a cerebral aneurysm disease patient to 1:2, the binocular fundus photograph was taken. I learned using it. That is, the first experimental example is related to the model of FIG. 3, and the second and third experimental examples are related to the model of FIG. 6.

Experimental conditions of the above experimental examples were performed in the following environment. Learning is performed by random flip for data augmentation, random rotation between -30° and 30°, random rotation, random brightness adjustment and random contrast adjustment. adjustment). When training the model using binocular fundus photography, data reinforcement was applied to each fundus photography. Also, for normalization of images, Z-score normalization was used as a pre-processing technique. In the learning process, the batch setting value was set to 4 to learn 20,000 iterations, and the ADAM method was used for optimization. The optimization parameters were trained with the initial values (beta1: 0.9, beta2: 0.999) set. As a loss function, a cross-entropy function was used.

Because the data is small, the performance of the models 1 and 2 of FIGS. 3 and 6 was verified through a 5-fold corss validation method. Cross-validation is a verification method that ensures that all data is used as a test set at least once. It is a verification method that divides the data into multiple groups and then modifies the test set each time to evaluate the performance of the training model.

The training sample was divided into 5 folds to verify, and the first to fourth folds were training samples (ie, training data) with 572 fundus photos (286 aneurysms, 286 normals), and the subject's fundus photos. (I.e., test data) was set to 144, and the fifth fold was set to 576 sample fundus images for training (aneurysm 288, normal 288), and 140 subjects' fundus photos.

The learning results according to the first to third experimental examples were derived as quantitative results through the following four evaluation methods: ROC AUC (Receiver Operating Characteristic Area Under Curve), ACC (Accuracy), Se (Sensitivity), Sp (Specificity).

Referring to FIG. 8, both the brain aneurysm prediction model 1 trained using monocular fundus photography and the brain aneurysm prediction model 2 trained using binocular fundus photography have an ROC AUC of approximately 97 to 99%. In addition, a brain aneurysm prediction model (1) trained using monocular fundus photography has an ACC of approximately 90%, and a brain aneurysm prediction model (2) trained using binocular fundus photography requires at least 80% ACC. Have That is, it can be confirmed that the brain aneurysm prediction models 1 and 2 learned using the fundus photograph according to the present invention have excellent cerebral aneurysm prediction performance. This is a better result when a human doctor considers that an aneurysm cannot be predicted by a fundus picture, as screening tests such as invasive or high-radiation use were previously required to diagnose cerebral aneurysms.

According to the embodiments described above The operation by the apparatus and method for predicting cerebral aneurysms may be implemented, at least in part, by a computer program and recorded in a computer-readable recording medium. For example, implemented with a program product consisting of a computer-readable medium containing program code, which may be executed by a processor for performing any or all of the steps, operations, or procedures described.

The computer may be a desktop computer, a laptop computer, a laptop, a smart phone, or a computing device such as the like, or it may be any device that may be integrated. A computer is a device with one or more alternative, special-purpose processors, memory, storage space, and networking components (either wireless or wired). The computer may, for example, run an operating system such as Microsoft's Windows compatible operating system, Apple OS X or iOS, Linux distribution, or Google's Android OS.

The computer-readable recording medium includes all types of record identification devices that store data that can be read by a computer. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tapes, floppy disks, and optical data storage identification devices. In addition, the computer readable recording medium may be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the present embodiment will be readily understood by those skilled in the art to which this embodiment belongs.

The present invention described above has been described with reference to the embodiments shown in the drawings, but this is merely exemplary, and those skilled in the art will understand that various modifications and modifications of the embodiments are possible therefrom. However, it should be considered that such modifications are within the technical protection scope of the present invention. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the appended claims.

According to an aspect of the present invention, a device for predicting cerebral aneurysm is a non-invasive and non-invasive and one of the fourth industrial technology using machine learning, which is different from the conventional aneurysm-based inspection-oriented brain aneurysm prediction technology. Predictable. As a result, cerebral aneurysms can be predicted more safely than conventional cerebral aneurysms.

In particular, cerebral aneurysms can be predicted using fundus photography. The retina fundus of the eye is a film-like tissue of a camera that recognizes real light, and a fundus camera can be used to easily obtain a fundus picture. Fundus cameras do not require invasive treatment, do not use radiation, and have a short shooting time within minutes. Fundus examination using such a fundus camera is very inexpensive (current insurance price is 7990 won, general price is 20770 won). In addition, fundus cameras are not only distributed to almost all ophthalmology clinics and ophthalmology clinics, but also have a large number of examination centers, making them excellent in accessibility and dissemination. Above all, fundus photography provides optic nerve papilla, retina, and choroidal blood vessels (arteries, veins, capillaries, etc.) at very high resolution. Due to the tissue characteristics of the retinal fundus that can directly observe blood vessels in the retina, fundus photography is the latest self It has dozens of times the resolution of vascular resolution of resonance imaging equipment.

If it is possible to predict cerebral aneurysms using fundus photography with these advantages, it is expected to achieve a breakthrough in preventive treatment for cerebral aneurysms such as subarachnoid hemorrhage. Therefore, it is possible to obtain enormous effects that can reduce social and economic costs due to subarachnoid hemorrhage.

Furthermore, the present invention suggests the possibility of determining cerebrovascular abnormality from fundus photography. The optic nerve papilla and the blood vessels of the retina are genetically differentiated parts of the brain and have characteristics similar to those of the brain. In light of this, it is judged that it can open a new horizon in the medical field theory of the relationship between retinal vessel abnormalities and cerebrovascular abnormalities.

Claims (23)

1. In the method of providing a brain aneurysm prediction results performed by a computing device including a processor,

   Obtaining a fundus photograph of the subject; And

   Including the step of determining whether the subject has a cerebral aneurysm by applying the fundus photo data of the subject to a brain aneurysm prediction model;

   The cerebral aneurysm prediction model provides a method for predicting cerebral aneurysm, characterized in that the ability to classify an input fundus photograph into a cerebral aneurysm group using a plurality of training samples including a learning fundus photograph.
2. According to claim 1, wherein the brain aneurysm prediction model,

   An input layer into which a fundus picture is input;

   A feature extraction layer including a plurality of convolutional layers and a pooling layer and extracting features from the fundus photo of the input layer; And

   A method of providing a predicted result of a cerebral aneurysm comprising a complete connection layer including a plurality of nodes, and comprising a classification layer that classifies the fundus picture input based on the feature into a cerebral aneurysm group or a non-disease group.
3. According to claim 2,

   At least one of the plurality of convolutional layers, a method of providing an aneurysm prediction result, characterized in that it comprises a residual bottleneck block configured to recombine after the input data proceeds to a plurality of paths.
4. According to claim 3,

   The residual bottleneck block is configured such that input data proceeds through a first path and a second path, and data progressed through the first path and data progressed through the second path are combined,

   Method for providing a predicted result of a cerebral aneurysm, characterized in that the data proceeding through the first path proceeds through a plurality of sub-convolution layers configured to apply a convolution filter.
5. The method of claim 4,

   At least one of the plurality of sub-convolutional layer is configured to output data of a lower dimension compared to other convolutional filters.
6. The method of claim 5, wherein the residual bottleneck block,

   A first sub-convolution layer that outputs the first feature map by applying a plurality of 1×1 size filters to the input feature map, and a plurality of 3×3 sizes in the first feature map output from the first sub-convolution layer Applying a filter of the second sub-convolution layer to output the second feature map, and the second feature map output from the second sub-convolution layer by applying a plurality of 1 × 1 filter size of the third feature map A method for providing a predicted result of a cerebral aneurysm, comprising an output third subconvolution layer.
7. The method of claim 4,

   When at least one of the plurality of sub-convolution layers is modeled to output sampling data, a brain aneurysm characterized in that the input data proceeding through the second path is sampled to correspond to the sampling data of the first path. How to provide predictive results.
8. According to claim 3,

   The method for providing a predicted result of a cerebral aneurysm, wherein data combining in the residual bottleneck block is performed by element-wise addition processing.
9. According to claim 2,

   The feature extraction layer includes a first convolution layer,

   The first convolution layer is configured to output a first feature map using 64 filters of size 7×7 on the fundus image output from the input layer.
10. The method of claim 9,

    The feature extraction layer further comprises a max pooling layer configured to perform max pooling of the first feature map.
11. The method of claim 9,

    The feature extraction layer further includes a second to fourth convolution layer,

    At least one of the second convolutional layer to the fourth convolutional layer includes the residual bottleneck block.
12. The method of claim 11,

    A method for providing a brain aneurysm prediction result, wherein the residual bottleneck block included in at least one of the second convolution layer to the fourth convolution layer is a plurality.
13. The method of claim 12,

    If the plurality of residual bottleneck blocks, one or more residual bottleneck blocks are configured to sample the input data to output a brain aneurysm prediction method.
14. According to claim 1, wherein the brain aneurysm prediction model,

    Learning by updating the parameters included in the brain aneurysm prediction model in a direction that minimizes the cost function of the aneurysm prediction model,

    The cost function (loss function) is a method for providing a brain aneurysm prediction result, characterized in that it represents the difference between the actual value and the result of applying the fundus picture to the brain aneurysm prediction model.
15. The method of claim 14, wherein the model for predicting cerebral aneurysm,

    A method for providing a predicted result of a cerebral aneurysm, characterized by being trained using a learning sample including a monocular fundus photograph.
16. The method of claim 15, wherein the step of obtaining a fundus photograph of the subject,

    When the subject's binocular fundus photograph is obtained, a method of providing a cerebral aneurysm prediction result, further comprising filtering the binocular fundus photograph so that the monocular fundus photograph is applied to the cerebral aneurysm prediction model.
17. The method of claim 14, wherein the model for predicting cerebral aneurysm,

    A method for providing a predicted result of a cerebral aneurysm, characterized by learning using a learning sample including a binocular fundus photograph.
18. The method of claim 17, wherein the brain aneurysm prediction model,

    A first input layer into which a first fundus photograph is input;

    A first feature extraction layer for extracting features of the first fundus picture;

    A second input layer into which a second fundus photograph is input;

    A second feature extraction layer for extracting features of the second fundus picture;

    A combining layer configured to combine the feature values of the first feature extraction layer and the feature values of the second feature extraction layer; And

    A method of providing a predicted result of a cerebral aneurysm comprising a classification layer for classifying a ocular fundus photograph into a cerebral aneurysm group or a non-disease group based on the combined feature values.
19. 19. The method of claim 18, wherein the binding layer is configured to process vector concatenation.
20. A computer-readable recording medium that stores program instructions readable by a computing device and operable by the computing device, wherein the processor is executed when the program instructions are executed by a processor of the computing device. Computer-readable recording medium for performing method of predicting cerebral aneurysm according to any one of the preceding claims
21. A brain aneurysm prediction device for predicting cerebral aneurysm using fundus photography,

    A storage unit that stores a pre-trained brain aneurysm prediction model;

    A data acquisition unit that acquires a fundus photograph of the subject; And

    And a predictor configured to determine whether the subject has a cerebral aneurysm by applying the fundus photo data of the subject to the cerebral aneurysm prediction model.
22. The method of claim 21, wherein the brain aneurysm prediction model,

    An input layer into which a monocular fundus photograph is input;

    A feature extraction layer including a plurality of convolutional layers and a pooling layer and extracting features from the fundus photo of the input layer; And

    A device for predicting cerebral aneurysm, comprising a complete connection layer including a plurality of nodes, and comprising a classification layer that classifies the fundus picture input based on the feature into a cerebral aneurysm group or a non-disease group.
23. The method of claim 21, wherein the brain aneurysm prediction model,

    A first input layer into which a first fundus photograph is input;

    A first feature extraction layer for extracting features of the first fundus picture;

    A second input layer into which a second fundus photograph is input;

    A second feature extraction layer for extracting features of the second fundus picture;

    A combining layer configured to combine the feature values of the first feature extraction layer and the feature values of the second feature extraction layer; And

    And a classification layer for classifying a ocular fundus photograph into a cerebral aneurysm group or a non-disease group based on the combined feature values.

Images