KR20210140836A - Method and apparatuses to predict orbital and periorbital lesions - Google Patents

Abstract

Disclosed are an apparatus for predicting orbital and periorbital lesions, which predicts the location and type of lesions in a patient's orbit quickly and accurately, and a prediction method thereof. According to the present invention, the apparatus receives a target medical image of a patient's skull and inputting the target medical image to a first deep learning model to generate a classification result for the patient's orbital structure; inputs the classification result to a second deep learning model to generate an estimation result on whether a lesion has occurred in the orbital structure; inputs the classification result into a third deep learning model according to the estimation result to generate a prediction result for the location and type of the lesion occurring in the orbital structure and output the estimation result and the prediction result; and trains the first deep learning model, the second deep learning model, and the third on the basis of at least one of a sample medical image, biopsy information corresponding to the sample medical image, and a synthetic medical image generated on the basis of the sample medical image and the biopsy information.

Description

Apparatus for predicting orbital and peripheral lesions and a method for predicting the same

The present invention provides an apparatus for predicting orbital and peripheral lesions and a method for predicting the same.

The orbit refers to the periphery surrounding the eyeball, and various structures exist in the orbit. Although the incidence of lesions in the orbit of a patient is not high, if it does occur, it can be very fatal to the patient. For example, malignant tumors among lesions occurring in a patient's orbit have a very high mortality rate (e.g. 26.4% for orbital lymphoma, 38.5% for lacrimal gland cancer). In addition, even in the case of benign tumors, there is a high possibility that the tumor may compress the optic nerve and cause blindness (irreversible damage, blindness rate of 90%), or the tumor may invade the extraocular muscle and cause diplopia.

For lesions occurring in the patient's orbit, if the diagnosis is made quickly, appropriate treatment according to the lesion (procedure monitoring, surgery, drug treatment/chemotherapy, surgery, and radiation treatment for malignant tumors) is possible, but the treatment period is from 6 months to 6 months. Delays of more than a year often occur. In the case of lesions occurring in the orbit, especially inflammatory orbital and periorbital lesions (pseudotumors), biopsy is essential. In the case of anterior orbital lesions, biopsy is possible due to good accessibility. In the case of orbital apex), the accessibility of biopsy is poor, and the orbit itself is dangerous because major structures including the eye are concentrated in a narrow space. For this reason, differential diagnosis (a diagnostic method that compares and reviews diseases with similar characteristics to determine the name of the disease) is performed for orbital lesions that have occurred in locations where biopsy is not possible, but the initial diagnosis is difficult and the treatment period is highly likely to be prolonged.

Embodiments provide an orbital and peripheral lesion prediction apparatus and a prediction method capable of rapidly and accurately predicting the location and type of a lesion occurring in the orbit of a patient.

In addition, embodiments provide an orbital and peripheral lesion prediction apparatus and a prediction method capable of predicting the location and type of a lesion occurring in a location where a biopsy is impossible.

However, the technical task to be achieved by the present embodiment is not limited to the technical task as described above, and other technical tasks may exist.

The present specification receives a target medical image of a patient's skull, inputs the target medical image to a first deep learning model, generates a classification result for the patient's orbital structure, and inputs the classification result to a second deep learning model to generate an estimation result on whether or not a lesion has occurred in the orbital structure, and input the classification result into the third deep learning model according to the estimation result to generate a prediction result for the location and type of the lesion occurring in the orbital structure, and estimate A first deep learning model for outputting results and prediction results, based on at least one of a sample medical image, biopsy information corresponding to the sample medical image, and a synthetic medical image generated based on the sample medical image and biopsy information; Disclosed are an orbital and periorbital lesion prediction apparatus and prediction method for learning a second deep learning model and a third deep learning model.

An embodiment includes an input unit for receiving a target medical image of a patient's skull; a structure classification unit that inputs a target medical image to the first deep learning model and generates a classification result for the orbital structure of the patient; an estimator that inputs the classification result into the second deep learning model and generates an estimation result as to whether a lesion has occurred in the orbital structure; a lesion prediction unit that inputs a classification result to the third deep learning model according to the estimation result and generates a prediction result for the location and type of the lesion occurring in the orbital structure; an output unit for outputting an estimation result and a prediction result; and a first deep learning model, a second deep learning model, and a second deep learning model based on at least one of a sample medical image, biopsy information corresponding to the sample medical image, and a synthetic medical image generated based on the sample medical image and the biopsy information. 3 It provides an orbital and peripheral lesion prediction device including a model learning unit for learning a deep learning model.

Another embodiment may include an input step of receiving a target medical image obtained by photographing a patient's skull; a structure classification step of generating a classification result for a patient's orbital structure by inputting a target medical image into a first deep learning model; an estimation step of generating an estimation result as to whether a lesion has occurred in the orbital structure by inputting the classification result into the second deep learning model; a lesion prediction step of generating a prediction result for a location and type of a lesion occurring in an orbital structure by inputting a classification result into a third deep learning model according to the estimation result; and an output step of outputting an estimation result and a prediction result, wherein the first deep learning model, the second deep learning model, and the third deep learning model are a sample medical image, biopsy information corresponding to the sample medical image, and a sample medical image. and an orbital and peripheral lesion prediction method that is learned based on at least one of a synthetic medical image generated based on biopsy information.

According to the apparatus and method for predicting orbital and peripheral lesions according to embodiments, the location and type of lesions occurring around the orbit of a patient can be quickly and accurately predicted.

Further, according to the apparatus and method for predicting orbital and orbital lesions according to embodiments, it is possible to predict the location and type of a lesion occurring in a location where a biopsy is impossible.

1 is a block diagram illustrating an example of an orbital and peripheral lesion prediction apparatus.
2 is a diagram illustrating an example of a target medical image.
3 is a diagram illustrating an example of a classification result for an orbital structure in a target medical image.
4 is a block diagram illustrating another example of an orbital and peripheral lesion prediction apparatus.
5 is a diagram illustrating an example of generating preprocessing data.
6 is a block diagram illustrating another example of an orbital and peripheral lesion prediction apparatus.
7 is a flowchart illustrating an example of an operation of determining whether to generate a synthetic medical image.
8 is a diagram illustrating an example of generating a synthetic medical image.
9 is a flowchart illustrating an example of a method for predicting orbital and periorbital lesions.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement them. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is "connected" to another part, it includes not only the case where it is "directly connected" but also the case where it is "electrically connected" with another element interposed therebetween. . Also, when a part "includes" a component, it means that other components may be further included, rather than excluding other components, unless otherwise stated, and one or more other features However, it is to be understood that the existence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded in advance.

The terms "about", "substantially", etc. to the extent used throughout the specification are used in a sense at or close to the numerical value when the manufacturing and material tolerances inherent in the stated meaning are presented, and serve to enhance the understanding of the present invention. To help, precise or absolute figures are used to prevent unfair use by unscrupulous infringers of the stated disclosure. As used throughout the specification of the present invention, the term “step for (to)” or “step for” does not mean “step for”.

In this specification, a "part" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. In addition, one unit may be implemented using two or more hardware, and two or more units may be implemented by one hardware.

In this specification, some of the operations or functions described as being performed by the terminal, apparatus, or device may be performed instead of in a server connected to the terminal, apparatus, or device. Similarly, some of the operations or functions described as being performed by the server may also be performed in a terminal, apparatus, or device connected to the server.

In this specification, some of the operations or functions described as mapping or matching with the terminal means mapping or matching the terminal's unique number or personal identification information, which is the identification data of the terminal. can be interpreted as

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating an example of an orbital and peripheral lesion prediction apparatus.

Referring to FIG. 1 , the orbital and peripheral lesion prediction apparatus 100 includes an input unit 110 , a structure classification unit 120 , an estimator 130 , a lesion prediction unit 140 , an output unit 150 , and model learning. It may include a unit 160 .

The input unit 110 may receive a target medical image obtained by photographing the patient's skull. The target medical image may be an image taken from the front (face direction), side (ear direction), or upper part (parietal direction) of the patient's skull using an imaging device such as an X-ray device, ultrasound device, CT, or MRI. have.

In this case, the target medical image may be, for example, a CT image or an MRI image. CT images or MRI images may indicate orbital structures and various types of lesions occurring in orbital structures. CT or MRI can recognize that an abnormal element occupies the orbital space in the orbital structure, and can detect lesions (eg tumors) occurring in the orbital structure based on the degree of reflection/absorption of X-rays and the intensity of the resonance signal. because it can be perceived. A specific example of the target medical image will be described in detail below with reference to FIG. 2 .

The structure classification unit 120 may input the target medical image received from the input unit 110 into the first deep learning model MDL_1 to generate a classification result for a structure located in the patient's orbit, that is, the orbital structure.

In this case, there may be more than one orbital structure, and each structure includes orbital walls, fissures, canals, periorbita, optic nerve, extraocular muscle, and orbital fat. (orbital fat), nerve, vasculature, annulus of Zinn, intraorbital tissue, crystalline lens, corneal apex, orbit apex), lacrimal gland or vitreous orbital lesions, and orbital lesions of the nasal and paranasal sinuses. The structure classification unit 120 may generate a classification result for a specific orbital structure for which a lesion is to be searched among orbital structures.

In this case, the first deep learning model MDL_1 may be an object detection & segmentation model that receives a target medical image and performs detection and segmentation on an orbital structure.

The classification result of the orbital structure may indicate information on where the region corresponding to the orbital structure is in the target medical image. For example, as a result of classification of structures located in the orbit, information such as which part corresponds to the optic nerve in the target medical image and which part corresponds to the extraocular muscle in the target medical image is used for masking a specific region in the target medical image. can be directed through.

The estimation unit 130 may input the classification result generated by the structure classification unit 120 into the second deep learning model (MDL_2) to generate an estimation result as to whether a lesion has occurred in a structure located in the orbit of the patient. have.

In this case, the lesion occurring in the structure located in the orbit may be a fracture, an inflammatory disease, or a tumor (malignant/benign).

The second deep learning model MDL_2 may be an object detection model that detects a lesion based on the size/shape of a structure located in the orbit.

The estimation result generated by the estimator 130 may indicate a result of estimating whether the orbital structure is normal, that is, a lesion has not occurred or is abnormal, that is, a lesion has occurred. For example, the estimator 130 may indicate a result of estimating whether a specific lesion (eg malignant tumor) has occurred in a specific structure (eg extraocular muscle) located in the orbit as a probability value (eg 80%/90%). have.

The lesion prediction unit 140 inputs the classification result for the orbit structure generated by the structure classification unit 120 to the third deep learning model MDL_3 according to the estimation result generated by the estimation unit 130 to the orbit structure. It is possible to generate prediction results for the location and type of lesions that have occurred in For example, the prediction result for the type of lesion may be a result of predicting the presence or absence of thyroid eye disease, whether it is a fracture/inflammation/tumor, whether the tumor is malignant or benign, and the type of tumor in the case of a malignant tumor.

The third deep learning model MDL_3 may be a model that receives an image of an orbital structure, divides a portion corresponding to a lesion, and classifies a type of lesion.

For example, if the estimation result indicates that no lesion has occurred in the orbital structure, there is no need to predict the location and type of the lesion, so the lesion predicting unit 140 predicts the location and type of the lesion occurring in the orbital structure. I never do that.

On the other hand, when the estimation result indicates that a lesion has occurred in the orbital structure, the lesion prediction unit 140 predicts the location and type of the lesion occurring in the structure located in the orbit using the third deep learning model (MDL_3). do.

For example, the prediction result may be a probability value (e.g. 80%/90%) of a specific type (type) of a tumor (malignant) in a specific structure (e.g. extraocular muscle) located in the orbit.

The output unit 150 may output one or more of the estimation result generated by the estimator 130 and the prediction result generated by the lesion prediction unit 140 . The output unit 150 may visualize the estimated result and the predicted result in text, images, etc. and provide them to medical staff and patients.

For example, the output unit 150 may output information indicating that the orbital structure is normal (ie, no lesion has occurred) as a result of the estimation. As another example, when the orbital structure is abnormal (ie, a lesion occurs) as a result of the estimation, the output unit 150 may output information indicating that the orbital structure is abnormal and the location and type of the lesion generated in the structure.

Meanwhile, the output unit 150 may output the estimation result and the prediction result to a display device (eg, a smartphone, tablet, monitor, TV, LED) capable of displaying information. Based on at least one of an image, biopsy information corresponding to the sample medical image, and a synthetic medical image generated based on the sample medical image and the biopsy information, the first deep learning model (MDL_1), the second deep learning Learning on the model MDL_2 and the third deep learning model MDL_3 may be performed.

In this case, the sample medical image and biopsy information may be stored in the storage device 10 located outside the orbital and orbital lesion prediction apparatus 100 . The storage device 10 may be located outside the orbital and orbital lesion prediction apparatus 100, for example, and may be a PC, a server, a medical information system, etc. in which a database is built.

Meanwhile, the storage device 10 additionally stores the aforementioned first deep learning model (MDL_1), second deep learning model (MDL_2), and third deep learning model (MDL_3) as well as the aforementioned sample medical image and biopsy information. You can also save it. In this case, the orbital and peripheral lesion prediction apparatus 100 may load the first deep learning model (MDL_1), the second deep learning model (MDL_2), and the third deep learning model (MDL_3) stored in the storage device 10 . and the first deep learning model (MDL_1), the second deep learning model (MDL_2), and the third deep learning model (MDL_3), which have been trained and updated, may be written back to the storage device 10 .

Like the target medical image, the sample medical image may be a CT image or an MRI image of various patients' skulls from the front, side, and upper sides.

The biopsy information corresponding to the sample medical image may indicate a result of performing a biopsy on the orbital structure in which the corresponding sample medical image is captured (ie, an examination for checking cells constituting the structure through a microscope). For example, the biopsy information may indicate whether a tumor has occurred in the orbital structure (e.g. extraocular muscle), whether the tumor is malignant or benign, and information on the type of tumor.

There is a correlation between the sample medical image and the biopsy information corresponding thereto, and the lesion, especially the tumor, generated in the structure located in the orbit has morphological similarity according to the histologic findings regardless of the location. The synthetic medical image is generated based on the sample medical image, and may be generated under a specific condition. Hereinafter, conditions and methods for generating a synthetic medical image will be described in detail with reference to FIGS. 7 to 8 .

2 is a diagram illustrating an example of a target medical image.

1 , the target medical image may be a CT image or an MRI image.

2A is a CT image of a skull of a patient with Graves' disease, an endocrine orbitopathy. And FIG. 2B is a CT image of the skull of a patient with retrobulbar cavernous hemangioma in the left eye. And FIG. 2c is an MRI image of the skull of a patient with an optic sheath meningioma in the orbital apex of the right eye. And FIG. 2d is a CT image of the skull of a patient with non-Hodgkin lymphoma (NHL) at the apex of the orbit of the left eye.

3 is a diagram illustrating an example of a classification result for an orbital structure in a target medical image.

Referring to FIG. 3 , the structure classification unit 120 includes a corneal apex, a crystalline lens, a vitreous, an extraocular muscle, and an optic nerve located in the orbit from the target medical images a and b. It is possible to classify orbital structures such as optic nerve) and orbit apex, and to mask each orbital structure.

4 is a block diagram illustrating another example of an orbital and peripheral lesion prediction apparatus.

Referring to FIG. 4 , the orbital and orbital lesion prediction apparatus 100 includes the input unit 110 , the structure classification unit 120 , the estimator 130 , the lesion prediction unit 140 , and the output unit ( 150) and the model learning unit 160 may further include a preprocessing unit 170.

The preprocessor 170 is configured to learn the first deep learning model (MDL_1), the second deep learning model (MDL_2), and the third deep learning model (MDL_3) described above in FIG. 1 based on the sample medical image and the biopsy information. It is possible to generate preprocessing data used for

5 is a diagram illustrating an example of generating preprocessing data.

Referring to FIG. 5 , the preprocessor 170 may add masking information on a structure located in the orbit to the sample medical image. In this case, the masking information may be added by an automated tool or may be manually added by a medical staff.

And the preprocessor 170 may encode the tissue examination information based on the set mapping table.

The preprocessor 170 may generate preprocessing data including masking data obtained by adding the above-described masking information to a sample medical image or encoded data obtained by encoding biopsy information.

Meanwhile, the pre-processing unit 170 may additionally perform an operation of deleting personal information of a patient in the process of generating the aforementioned pre-processing data. The process of deleting a patient's personal information may be performed by an automated tool or may be performed manually by a medical staff.

6 is a block diagram illustrating another example of an orbital and peripheral lesion prediction apparatus.

Referring to FIG. 6 , the orbital and peripheral lesion prediction apparatus 100 includes the input unit 110 , the structure classification unit 120 , the estimator 130 , the lesion prediction unit 140 described above in FIGS. 1 and 4 , In addition to the output unit 150 , the model learning unit 160 , and the preprocessing unit 170 , the synthetic medical image generation unit 180 may be additionally included.

The synthetic medical image generator 180 generates a first deep learning model (MDL_1), a second deep learning model (MDL_2), and a third deep learning model (MDL_3) based on the preprocessing data generated by the preprocessor 170 . It is possible to create a synthetic medical image for learning.

The reason why the synthetic medical image generating unit 180 generates the synthetic medical image is the number of normal sample medical images, that is, the number of sample medical images in which no lesion has occurred in the captured orbital structure, and the number of abnormal samples in the process of learning the deep learning model. This is to prevent a problem in which the accuracy of the deep learning model is lowered due to an imbalance in the number of medical images, that is, sample medical images in which lesions occur in the captured orbital structures.

In particular, among the above-described sample medical images, since the number of abnormal sample medical images is highly likely to be significantly smaller than the number of normal sample medical images, the deep learning model trained based on the sample medical image has lower prediction accuracy for abnormal cases. likely to be low. Therefore, the synthetic medical image generating unit 180 generates an abnormal synthetic medical image, that is, a synthetic medical image including an image of a structure with a lesion, when the number of abnormal sample medical images is significantly less than the number of normal sample medical images. It is possible to increase the prediction accuracy for the case where the deep learning model is abnormal. In addition, since synthetic medical image information does not include the patient's personal information, there is no need to take measures to protect personal information under the Personal Information Protection Act.

7 is a flowchart illustrating an example of an operation of determining whether to generate a synthetic medical image.

Referring to FIG. 7 , the synthetic medical image generator 180 may calculate the number of normal sample medical images, that is, the number A of sample medical images in which the orbital structure in which the lesion does not occur is captured ( S710 ).

In addition, the synthetic medical image generator 180 may calculate the number of abnormal sample medical images, that is, the number B of sample medical images in which the orbital structure in which the lesion does not occur is captured ( S720 ). Meanwhile, in FIG. 7 , the case where step S720 is executed after step S710 has been described as an example, but step S720 may be executed before or simultaneously with step S710.

The synthetic medical image generator 180 determines whether the ratio of the number of abnormal sample medical images to the number of normal sample medical images, that is, (B/A) is less than or equal to a set threshold ratio THR ( S730 ). In this case, the value of the threshold ratio (THR) may be determined based on the accuracy of the learned first deep learning model (MDL_1), the second deep learning model (MDL_2), and the third deep learning model (MDL_3) as an example. For example, the value of the threshold ratio THR may be 1/50 or 1/20.

If (B/A) is equal to or less than the set threshold ratio (THR) (S730-Y), this means that the number of abnormal sample medical images is significantly smaller than that of normal sample medical images. Accordingly, the synthetic medical image generator 180 may generate a synthetic medical image based on the sample medical image ( S740 ).

On the other hand, when (B/A) exceeds the set threshold ratio THR ( S730-N ), this means that the number of abnormal sample medical images is sufficient compared to normal sample medical images. Accordingly, the synthetic medical image generator 180 may not generate a synthetic medical image (S750).

8 is a diagram illustrating an example of generating a synthetic medical image.

The synthetic medical image generator 180 may generate training data by cropping a region in which a lesion occurs from an abnormal sample medical image included in the preprocessing data. Whether the sample medical image included in the preprocessing data is abnormal may be confirmed through biopsy information corresponding to the sample medical image.

In this case, the cropped region may be selected by the aforementioned first deep learning model MDL_1.

The synthetic medical image generator 180 may generate synthetic data corresponding to the cropped region by inputting the cropped training data to the fourth deep learning model. In this case, the fourth deep learning model may be, for example, a Generative Adversarial Network (GAN) model.

The synthetic medical image generator 180 may generate a synthetic medical image by inserting the above-described synthetic data into a normal sample medical image. In this case, a normal sample medical image into which the synthetic data is inserted may be randomly selected.

9 is a flowchart illustrating an example of a method for predicting orbital and periorbital lesions.

The orbital and peripheral lesion prediction method 900 may include an input step S910 , a structure classification step S920 , an estimation step S930 , a lesion prediction step S940 , and an output step S950 .

In the input step ( S910 ), a target medical image obtained by photographing a patient's skull may be input. In this case, the target medical image may be a CT image or an MRI image.

The structure classification step S920 may generate a classification result for the orbital structure by inputting the target medical image received in the input step S910 into the first deep learning model. In this case, for example, the orbital structure is an optic nerve, an extraocular muscle, a crystalline lens, a corneal apex, an orbit apex, a lacrimal gland, or a vitreous. can

In the estimation step (S930), the classification result generated in the structure classification step (S920) may be input to the second deep learning model to generate an estimation result as to whether a lesion has occurred in the orbital structure. In this case, the lesion may be a fracture, an inflammatory disease, or a tumor.

In the lesion prediction step (S940), according to the estimation result generated in the estimation step (S930), the classification result generated in the structure classification step (S920) is input to the third deep learning model to determine the location and type of the lesion generated in the orbital structure. prediction results can be generated.

In addition, the output step S950 may output the above-described estimation result and prediction result.

Meanwhile, the first deep learning model, the second deep learning model, and the third deep learning model described above are synthetic medical images generated based on a sample medical image, biopsy information corresponding to the sample medical image, and sample medical image and biopsy information. It may be learned based on one or more of the images.

The preprocessing data for learning the first deep learning model, the second deep learning model, and the third deep learning model may be generated by preprocessing the above-described sample medical image and biopsy information. For example, the preprocessing data may include masking data obtained by adding masking information on a structure located in the orbit to the sample medical image. As another example, the preprocessing data may include encoded data encoded based on a mapping table in which tissue examination information is set.

The aforementioned synthetic medical image may be generated based on the aforementioned pre-processing data. For example, the synthetic medical image may be generated by inputting preprocessing data into the fourth deep learning model when the number of sample medical images in which lesions are generated compared to the number of sample medical images in which lesions are not generated among the aforementioned sample medical images is less than or equal to a set threshold ratio. can

Meanwhile, the above-described orbital and perorbital lesion predicting method 900 may be executed by the orbital and perorbital lesion predicting apparatus 100 described with reference to FIGS. 1, 4 or 6 .

The orbital and peripheral lesion prediction apparatus 100 and the orbital and peripheral lesion prediction method 900 described in the embodiments of the present invention apply artificial intelligence, particularly deep learning, to enable rapid decision-making of medical staff. can support By applying artificial intelligence to medical images obtained from various imaging devices, it will be possible to provide high-accuracy analysis results to medical staff, helping to quickly and accurately predict the location and type of lesions around the patient's orbit. It is expected. In addition, by predicting the location and termination of the lesion generated in the orbit only with the target medical image, the location and type of the lesion occurring at a location where a biopsy is impossible can be predicted.

A deep learning model such as the above-described first deep learning model (MDL_1), second deep learning model (MDL_2), third deep learning model (MDL_3), and fourth deep learning model (MDL_4) is an artificial neural network may be a model in which the . That is, the deep learning model automatically learns features of the input value by learning a large amount of data from a deep neural network consisting of a multi-layered network, and through this, the error in the objective function, that is, the prediction accuracy, is eliminated. It is a type of model that trains the network to minimize it.

The deep learning model may be a Convolutional Neural Network (CNN), Deep Hierachical Network (DHN), Convolutional Deep Belief Network (CDBN), Deconvolutional Deep Network (DDN), Recurrent Neural Network (RNN), Generative Adversarial Network (GAN), etc. The present invention is not limited thereto, and various deep learning models that can be used now or in the future can be used.

Deep learning models can be implemented through deep learning frameworks. The deep learning framework provides functions commonly used when developing a deep learning model in the form of a library and plays a role in supporting the good use of system software or hardware platforms. In this embodiment, the deep learning model may be implemented using any deep learning framework that is currently published or will be published in the future.

The orbital and peripheral orbital lesion prediction apparatus 100 may be implemented by a computing device including at least some of a processor, a memory, a user input device, and a presentation device. A memory is a medium that stores computer-readable software, applications, program modules, routines, instructions, and/or data, etc. coded to perform specific tasks when executed by a processor. The processor may read and execute computer-readable software, applications, program modules, routines, instructions, and/or data stored in the memory and/or the like. The user input device may be a means for allowing the user to input a command to the processor to execute a specific task or to input data required for the execution of the specific task. The user input device may include a physical or virtual keyboard or keypad, key button, mouse, joystick, trackball, touch-sensitive input means, or a microphone. The presentation device may include a display, a printer, a speaker, or a vibrator.

Computing devices may include various devices such as smartphones, tablets, laptops, desktops, servers, clients, and the like. The computing device may be a single stand-alone device, or may include a plurality of computing devices operating in a distributed environment comprising a plurality of computing devices cooperating with each other through a communication network.

In addition, the above-described orbital and peripheral lesion prediction method includes a processor, and when executed by the processor, computer-readable software, applications, program modules, routines coded to perform an imaging method using a deep learning model , instructions, and/or data structures may be executed by a computing device having a memory.

The above-described embodiments may be implemented through various means. For example, the present embodiments may be implemented by hardware, firmware, software, or a combination thereof.

In the case of hardware implementation, the image diagnosis method using the deep learning model according to the present embodiments includes one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), It may be implemented by Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), processors, controllers, microcontrollers or microprocessors, and the like.

For example, the orbital and orbital lesion prediction method according to the embodiments may be implemented using an artificial intelligence semiconductor device in which neurons and synapses of a deep neural network are implemented with semiconductor elements. In this case, the semiconductor device may be currently used semiconductor devices, for example, SRAM, DRAM, NAND, or the like, or may be next-generation semiconductor devices, RRAM, STT MRAM, PRAM, or the like, or a combination thereof.

When the orbital and peripheral lesion prediction method according to the embodiments is implemented using an artificial intelligence semiconductor device, the result (weight) of learning the deep learning model with software is transferred to a synaptic mimic device arranged in an array, or an artificial intelligence semiconductor device You can also proceed with learning.

In the case of implementation by firmware or software, the orbital and periorbital lesion prediction method according to the present embodiments may be implemented in the form of an apparatus, procedure, or function that performs the functions or operations described above. The software code may be stored in the memory unit and driven by the processor. The memory unit may be located inside or outside the processor, and may exchange data with the processor by various known means.

Also, as described above, terms such as "system", "processor", "controller", "component", "module", "interface", "model", or "unit" generally refer to computer-related entities hardware, hardware and software. may mean a combination of, software, or running software. For example, the aforementioned component may be, but is not limited to being, a process run by a processor, a processor, a controller, a controlling processor, an object, a thread of execution, a program, and/or a computer. For example, both an application running on a controller or processor and a controller or processor can be a component. One or more components may reside within a process and/or thread of execution, and the components may be located on one device (eg, a system, computing device, etc.) or distributed across two or more devices.

On the other hand, another embodiment provides a computer program stored in a computer recording medium for performing the above-described method for predicting orbital and peripheral lesions. In addition, another embodiment provides a computer-readable recording medium in which a program for realizing the above-described orbital and peripheral lesion prediction method is recorded.

The program recorded on the recording medium can be read by a computer, installed, and executed to execute the above-described steps.

In this way, in order for the computer to read the program recorded on the recording medium and execute the functions implemented as the program, the above-described program is C, C++ that the computer's processor (CPU) can read through the computer's device interface (Interface). , JAVA, python, R, may include code coded in a computer language such as machine language.

Such code may include a function code related to a function defining the above-mentioned functions, etc., and may include an execution procedure related control code necessary for the processor of the computer to execute the above-mentioned functions according to a predetermined procedure.

In addition, this code may further include additional information necessary for the processor of the computer to execute the above functions or code related to memory reference for which location (address address) in the internal or external memory of the computer should be referenced. .

In addition, when the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the functions described above, the code can be executed by the processor of the computer using the communication module of the computer. It may further include a communication-related code for how to communicate with other computers or servers, and what information or media to transmit and receive during communication.

The computer-readable recording medium in which the program as described above is recorded is, for example, ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical media storage device, etc., and also carrier wave (eg, , transmission over the Internet) may be implemented in the form of.

In addition, the computer-readable recording medium is distributed in a computer system connected through a network, so that the computer-readable code can be stored and executed in a distributed manner.

And, in consideration of the system environment of a computer that reads a recording medium and executes a program by reading a recording medium, a functional program and related codes and code segments for implementing the present invention, programmers in the technical field to which the present invention belongs may be easily inferred or changed by

The orbital and peripheral orbital lesion prediction method described with reference to FIG. 10 may also be implemented in the form of a recording medium including instructions executable by a computer, such as an application or program module executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer-readable media may include any computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

The orbital and peripheral lesion prediction method described above may be executed by an application basically installed in the terminal (which may include a program included in a platform or operating system basically mounted in the terminal), and the user may use an application store server, It may be executed by an application (ie, a program) installed directly on the master terminal through an application providing server such as an application or a web server related to the corresponding service. In this sense, the above-described orbital and orbital lesion prediction method is implemented as an application (ie, program) installed by default in the terminal or directly installed by the user, and may be recorded in a computer-readable recording medium such as a terminal. .

The above description of the present invention is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and likewise components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

Claims (21)

an input unit for receiving a target medical image obtained by photographing a patient's skull;
a structure classification unit that inputs the target medical image to a first deep learning model and generates a classification result for the orbital structure of the patient;
an estimator for inputting the classification result into a second deep learning model to generate an estimation result as to whether a lesion has occurred in the orbital structure;
a lesion predictor configured to input the classification result into a third deep learning model according to the estimation result and generate a prediction result for a location and type of a lesion occurring in the orbital structure;
an output unit for outputting the estimation result and the prediction result; and
The first deep learning model and the second deep learning model based on at least one of a sample medical image, biopsy information corresponding to the sample medical image, and a synthetic medical image generated based on the sample medical image and the biopsy information And orbital and peripheral lesion prediction device comprising a model learning unit for learning the third deep learning model.
According to claim 1,
The target medical image is a CT image or an MRI image, and an orbital and peripheral lesion prediction device.
According to claim 1,
The orbital structures include orbital walls, fissures, canals, orbital periorbita, optic nerve, extraocular muscle, orbital fat, nerve. , vasculature, annulus of Zinn, intraorbital tissue, crystalline lens, corneal apex, orbit apex, lacrimal gland, A device for predicting orbital and periorbital lesions that are vitreous or nasal and paranasal sinus.
According to claim 1,
The lesion occurring in the orbital structure is an orbital and peripheral lesion prediction device that is a fracture, inflammation, or tumor.
According to claim 1,
Orbital and peripheral lesions further comprising a preprocessor for generating preprocessing data for learning the first deep learning model, the second deep learning model, and the third deep learning model based on the sample medical image and the biopsy information prediction device.
6. The method of claim 5,
The preprocessor is
An orbital and periorbital lesion prediction apparatus for adding masking data obtained by adding masking information on structures around the orbit to the sample medical image to the pre-processing data.
6. The method of claim 5,
The preprocessor is
An orbital and peripheral lesion prediction apparatus for adding encoding data encoded based on the mapping table setting the biopsy information to the pre-processing data.
6. The method of claim 5,
The orbital and peripheral lesion prediction apparatus further comprising a synthetic medical image generator generating the synthetic medical image based on the pre-processing data.
9. The method of claim 8,
The synthetic medical image generation unit,
orbit generating the synthetic medical image when a ratio of the number of sample medical images in which lesioned orbital structures are photographed to the number of sample medical images in which lesioned orbital structures are photographed in the sample medical images is less than or equal to a set threshold ratio and a device for predicting periorbital lesions.
10. The method of claim 9,
The synthetic medical image generation unit,
An orbital and peripheral lesion prediction apparatus for generating the synthetic medical image by inputting the pre-processing data into a fourth deep learning model.
an input step of receiving a target medical image of a patient's skull;
a structure classification step of generating a classification result for the orbital structure of the patient by inputting the target medical image into a first deep learning model;
an estimation step of inputting the classification result into a second deep learning model to generate an estimation result as to whether a lesion has occurred in the orbital structure;
a lesion prediction step of inputting the classification result into a third deep learning model according to the estimation result to generate a prediction result for the location and type of the lesion occurring in the orbital structure; and
An output step of outputting the estimation result and the prediction result,
The first deep learning model, the second deep learning model, and the third deep learning model may include a sample medical image, biopsy information corresponding to the sample medical image, and a synthetic medical image generated based on the sample medical image and biopsy information. An orbital and periorbital lesion prediction method that is learned based on one or more of.
12. The method of claim 11,
The target medical image is a CT image or an MRI image, and the orbital and peripheral lesion prediction method.
12. The method of claim 11,
The orbital structures include orbital walls, fissures, canals, orbital periorbita, optic nerve, extraocular muscle, orbital fat, nerve. , vasculature, annulus of Zinn, intraorbital tissue, crystalline lens, corneal apex, orbit apex, lacrimal gland, A method for predicting orbital and periorbital lesions that are vitreous or nasal and paranasal sinus.
12. The method of claim 11,
The lesion occurring in the orbital structure is a fracture, inflammation, or tumor in the orbit and surrounding orbital lesion prediction method.
12. The method of claim 11,
Pre-processing data for learning the first deep learning model, the second deep learning model, and the third deep learning model is an orbital and peripheral lesion prediction method that is generated based on the sample medical image and the biopsy information.
16. The method of claim 15,
The preprocessed data is
The orbital and orbital lesion prediction method including masking data in which masking information on structures around the orbit is added to the sample medical image.
16. The method of claim 15,
The preprocessed data is
An orbital and peripheral lesion prediction method comprising encoding data generated by encoding the biopsy information based on a set mapping table.
16. The method of claim 15,
The synthetic medical image is an orbital and periorbital lesion prediction method that is generated based on the pre-processing data.
19. The method of claim 18,
When the ratio of the number of sample medical images in which lesion-occurring orbital structures are photographed to the number of sample medical images in which lesion-free orbital structures are photographed in the sample medical images is less than or equal to a set threshold ratio, the orbit in which the synthetic medical images are generated and a method for predicting periorbital lesions.
20. The method of claim 19,
The synthetic medical image is
An orbital and peripheral lesion prediction method generated by inputting the pre-processing data into a fourth deep learning model.
A computer-readable recording medium in which a program for implementing the method for predicting orbital and orbital lesions according to any one of claims 11 to 20 is recorded.

Images