KR101848322B1 - Method for facilitating generation of finding and dignosis data of subject based on fovea image thereof and apparatus using the same - Google Patents

Abstract

The present invention relates to a method for supporting reading of a fundus image of a subject and a computing apparatus using the same. More specifically, the computing apparatus acquires a fundus image of a subject, generates or receives information on whether the fundus image is normal, generates segment information which indicates an individual segment of the fundus image, generates or receives finding information corresponding to the individual segment, generates or receives diagnostic information on the fundus image based on the finding information, and stores the finding and diagnostic information on the fundus image or provides the finding and diagnostic information on the fundus image to an external entity. Therefore, the method provides convenience of reading the fundus image.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for reading an image of a fundus of a subject,

The present invention relates to a method of supporting funduscopic image reading on a subject and a computing device using the same. Specifically, the computing apparatus according to the present invention may further include: an acquisition unit configured to acquire a fundus image of the subject, generate or input normal information on the fundus image, generate segmentation information indicating an individual segment of the fundus image, And generating or inputting diagnostic information corresponding to the funduscopic image based on the finding information, storing the diagnostic information and diagnostic information on the funduscopic image or storing the diagnostic information corresponding to an external entity entity.

Fundus images are frequently used in ophthalmology because they can detect abnormalities of the retina, optic nerve, and macula. However, the conventional labeling system provided for reading the fundus image was not effective or effective in several aspects.

First, the conventional labeling system does not take into account the logical relationship between the selection items, and therefore, unnecessary or unreasonable labeling is sometimes performed. Specifically, there was a case where an exclusive relationship between diseases was not taken into consideration, allowing pathologically impossible labeling.

In addition, the conventional labeling system has a disadvantage in that efficient data management is difficult due to the combination of finding and diagnosis, and the compartment setting for displaying the position corresponding to the findings does not reflect the structural characteristics of the eye.

Therefore, in the present invention, a funduscopic image reading support method and a device using the funduscopic image reading apparatus which solve such problems and provide medical staff with more efficient and accurate labeling of the fundus image are proposed.

Non-Patent Document 1: Goodfellow, Ian J .; Pouget-Abadie, Jean; Mirza, Mehdi; Xu, Bing; Warde-Farley, David; Ozair, Sherjil; Courville, Aaron; Bengio, Yoshua (2014). "Generative Adversarial Networks" Non-Patent Document 2: Ronneberger O., Fischer P., Brox T. (2015) U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab N., Hornegger J., Wells W., Frangi A. (eds) Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015. Lecture Notes in Computer Science, vol 9351. Springer, Cham

An object of the present invention is to provide convenience of funduscopic image reading by providing regions to be divided by reflecting anatomical characteristics using artificial intelligence.

In addition, the present invention includes not only the logically relevant items but also the inclusion of the logically appropriate items in the labeling system, thereby reflecting the exclusive relationship of the disease to the labeling item, and also allowing the data to be sorted efficiently by distinguishing between the findings and diagnosis do.

The present invention is intended to facilitate segmentation by automatically detecting the position of the macula and the optic nerve head in consideration of the positional characteristics of the main findings and the structural characteristics of the eye.

The characteristic configuration of the present invention for achieving the object of the present invention as described above and realizing the characteristic effects of the present invention described below is as follows.

According to an aspect of the present invention, there is provided a method of supporting reading of funduscopic images of a subject, the method comprising: (a) the computing device acquiring fundus images of the subject or interfacing To obtain another device to be acquired; (b) a process in which the computing device generates (i) generates information on whether or not the fundus image is normal or generates the other device, and (ii) receives the normal information from at least one user, Performing at least one of the following processes: (c) the computing device generates partition information indicating an individual partition of the funduscopic image or supports the other device to generate the information; (i) generating the finding information corresponding to the individual partition or (Ii) performing the at least one of receiving the finding information from the at least one user or supporting the other device to receive input; (d) the computing device is configured to: (i) generate diagnostic information for the fundus image or generate the other device based on the finding information; and (ii) And a process of receiving input of the diagnostic information corresponding to the finding information or supporting the other device to receive input; And (e) supporting the computing device to store or provide the diagnostic information and the diagnostic information on the fundus image to an external entity or to store or provide the external device.

According to another aspect of the present invention, there is also provided a computer program stored in a machine readable non-transitory medium, comprising instructions embodied to perform the method according to the invention.

According to still another aspect of the present invention, there is provided a computing device that supports reading funduscopic images of a subject, the computing device including: a communication unit for implementing an input module for acquiring fundus images of the subject; And (1-i) generating normal information on the funduscopic image or generating other apparatuses to be interlocked through the communication unit, and (1-ii) transmitting, via at least one user, And a process of supporting at least one of the processes of receiving input from the other device or receiving input from the other device, wherein the discrimination module generates segmentation information indicating an individual segment of the funduscopic image, (2-i) a process of generating or generating (2-ii) finding information corresponding to the individual section or creating the other device; and (2-ii) Receiving the finding information from one user or receiving the other information from the other device A feature input module for performing at least one of the processes for supporting the process; (3-i) a process of generating diagnosis information for the funduscopic image or generating the diagnostic information for the other device based on the finding information; and (3-ii) a process of generating diagnostic information for the funduscopic image from the at least one user And a process of inputting the diagnosis information corresponding to the finding information or supporting the other device to receive input, And a storage and transmission module for storing the diagnostic information and diagnostic information on the funduscopic image or providing the stored or transmitted information to an external entity or storing or providing the other device through the communication unit.

According to the present invention, unnecessary or unreasonable reading can be prevented logically and pathologically as in the conventional fundus image labeling method, and the position corresponding to the finding can be effectively displayed due to the partition setting considering the structural characteristics of the eyeball It is effective.

Further, according to the present invention, it is possible to organize data efficiently, to easily acquire data for an artificial intelligence learning model, to acquire initial attribution information that can be easily used for reading through previously learned information, Information on the position of the macula and the optic nerve head, information on the blood vessel, information on the cup-to-disk ratio, etc. can be provided.

Accordingly, the present invention has the effect of eliminating logical and pathologically impossible readings from the system, thereby eliminating the room for the medical staff to make a mistake, and as the user's experience is accumulated, the learning model is continuously updated, There is an advantage that it can be raised.

In short, by enabling efficient and accurate reading, it is possible to save the time of the medical staff, improve the quality of the medical care, and innovate the workflow in the medical field.

According to the present invention, it is possible to utilize funduscopy images conventionally used in hospitals, such as fundus photographing devices and images acquired from a medical image storage transmission system (PACS), as the method of the present invention is applicable to computer operating systems It does not.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention to those skilled in the art Other drawings can be obtained based on these figures without an inventive task being performed.
1 is a diagram showing a main concept of a CNN (convolutional neural network) as an example of a machine learning model that can be used in the present invention.
2 is a conceptual diagram schematically illustrating an exemplary configuration of a computing device that performs a method of supporting reading of funduscopic images of a subject according to the present invention (hereinafter referred to as " funduscopic image reading support method ").
3 is a conceptual diagram illustrating a hardware and software architecture of a computing device that performs a funduscopic image reading support method according to the present invention.
4 is a diagram illustrating blood vessel information obtained by separating only blood vessel images in an embodiment of the funduscopic image reading support method according to the present invention.
5 is a diagram illustrating a conventional method for detecting the position of the optic disc and macula center.
FIG. 6 is a diagram illustrating a machine learning model for detecting the position of the optic disc center and the macula center in an embodiment of the funduscopic image read support method according to the present invention.
FIG. 7 is a diagram illustrating an example of segmentation information indicating a segment of an eye fundus image extracted in an embodiment of the funduscopic image reading support method according to the present invention.
FIG. 8 is a flowchart exemplarily showing an embodiment of a funduscopic image reading support method according to the present invention.
9 is a flowchart exemplarily showing another embodiment of a funduscopic image reading support method according to the present invention.
FIGS. 10A through 10E are exemplary views illustrating a user interface (UI) provided at each step of the embodiment illustrated in FIG.
FIG. 11 is a diagram showing a Bayesian network as one of methods for generating diagnostic information from feature information in the embodiment illustrated in FIG.

The following detailed description of the invention refers to the accompanying drawings, which illustrate, by way of example, specific embodiments in which the invention may be practiced in order to clarify the objects, technical solutions and advantages of the invention. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention.

Throughout the detailed description and claims of the present invention, 'learning' or 'learning' refers to performing machine learning through computing according to a procedure, It will be understood by those of ordinary skill in the art that the present invention is not intended to be so-called.

And throughout the description and claims of this invention, the word 'comprise' and variations thereof are not intended to exclude other technical features, additions, elements or steps. Other objects, advantages and features of the present invention will become apparent to those skilled in the art from this description, and in part from the practice of the invention. The following examples and figures are provided by way of illustration and are not intended to limit the invention.

Moreover, the present invention encompasses all possible combinations of embodiments shown herein. It should be understood that the various embodiments of the present invention are different, but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with one embodiment. It should also be understood that the position or arrangement of individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the present invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is to be limited only by the appended claims, along with the full scope of equivalents to which such claims are entitled, if properly explained. In the drawings, like reference numerals refer to the same or similar functions throughout the several views.

Unless otherwise indicated herein or clearly contradicted by context, items referred to in the singular are intended to encompass a plurality unless otherwise specified in the context. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, so that those skilled in the art can easily carry out the present invention.

1 is a diagram showing a main concept of CNN as an example of a machine learning model used in the present invention.

Referring to FIG. 1, among the machine learning models used in the present invention, a deep neural network model can be briefly described as a form in which artificial neural networks are stacked in multiple layers. That is, it is expressed as a deep neural network or a deep neural network in the sense of a network of deep structure. As shown in FIG. 1, by learning a large amount of data in a structure of a multi-layer network, , And automatically learns the relationship between the biological signals and thereby learns the network by a method that minimizes the error of the objective function, that is, the prediction accuracy of the fatal symptom. This is also expressed as a connection between neurons in the human brain, and such a deep neural network is becoming a next generation model of artificial intelligence.

Next, FIG. 2 is a conceptual diagram schematically showing an exemplary configuration of a computing device that performs a funduscopic image reading support method according to the present invention.

2, a computing device 200 according to an embodiment of the present invention includes a communication unit 210 and a processor 220. The communication unit 210 communicates with an external computing device (not shown) Communication is possible.

In particular, the computing device 200 may be a computing device, such as a computer, a processor, a memory, a storage, an input and output device, or any other device capable of including components of a conventional computing device; Electronic communication devices, electronic information storage systems such as Network Attached Storage (NAS) and Storage Area Networks (SAN)) and computer software (i. E., Instructions that cause a computing device to function in a particular manner) System performance.

The communication unit 210 of such a computing device can send and receive requests and responses to and from other interworking computing devices. As an example, such requests and responses can be made by the same TCP session, For example, as a UDP datagram. In addition, in a broad sense, the communication unit 210 may include a keyboard, a mouse, and other external input devices for receiving commands or instructions.

The processor 220 of the computing device may include a hardware configuration such as a micro processing unit (MPU) or a central processing unit (CPU), a cache memory, and a data bus. It may further include a software configuration of an operating system and an application that performs a specific purpose.

3 is a conceptual diagram illustrating a hardware and software architecture of a computing device that performs a funduscopic image reading support method according to the present invention.

3, the computing device 200 may include an image acquisition module 310 as a component of the method and apparatus according to the present invention. It is to be understood that the image acquisition module 310 may be implemented by the communication unit 210 included in the computing device 200 or by the interlocking of the communication unit 210 and the processor 220, will be.

The image acquisition module 310 can acquire fundus images of the subject. For example, it may be obtained from a funduscopic imaging device or a medical image storage transmission system (PACS) of the subject, but is not limited thereto. Since most fundus images are intermediate images, 7-segment montage of diabetic patients may be possible.

Next, the obtained fundus image can be transmitted to the determination module 320, and the determination module 320 extracts the fundus image attribute information based on the machine learning model.

For example, the attribute information of the funduscopic image includes (i) blood vessel information showing only the image of the optic nerve blood vessel included in the fundus image, (ii) binocular discrimination information indicating whether the funduscopic image is the left eye image or the right eye image (iv) position information indicating the position of at least one of the macula and the optic nerve head contained in the fundus image, (iv) segment information indicating a segment of the fundus image, (v) C / D ratio (cup-to-disk ratio) information, and the like.

First, in the case of the blood vessel information, the machine learning model of the discrimination module may include a synthetic neural network and generic adversarial networks (GAN).

Non-patent document 1: [Goodfellow, Ian J .; Pouget-Abadie, Jean; Mirza, Mehdi; Xu, Bing; Warde-Farley, David; Ozair, Sherjil; Courville, Aaron; Bengio, Yoshua (2014). According to "Generative Adversarial Networks", the GAN generator is aimed at tricking the discriminator into generating realistic data to identify the similar data as actual data, and the discriminator separates the actual data from the generated similar data I have a goal to put out. In the process of learning by this GAN, the generator and the discriminator update the network weights to achieve the respective goals. After sufficient learning, the generator generates data similar to the actual data, and the discrimination rate by the discriminator is theoretical And converges to 0.5 at. As a result, the generator fully learned by the GAN generates data close to the actual data, so that the data unbalance problem in machine learning can be solved.

FIG. 4 is a block diagram showing the structure of the present invention. FIG. 4 is a block diagram showing the structure of the present invention. In FIG. 4, FIG. 2 is a view illustrating an example of blood vessel information obtained by separating only an image of a blood vessel according to an embodiment of the funduscopic image reading support method according to the present invention.

Second, in the case of the binocular discrimination information, the machine learning model includes segmentation networks, and based on the machine learning model, the shape of the optic disk of the fundus image and the shape of the optic disc By discriminating at least one of running with blood, binocular discrimination information of the funduscopic image can be extracted according to the shape of the optic nerve head and the running of the blood vessel.

Specifically, in the extraction of the binocular discrimination information, it may be impossible to determine whether the fundus image is the left eye image or the right eye image only by the position of the optic nerve head. In order to improve the accuracy of discrimination, The learning of the machine learning model is carried out using the funduscopic image in the case where judgment is impossible using only the position of the camera. In the case of the right eye, the optic nipple is on the right side because the image normally taken in the ophthalmology is focused on the center, but the learning data includes the image in which the optic nipple is located on the left or lower side, Can be increased.

The z-score (standardization score) of an individual channel of the fundus image is obtained and used as an input of the synthesis neural network. The z-score is expressed by the following Equation 1.

By using this z-score, it is possible to distinguish the binocular using the type information shown in the image without being influenced by the intensity or contrast of the funduscopic image. An exemplary form of fraction network that can be used here is as follows.

conv-conv-maxpool (2) -conv-conv-maxpool (2) -conv-conv-maxpool (2) conv-dense (256) -dense (1)

Here, the input data is an RGB image converted to a z-score of 512x512, and the stride of conv and maxpool is represented by (x). Here, the activation function is selu, the last dense layer is sigmoid, and the loss function is binary cross entropy.

Thirdly, in the case of the position information, the machine learning model includes a fraction network, and based on this, the position of at least one of the macula and the optic nerve head can be extracted.

FIG. 5 is a view for explaining a conventional method for detecting the position of the optic disc and macula center. The optic disc and the macula function as a reference point in reading the fundus image. Conventionally, the position of the macula is measured by optical coherence tomography ). The optic nerve head was detected from the fundus image because the optic nerve head was bright and clear in the fundus image and its position was relatively easy to determine. Unlike the conventional technique, the present invention can detect both the position of the macula and the optic nerve head from the fundus image without using the OCT image.

FIG. 6 is a diagram illustrating a machine learning model for detecting the position of the optic disc center and the macula center in an embodiment of the funduscopic image read support method according to the present invention.

Specifically, in the detection of the optic nerve head and macular center position, not only a normal fundus image but also an eye fundus image having a lesion is labeled by a medical professional (more specifically, the position coordinate is captured) As an example of a machine learning model, learning of a machine learning model has been progressed. [Non-Patent Document 2] [Ronneberger O., Fischer P., Brox T. (2015) U-Net: Convolutional Networks for Biomedical Image Segmentation. Padding < / RTI > as disclosed in Springer, Cham, in < RTI ID = 0.0 > the depth concatenation technique used in U-Net may be utilized as parameters of the type of padding, the number of layers, and the depth of individual layers. The concrete network configuration is as illustrated in Fig.

In this network illustrated in FIG. 6, a z-score is obtained for each channel of 640x640 RGB image, and a segmented vessel is used as an input. Batch-normalization is used, and a loss function is binary cross entropy was used. In FIG. 6, operations represent operations of individual blocks, skip-connected represents a depth concat layer in U-Net, and upsample means bilinear interpolation to increase the length and width twice in each case.

Fourthly, in the case of the segment classification information, the machine learning model includes a fraction network, and based on this, the position of the macula and the optic nerve head are extracted based on which the center position of the macula and the center position of the optic nerve head The segmentation information that is information for dividing the funduscopic image into individual segments can be extracted based on the segmentation information. Preferably, the individual compartments are comprised of three or more, and may be configured in various ways according to the needs of the medical staff.

In one embodiment for distinguishing individual compartments, the individual compartments may include a macula region, a superior disc region, an inferior disc region, a temporal region, a superotemporal region, Region, an inferotemporal region, a superonasal region, and an inferonasal region.

Fig. 7 is a diagram exemplifying the segmentation information indicating the segment of the fundus image extracted in this embodiment.

7, when the distance between the central position of the macula and the center position of the optic nerve head is d, k1 * d is a radius of a predetermined k1 satisfying 0 < k1 &lt; = 1, And may be the inner region of the macular center circle centered at the central position of the macula.

In addition, the above-mentioned optic papilloterminal region may have a radius k2 * d for a predetermined k2 satisfying the distance d and 0 < k2 &lt; = 1 and an inner region of the optic nerve head center circle having the center position of the optic nerve head as a center The optic disc region may be the upper one of the regions divided by the straight line l1 passing through the central position of the macula and the center position of the optic disc.

And the lower optic disc region may be a lower region of the optic disc region divided by the straight line l1.

In addition, the above-mentioned upper-side region is defined by a straight line l2 passing through the center position of the optic nerve head perpendicular to the above-mentioned straight line l1, and an area above the optic papillary region in a region farther from the macula among the regions divided by the straight line l1 May be excluded.

Next, the rabbit-side region may be a region excluding the lower region of the optic disc in the region far from the macula among the regions divided by the straight line l2 and the straight line l1.

In addition, the outer region may be divided into two rectilinear l3 and l4 parallel to the straight line l1 and extending in a direction away from the optic nerve head, contacting the center of the macular center, and the straight line l1 The region excluding the macular region may be the region including the macular region.

The upper-side region may be a region excluding the upper side of the optic disc region, the macula region, and the outer region in an upper-side region of the macular region, which is divided by the straight line l2 and the straight line l1.

In addition, the lower outer region may be a region excluding the lower optic disc region, the macular region, and the outer region in the lower region near the macula among the regions divided by the straight line l2 and the straight line l1.

Preferably, k1 = 2/3, k2 = 2/5, and the result is as exemplarily shown in Fig.

Fifth, in the case of the above-mentioned cup-to-disk ratio information, the machine learning model includes segmentation networks, and based on the optic disk and optic cup The C / D ratio can be extracted.

In the clinical examination of the fundus, the optic cup refers to the part of the optic disc where the blood vessels are connected to the brain, and because the optic bundle enters the optic disc between the optic disc and the optic disc, This suggests that glaucoma can be a source of suspicion.

The fraction network model used here is a model network in which a convolution block in the form of a VGG (Visual Geometry Group) network is stacked and a fully-connected layer is connected to the synthesis block. As a result, glaucoma can be automatically read with high accuracy (ROC_AUC 0.8617 or higher).

As described above, learning of the machine learning model used in the determination module 320 may be performed by the determination module 320 or may be learned by a separate learning module 330. [

Once attribute information has been extracted for an individual fundus image based on the machine learning model, the attribute information may be stored in the storage and transmission module 340 or provided to an external entity or to other devices associated with the computing device 200 (Not shown).

3, there are shown a result input module 350 that can obtain evaluation information evaluated with respect to the extracted attribute information and modified information obtained by arbitrarily modifying the extracted attribute information, An update module 360 that can update the machine learning model based on the communication module 210 and the processor 210 may be further provided as a component of the computing device 200, (Not shown).

More specifically, the result input module 350 may further include a feature input module 352 (not shown) and a diagnostic result input module 354 (not shown) in the second embodiment of the funduscopic image read support method described below .

A first embodiment of a funduscopic image reading support method according to the present invention will now be described in detail with reference to FIG. FIG. 8 is a flowchart illustrating an exemplary method for supporting funduscopic image reading according to the first embodiment of the present invention.

Referring to FIG. 8, an image capturing module 310 implemented by the communication unit 210 of the computing device 200, as described above with reference to FIG. 3, (Step S810) to acquire a fundus image of the subject or to acquire another device (not shown) interlocked with the computing device 200 (S810).

The method for supporting the funduscopic image reading according to an embodiment of the present invention is characterized in that the discrimination module 320 implemented by the processor 220 of the computing device 200 includes a function for extracting attribute information of the funduscopic image (S820) of extracting the attribute information from the fundus image or extracting the attribute information from the fundus image based on a machine learning model (S820); a communication unit (210) of the computing device (200) 220 further includes a step S830 of providing the extracted attribute information to an external entity or providing the other device with the attribute information extracted by the storage and transmission module 340 implemented in step S830.

Here, the external entity includes a user, an administrator of the above-described computing apparatus, a medical professional in charge of the subject, and the like, but it should be understood that any subject can read the funduscopic image something to do.

Meanwhile, in order to perform the fundus camera image reading support method according to this embodiment, the learning module 330 for performing the machine learning model needs to be pre-learned (not shown) ).

For example, the learning module 330 may learn the machine learning model using individual fundus images of a plurality of existing subjects and data labeled with them as learning data.

In this embodiment, it is possible to support the reading of the fundus image based on the previously learned machine learning model. If information obtained by evaluating or modifying the extracted attribute information is used again as the data of re-learning for the machine learning model, In order to take advantage of this advantage, the method of supporting fundus camera image reading according to this embodiment of the present invention is characterized in that the processor 220 performs the evaluation of the extracted attribute information Acquires modification information obtained by arbitrarily modifying one evaluation information or the extracted attribute information through the result input module 350 and updates the machine learning model through the update module 360 based on the evaluation information or the modification information Or supporting the other device to update (S840). At this time, the fundus image and the labeling data, which were not taken into consideration in the previous learning, are further considered, and the error in the previous learning can be corrected, so that the accuracy of the machine learning model is improved. As the data accumulates, The performance of the system is continuously improved.

Here, the evaluation information and the modification information may be provided from an external entity such as the medical professional.

Next, a second embodiment of a funduscopic image reading support method according to the present invention will be described with reference to FIG. 9 and FIGS. 10A to 10E.

FIG. 9 is a flow chart illustrating a second embodiment of a funduscopic image reading support method according to the present invention. FIGS. 10A to 10E illustrate a user interface (UI) provided at each step of the embodiment illustrated in FIG. As shown in FIG.

Referring to FIG. 9, in the fundus camera image reading support method according to the second embodiment, an image acquisition module 310 implemented by the communication unit 210 of the computing device 200 receives an eye fundus image (S910) to acquire or acquire another device associated with the computing device (200).

As shown in FIG. 10A, at step S910, at least one of the identification number, age, gender, and binocular discrimination information of the funduscopic image for the convenience of the user, that is, the reader, Lt; / RTI &gt; Also, since the reading may be performed by a plurality of users, at least one user supporting the reading of the funduscopic image according to the method of this embodiment is a user who is Kn in the nth user group among the total M user groups , A plurality of readings can be performed on one fundus image, thereby enabling cross-validation of the labeling data as the readout result.

Referring to FIG. 9, the fundus camera image reading support method according to this embodiment includes a discrimination module 320 implemented by the processor 220 of the computing device 200, (i) (Ii) performing at least one of inputting or outputting the normal information from at least one user or supporting the input of the other device (S920 ).

As illustrated in FIG. 10B, the normalization information may be one of an abnormal fundus and a normal fundus. For example, if a normal fundus is selected from among these, the method of this embodiment for the fundus image can be completed and the next fundus image can be loaded.

In the case where the quality of the funduscopic image is to be evaluated before the step S920, the funduscopic image reading support method may include the steps of (i) (Ii) a process of receiving input of the image quality information from the at least one user or supporting the input of the other device by the other device, (S915; not shown).

10A, wherein the image quality information includes at least one of: good, media opacity, small pupil / defocusing, and ungradable. can do. For example, if it is determined that the determination is impossible, the method of this embodiment for the funduscopic image can be completed and the next fundus image can be loaded.

After the execution of step S920, the fundus image read support method according to this embodiment is a method in which the processor 220 of the computing device 200 reads the individual compartments of the funduscopic image through the determination module 320 (I) generating the feature information corresponding to the individual compartment or supporting the other device to generate the feature information corresponding to the individual compartment, And (ii) a process of receiving the identification information from the at least one user via the communication unit 210 or supporting the other device to receive the input via the communication unit 210 (S930).

As shown diagrammatically in FIG. 10C, the finding information corresponding to the individual compartments of the fundus image herein is indicative of bleeding, hard exudates, cotton swabs, drusen and drusenoid deposits (drusen & drusenoid deposits) But not limited to, macular hole, vascular abnormality, membrane, fluid accumulation, chorioretinal atrophy / scar, choroidal lesion, myelinated nerve fiber, RNFL defect, galucomatous disc change, non-glaucomatous disc changes, other findings or artifacts.

The method for supporting fundus camera image reading according to this embodiment is characterized in that the processor 220 of the computing device 200 transmits the diagnosis result input module 354 through the diagnosis result input module 354 to (i) (Ii) a process of generating diagnostic information for the image or generating the other device; and (ii) receiving the diagnostic image information corresponding to the fundus image and the feature information from the at least one user, (S940) a process of supporting at least one of the processes for supporting the reception of the message.

Diagnostic information may include, but is not limited to, dry AMD, Wet AMD, Early DR, Advanced DR, CRVO, BRVO / Hemi-CRVO, Epiretinal membrane, Macular Hole, Other Retinal / Choroidal Diseases / Findings , Glaucoma Suspect, Other Disc Diseases / Findings, Floaters / Artifacts Suspect.

One such diagnostic information may be selected as well as one. However, there is an exclusive relationship between these diagnostic information, the choices of Dry AMD and Wet AMD are mutually exclusive, the choice of Early DR and Advanced DR is mutually exclusive, and the choice of CRVO and BRVO / Hemi-CRVO is mutually exclusive. This is to prevent logical or pathologically unnecessary or unreasonable readings.

As described above, a deep learning method such as CNN may be used as a machine learning model for generating diagnostic information based on the finding information, but other statistical methods well known in the art such as a Bayesian network may be utilized.

FIG. 11 is a diagram showing a Bayesian network as one of methods for generating diagnostic information from feature information in the embodiment illustrated in FIG.

The Bayesian network can be used to probabilistically deduce the final diagnostic information from the input information, that is, the connection between each node illustrated in FIG. 11 corresponds to the probability value.

In the method of supporting funduscopic image reading according to this embodiment of the present invention, the processor 220 may store the diagnostic information and the diagnostic information on the fundus image through the storage and transmission module 340, (S950) to provide an entity with or store or provide the other device.

In step S950, as in the user interface illustrated in FIG. 10E, the ophthalmologist may provide the ophthalmologist with the finding information and the diagnosis information on the fundus image according to the selection of the at least one user. Thereby helping to modify the diagnostic information and the diagnostic information.

As in the first embodiment, the fundus image read support method according to the second embodiment is also characterized in that correction information obtained by arbitrarily modifying diagnostic information or diagnostic information determined by the machine learning model and diagnostic information with regard to the diagnostic information and diagnostic information Through the input module 352 or the diagnostic result input module 354 and updates the machine learning model through the update module 360 based on the evaluation information or the modification information or supports the other device to update the machine learning model (S960) (not shown). At this time, since the error in the previous learning can be corrected, the accuracy of the machine learning model is improved, and the performance of the machine learning is continuously improved as the data accumulates.

As described above, according to the embodiments of the present invention, it is possible to efficiently and accurately read the fundus images compared to conventional medical professionals depending on their experience or knowledge, thereby saving time for the medical staff, It has the effect of increasing the quality and innovating the workflow in the medical field.

Based on the description of the above embodiments, one of ordinary skill in the art can clearly understand that the present invention can be achieved through a combination of software and hardware, or can be accomplished by hardware alone. Objects of the technical solution of the present invention or portions contributing to the prior art can be recorded in a machine-readable recording medium implemented in the form of program instructions that can be executed through various computer components. The machine-readable recording medium may include program commands, data files, data structures, and the like, alone or in combination. The program instructions recorded on the machine-readable recording medium may be those specially designed and constructed for the present invention or may be those known to those of ordinary skill in the computer software arts. Examples of the machine-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like.

The hardware device may be configured to operate as one or more software modules for performing the processing according to the present invention, and vice versa. The hardware device may include a processor, such as a CPU or a GPU, coupled to a memory, such as ROM / RAM, for storing program instructions, and configured to execute instructions stored in the memory, And a communication unit. In addition, the hardware device may include a keyboard, a mouse, and other external input devices for receiving commands generated by the developers.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, And various modifications and changes may be made thereto by those skilled in the art to which the present invention pertains.

Therefore, the spirit of the present invention should not be construed as being limited to the above-described embodiments, and all of the equivalents or equivalents of the claims, as well as the following claims, I will say.

Equally or equivalently modified such methods will include logically equivalent methods which can yield, for example, the same results as those of the method according to the invention.

Claims (7)

A method of supporting funduscopic imaging of a subject,
(a) supporting a computing device to acquire a fundus image of the subject or acquire another device associated with the computing device;
(b) a process in which the computing device generates (i) generates information on whether or not the fundus image is normal or generates the other device, and (ii) receives the normal information from at least one user, Performing at least one of the following processes:
(c) supporting the computing device to generate or generate compartment discrimination information indicative of individual compartments of the funduscopic image; and (i) generate discovery information corresponding to the individual compartments, (Ii) a process of receiving input of the finding information from the at least one user or supporting the input of the other device by the at least one user;
(d) the computing device is configured to: (i) generate diagnostic information for the fundus image or generate the other device based on the finding information; and (ii) And a process of receiving input of the diagnostic information corresponding to the finding information or supporting the other device to receive input; And
(e) storing the diagnostic information and the diagnostic information on the fundus image, or providing an external entity, or allowing the other device to store or provide
, &Lt; / RTI &
The step (a)
(i) at least one of the identification number, age and sex of the subject; And (ii) attribute information of the fundus image is provided to the at least one user,
The attribute information of the funduscopic image is,
Binocular discrimination information indicating whether the funduscopic image is a left eye image or a right eye image,
Position information indicating the position of at least one of the macula and the optic nerve head included in the fundus image,
Division division information indicating a division of the fundus image,
Blood vessel information showing only the image of the optic nerve blood vessel included in the fundus image, and
The information of the C / D ratio (cup-to-disk ratio) according to the fundus image
Wherein the at least one fundus image comprises at least one of the following: The method according to claim 1,
Before the step (b)
(b0) a process for the computing device to generate (i) image quality information for evaluating the quality of the funduscopic image or to assist the other device to generate; and (ii) Or a process of supporting the other device to receive an input
Further comprising the steps of: delete The method according to claim 1,
Wherein the at least one user is selected from Kn users included in the n-th user group among all M user groups, whereby a plurality of readings are performed for one fundus image. The method according to claim 1,
The step (c)
Wherein the generated partition identification information is provided to the at least one user so that modification by the at least one user of the partition identification information is enabled. A computer-readable medium having stored thereon instructions for implementing a method of any one of claims 1, 2, 4, and 5, program. A computing device that supports reading funduscopic images of a subject, comprising:
A communication unit for implementing an input module for acquiring a fundus image of the subject; And
(1-i) generating normal information on the funduscopic image or generating other apparatuses to be interlocked through the communication unit, and (1-ii) transmitting the normal information from at least one user through the communication unit And a process of receiving input or supporting the other device to receive input,
, &Lt; / RTI &
The discrimination module comprises:
Generates partition information indicating individual sections of the funduscopic image or supports the other apparatus to generate,
The processor comprising:
(2-i) a process of generating or generating the finding information corresponding to the individual segments, and (2-ii) receiving the finding information from the at least one user through the communication unit, A process for supporting at least one of the processes of receiving input from other devices;
(3-i) a process of generating diagnosis information for the funduscopic image or generating the diagnostic information for the other device based on the finding information, and (3-ii) a process of generating diagnostic information for the fundus image from the at least one user And a process of inputting the diagnosis information corresponding to the finding information or supporting the other device to receive input, And
Further comprising a storage and transmission module for storing the finding information and the diagnosis information on the funduscopic image or for providing or storing the diagnostic information to an external entity or storing or providing the other device through the communication unit,
(i) at least one of the identification number, age and sex of the subject; And (ii) attribute information of the fundus image is provided to the at least one user,
The attribute information of the funduscopic image is,
Binocular discrimination information indicating whether the funduscopic image is a left eye image or a right eye image,
Position information indicating the position of at least one of the macula and the optic nerve head included in the fundus image,
Division division information indicating a division of the fundus image,
Blood vessel information showing only the image of the optic nerve blood vessel included in the fundus image, and
The information of the C / D ratio (cup-to-disk ratio) according to the fundus image
Wherein the at least one of the first and second fundus images includes at least one of the first and second fundus images.

Images