WO2019189971A1

WO2019189971A1 - Artificial intelligence analysis method of iris image and retinal image to diagnose diabetes and presymptom

Info

Publication number: WO2019189971A1
Application number: PCT/KR2018/003806
Authority: WO
Inventors: 남궁종; 길용현
Original assignee: 주식회사 홍복
Priority date: 2018-03-30
Filing date: 2018-03-30
Publication date: 2019-10-03

Abstract

An artificial intelligence analysis method of an iris image and a retinal image to diagnose diabetes and presymptom on a smartphone according to an embodiment of the present invention comprises the steps of: acquiring a captured image of a user's eye by a user terminal; pre-processing, by the user terminal, the captured image to extract a region of interest; expanding, by the user terminal, the resolution of the extracted region of interest; compressing, using a library for reducing the amount of data, the region of interest to be transmitted to a server, by the user terminal; performing, by the user terminal, user authentication on the iris image; encrypting data of the iris image and the retina image to be stored in the server by the user terminal; learning, by the server, an artificial neural network to diagnose the diabetes and the presymptom on the basis of the stored data; and predicting a risk of developing diabetes using the learned artificial neural network by the user terminal, wherein the step of pre-processing the captured image further comprises a step of gray-processing an area not extracted as the region of interest of the captured image to extract only the region of interest, and the step of predicting the risk of developing diabetes further comprises a step of predicting the type of diabetes on the basis of the position and shape of a lesion area.

Description

How to artificially analyze iris and retinal images to diagnose diabetes and prognostic symptoms

The present invention relates to a method of artificially analyzing iris images and retinal images to diagnose diabetes and prognostic symptoms, and can accurately predict diabetes and reliably predict the possibility of diabetes through iris and retinal images on a smartphone. It is about providing a service.

Diabetes is a type of metabolic disease, such as insufficient insulin secretion or normal functioning. It is characterized by high blood glucose, which increases the concentration of glucose in the blood, and causes many symptoms and signs and releases glucose from urine. .

Diabetes is classified into type 1 diabetes and type 2 diabetes. Type 1 diabetes is a disease caused by the inability to produce insulin at all, usually occurs in children or adolescents, type 2 diabetes is caused by the inability to effectively burn glucose due to the insulin function that lowers blood sugar.

Conventional diabetes diagnosis evaluates the severity through three diagnostic tests. Blood glucose test, oral glucose load test, glycated hemoglobin (HbA1c) test. Blood glucose test is one of the most commonly used methods of diagnosing diabetes. If the fasting blood sugar (fasting blood sugar after fasting for more than 8 hours) is 126 mg / dL or more, and whether or not the measured blood sugar is 200 mg / dL or more, the diagnosis is diabetes. do. The oral glucose test is performed to confirm the blood sugar level, which is not high enough to be called diabetes in the blood glucose test, but also in the normal range. The test method is to perform venous blood collection without fasting for more than 8 hours. After 75g of oral glucose was mixed with 300ml of water, 30 minutes later, 60 minutes later, and 120 minutes later, blood collection was done. After 2 hours, the blood glucose level was 200mg / dL. do. The glycated hemoglobin test is an important diagnostic method for diagnosing diabetes as well as long-term glycemic control in the medical community. The glycated hemoglobin test reflects the average blood glucose level, that is, the degree of blood glucose control over the last two to three months. The normal range is 4-6%, and 6.5% or more is diagnosed as diabetes. It is classified into impaired glucose tolerance, impaired fasting glucose, and diabetes mellitus according to blood glucose level.

On the other hand, the prior art has a variety of disease diagnosis services through artificial intelligence, the existing AI disease diagnosis service is a server-client structure, using MRI, PET, retina, etc. as a medical image for learning or diagnosis. The characteristics of each disease are extracted from the convolutional neural network (CNN), and the location of the disease and which part is a problem with high accuracy.

The above-described conventional techniques distinguish diseases with high accuracy, but hospitals must use expensive equipment to acquire images (MRI, PET, retina, etc.), and individuals go directly to the hospital for imaging, and high cost of imaging You have to pay And since it is server-client based, it has a problem that it is difficult to use due to the lack of medical facilities and the lack of medical staff in underdeveloped areas such as Africa. Since 75% of diabetics live in developing countries, a number of diabetics will be prevented if a low-cost, diabetic diagnosis is available that does not require complex medical facilities.

Therefore, there is a need for low-cost and accurate diagnosis services wherever an individual can easily receive diabetes and prognostic symptoms diagnosis services without the help of medical staff.

An object of the present invention is to diagnose diabetes and prognostic symptoms in a smart phone in order to solve the high cost, space constraints, discomfort that is a problem of disease diagnosis using the existing artificial neural network. Diagnosing diabetes and prognostic symptoms with iris and retinal images taken with a dedicated iris authentication lens attached to the front of the smartphone has the advantage of low cost and no space limitation, while the limited hardware performance of the smartphone This can make the use of wide and deep neural networks less accurate. Therefore, even in an area where there is no network, the purpose is to provide a service for diagnosing diseases through iris and retinal imaging in a smartphone and diagnosing diabetes and prognostic symptoms with high accuracy.

Technical problems to be achieved in the present invention are not limited to the above technical problems, and other technical problems that are not mentioned will be clearly understood by those skilled in the art from the following description.

According to an embodiment of the present invention, a method of artificially analyzing an iris image and a retinal image to diagnose diabetes and prognostic symptoms according to an embodiment of the present invention may include: Obtaining; Preprocessing the captured image to extract a region of interest at the user terminal; Expanding the resolution of the extracted region of interest at the user terminal; Compressing, at the user terminal, the library for reducing the amount of data for the region of interest to be transmitted to a server; At the user terminal, performing user authentication through the iris image; Encrypting, at the user terminal, data of the iris image and the retina image to be stored in the server; Learning, at the server, an artificial neural network to perform the diagnosis of the diabetes and the prognostic symptoms based on the stored data; And predicting, in the user terminal, a probability of developing diabetes using the learned artificial neural network, and the preprocessing of the captured image includes graying out an area that is not extracted to the ROI and extracting only an ROI. The method may further include estimating the type of diabetes, based on the location and shape of the lesion in the ROI.

In the method of artificially analyzing the iris image and the retina image according to an embodiment of the present invention, the step of acquiring the captured image may include the iris image and the retina using an iris authentication-dedicated lens attachable to the user terminal. The method may further include extracting an image.

According to an embodiment of the present invention, a method of artificially analyzing an iris image and a retina image may include: preprocessing the captured image, extracting a region of interest including an iris region and a retina region from the captured image; Setting an axis for obtaining rotation rates of the iris region and the retinal region in the region of interest; Extracting only the iris region and the retina region from the region of interest; And repositioning the iris region and the retinal region based on the rotation rate.

According to an embodiment of the present invention, a method of analyzing an iris image and a retinal image by artificial intelligence may further include setting the axis to set an axis for obtaining the rotation rate after performing eyelid segmentation. Repositioning the iris region and the retinal region is performed such that when the iris region and the retinal region are inclined with respect to the vertical direction, the iris region and the retinal region are 0 ° with respect to the vertical direction using the rotation rate. It characterized in that it further comprises the step of repositioning by rotation and interpolation to be.

In the method of artificially analyzing the iris image and the retinal image according to an embodiment of the present invention, the step of expanding the resolution may include: dividing the ROI into a specific patch size, Applying a convolutional neural network to further extend the resolution of the region of interest to a predetermined resolution, and before the convolutional neural network is applied, zero padding is added and the convolutional neural network is applied. In this case, a dense block for re-learning the features of the previous step is used, and a skip connection is applied.

In the method of artificially analyzing the iris image and the retina image according to an embodiment of the present invention, the step of encrypting, encrypting the code value extracted from the data of the iris image and the retina image to be stored in the server And encrypting using the key.

According to an embodiment of the present disclosure, a method of artificially analyzing an iris image and a retinal image may include: learning the artificial neural network, storing the data of the iris image and the retinal image in a database; Learning on the stored data using the artificial neural network to perform parallel classification, detection and segmentation for diagnosing the diabetes or prognostic symptoms; The artificial neural network includes a factorization method, a detachable convolution method, and a pointwise convolution method, classifying the data of the iris image and the retinal image, Extracting a feature map for the data; Extracting a region of interest corresponding to a predetermined anchor box by applying a region proposal network (RPN) to the feature map; Applying pooling to the learned region to convert regions of different sizes into regions of the same size; And classifying, detecting, and dividing a region corresponding to the diabetic or the prognostic symptoms based on the same size region.

In the method of artificially analyzing the iris image and the retinal image according to an embodiment of the present invention, the step of predicting the occurrence of diabetes mellitus, by applying the artificial neural network to the data of the iris image and the retinal image And predicting the probability of developing diabetes from the prognostic symptoms.

In the method of artificially analyzing the iris image and the retinal image according to an embodiment of the present invention, after predicting the probability of diabetes, the user's terminal, the individual constitution diagnosis service, complication diagnosis service and eating habits management service It characterized in that it further comprises the step of providing.

According to the present invention as described above it can be obtained the effect described below. However, effects obtained through the present invention are not limited thereto.

First, according to the present invention, regardless of the location using the iris authentication dedicated lens attached to the front of the smartphone to extract the iris and retina image (guaranteed excellent image quality) and in the smart phone through a pre-learned neural network, not the server-client method Helps diagnose diabetes and prognostic symptoms in older areas where networks are not connected.

Second, according to statistics released by the International Diabetes Federation (IDF) in 2015, 415 million of the world's adults were diagnosed with diabetes, and 310 million were at high risk of developing diabetes. Currently, 5-20% of the major national welfare budgets are used to prevent and treat diabetes-related diseases, and 75% of the world's diabetics live in developing countries, preventing the prevention of diabetes in underdeveloped areas. Therefore, when the present invention becomes a service, an individual can easily diagnose a diabetes in any region at a low price, thereby lowering the incidence of diabetes, thereby reducing diabetes related welfare budgets and allocating more necessary places to improve individual's quality of life. You can.

1 is a conceptual diagram showing an entire system according to an embodiment of the present invention.

2 is a flowchart illustrating a method of artificially analyzing an iris image and a retinal image to diagnose diabetes and prognostic symptoms according to an embodiment of the present invention.

3 is a conceptual diagram of diabetes and prognostic symptoms AI diagnosis service through iris and retinal image on a smart phone according to an embodiment of the present invention.

4 is a system configuration diagram of diabetes and prognostic symptoms artificial intelligence diagnosis service through iris and retinal image on a smart phone according to an embodiment of the present invention.

FIG. 5 is a schematic block diagram of an Image Super Resolution artificial intelligence neural network for raising the iris and retinal image shown in FIG. 3 from the low resolution to the high resolution.

FIG. 6 is a block diagram of a database structure for storing data in the neural network learning unit shown in FIG. 3 of the present invention.

7 is a schematic block diagram of the artificial intelligence based disease diagnosis neural network shown in FIG. 4 of the present invention.

8 is an additional service configuration diagram in addition to diabetes and prognostic symptoms diagnosis service in the service unit shown in FIG. 4 of the present invention.

9 is a schematic diagram of an encryption method for storing data of an iris and retina image data encryption module in the security unit shown in FIG. 4 of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. The detailed description, which will be given below with reference to the accompanying drawings, is intended to explain exemplary embodiments of the present invention and is not intended to represent the only embodiments in which the present invention may be practiced.

The present embodiments are merely provided to complete the disclosure of the present invention and to fully inform the scope of the invention to those skilled in the art, and the present invention is defined by the scope of the claims. It is only.

In some instances, well-known structures and devices may be omitted or shown in block diagram form centering on the core functions of the structures and devices in order to avoid obscuring the concepts of the present invention. In addition, the same components will be described with the same reference numerals throughout the present specification.

Throughout the specification, when a part is said to "comprising" (or including) a component, this means that it may further include other components, except to exclude other components unless specifically stated otherwise. do.

In addition, the term "… unit" described in the specification means a unit for processing at least one function or operation, which may be implemented in hardware or software or a combination of hardware and software. Furthermore, "a" or "an", "one", and the like shall be, in the context of describing the present invention, both singular and plural unless the context clearly dictates otherwise or is clearly contradicted by the context. It can be used as a meaning including.

In addition, specific terms used in the embodiments of the present invention are provided to aid the understanding of the present invention, and unless otherwise defined, all terms used herein, including technical or scientific terms, are used to describe the present invention. It has the same meaning as commonly understood by one of ordinary skill in the art. Use of this specific terminology may be modified in other forms without departing from the spirit of the invention.

Referring to FIG. 1, the user terminal may be an electronic device including a smart phone, a tablet PC, a laptop, and the like. The user terminal may include a camera, and the camera may include an iris authentication dedicated camera or an iris authentication dedicated lens may be attached to the camera. Therefore, when the user's eye is photographed using the user terminal, a high resolution screen or image may be secured for a specific area including the iris and the retina. The captured image obtained by photographing the eyes of the user through the user terminal may be used to predict and diagnose diabetes and prognostic symptoms. Hereinafter, with reference to the diagnosis of diabetes, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings.

Referring to FIG. 2, in a method of artificially analyzing an iris image and a retinal image to diagnose diabetes and prognostic symptoms according to an embodiment of the present invention, a user terminal acquires an image captured by a user's eyes. In operation S11, the user terminal may preprocess the captured image to extract a region of interest. When extracting a region of interest, for example, an area that is not extracted from the region of interest is an unnecessary region, and thus a gray process is performed to extract only the region of interest, thereby reducing the amount of computation when diagnosing diabetes and prognostic symptoms from the image. More accurate diagnosis can be made. Further, in the user terminal, the resolution of the extracted region of interest is expanded (S13), and the user terminal is compressed using a library for reducing the amount of data for the region of interest to be transmitted to the server (S14). In step S15, user authentication is performed through the iris image, and at the user terminal, data of the iris image and the retinal image to be stored in the server is encrypted (S16). The artificial neural network to be diagnosed may be learned (S17), and the user terminal may predict the onset probability of diabetes using the learned artificial neural network (S18). At this time, the type of diabetes can also be predicted based on the position and shape of the lesion area or the like in the region of interest. Diabetes may be classified into type 1 diabetes, type 2 diabetes, and the like according to insulin function, or may be classified into impaired glucose tolerance and impaired fasting glucose according to blood glucose levels. Hereinafter, the configuration of performing the above-described S11 to S18 through FIG. 3 will be described in detail.

Figure 3 shows a conceptual diagram of a diabetic and prognostic symptoms AI diagnosis service through iris and retinal image on a smart phone according to an embodiment of the present invention.

Referring to FIG. 3, the diabetic and prognostic symptoms AI diagnosis service 100 through the iris and the retinal image on the smart phone according to an embodiment of the present invention includes an image capturing unit 101, an image preprocessing unit 102, and a neural network. The learning unit 103 is provided through the disease diagnosis unit 104.

The image capturing unit 101 captures an eye image through a smartphone, and photographs using an iris authentication-dedicated lens that can be attached to the front of the smartphone, and acquires a captured image (S11). While cameras built into existing smartphones can capture eye images, iris-certified lenses use short-focus lenses and infrared LEDs to capture clear iris and retinal images with less impact on the environment. The sharper and less affected the light is, the better it is for iris recognition and diabetic and diagnosing AI symptoms. The captured image may be different for each iris color.

The image preprocessing unit 102 may extract only the iris and retinal regions necessary for diagnosing a disease from the captured eye image. First, only the region-of-interest (RoI) can be extracted from the eye image using the JPEG2000 library. Here, the region of interest refers to the minimum region necessary for extracting the iris and the retina. The reason for extracting the region of interest is that the amount of computation for the extraction and relocation of the iris and retina can be reduced. After performing eyelid segmentation in the extracted region of interest, the iris and retina parts are extracted separately and the outer part of the eyelid is grayed out. In the grayed image, connect the edges of the both ends of the eyelids with a line, and then set the axis, and set the axis's rotation rate based on 0 degree, and then rotate and interpolate the extracted iris and retinal images by the rotation rate. Relocate through the back. The reason for relocating the iris and retinal images is that the location of lesions such as tears, pigmentation, and cockroaches is important information for diagnosing diabetic prognostic symptoms through iris and retinal imaging. Since it means that there is a problem with a specific organ according to the location, the iris and retinal images may be rearranged based on 0 ° to determine accurate prognostic symptoms (S12). In addition, when the iris and retinal images have low resolution, the data to be referred to when diagnosing disease may be relatively inaccurate. Accordingly, the artificial neural network converting the low resolution to the high resolution may be converted to the high resolution (S13). Artificial neural networks that convert low resolution to high resolution can separate iris and retinal images into a specific patch size and then expand the resolution through a convolutional neural network (CNN). Since the original image is expanded by a specific patch unit, the image can be restored by merging the output images. The neural network that converts the low resolution into high resolution uses a dense block to enhance and reuse the feature propagation of the previous stage and extracts the input image to learn more about the boundary of the image. You can use the Skip Connection to connect to the previous video. If you proceed to this point, you can get the images needed to diagnose diabetes and prognostic symptoms with iris and retinal images, and send the iris and retinal images to a remote database through the network. After compressing the image using the JPEG2000 library (S14), it can be transmitted over a network. In addition, user authentication may be performed through the iris image (S15).

The neural network learning unit 103 is composed of a database storing iris and retinal image data and a neural network model learning from the stored data. There are many different types of databases, but you can use Oracle's highly reliable database. The information to be stored in the database must be stored encrypted with bio information, unique identification information (resident registration number, passport number, driver's license number, alien registration number) and password corresponding to personal information according to the Personal Information Protection Act. Since the iris and retinal image data correspond to bioinformation, the iris and retinal image data should be stored after encryption (S16). Encryption methods for storing data include application self-encryption, DB server encryption, DBMS self-encryption, DBMS encryption function call, and operating system encryption. The performance impact of each encryption method differs depending on the deployment environment. DB encryption method should be selected in consideration of constraints. When encrypting, it is very important to set an encryption key. The encryption key uses a code value extracted from iris and retinal image data, encrypts it using the encryption key, and stores it in a database. In addition, using the stored iris and retinal image data, the artificial intelligence neural network for diagnosing diabetes and prognostic symptoms can be learned (S17). Diagnosis of Diabetes and Prognostic Symptoms through Iris and Retinal Imaging The AI neural network should be trained to be classified, detected, or segmented for diabetic precursor symptoms. Therefore, this artificial intelligence neural network can pre-train the neural network part that classifies the precursor symptoms for more accurate classification in learning. Since the artificial neural network used for the classification part takes up a large part of the entire neural network, this part is made as lightweight as possible so that it can be performed smoothly on mobile. There are many ways to lighten the AI neural network. For example, multiplying a large filter size of 5 x 5 into 3 x 3 and small filters of 3 x 3 and performing a multiplication can reduce the factorization by 20-30% of the computation performed by the existing large filter. Method, the conventional convolution layer learns filters in horizontal-vertical (two-dimensional) and channels (one-dimensional), and each filter simultaneously requires mapping and spatial correlation. Depthwise Separable Convolution method, which reduces the amount of computation by 1 / N + 1 / D_K compared to the conventional convolution, by processing the inter-channel and spatial correlation separately Instead of dealing with spatial correlation in the conventional convolution layer, only the correlation between channels is used to increase or decrease the dimension of the output feature map. Using a combination of pointwise convolution methods to reduce the artificial intelligence neural network. After performing pre-training on the classification part, we can learn to find the location of the bounding box of the probable part with the extracted feature map. Since the extracted bounding boxes are different sizes, ROI pooling is performed to fit a certain size, and the classification of diabetic prognostic symptoms is carried out through a convolutional neural network (CNN). Can learn / Detection / Segmentation. For classes that are not prognostic symptoms, students learn to penalize penalties for more accurate classification, detection, and segmentation.

The disease diagnosis unit 104 may diagnose diabetes by detecting diabetes mellitus and prognostic symptoms using the iris and retinal images preprocessed with the artificial intelligence neural network learned by the neural network learning unit 103 (S18).

4, the diabetic and prognostic symptoms AI diagnosis system 200 through the iris and the retina image on the smart phone according to an embodiment of the present invention is a user unit 201, platform unit 202, security unit 203 ), A service unit 204, a data server unit 205, and a neural network learning server unit 206. The user unit 201 may be an individual or a hospital and may be an institution that desires diagnostic services through the iris and retina. The platform unit 202 may be Android, IOS, WINDOWS, etc., because it must be serviced on the mobile. The security unit 203 may be divided into a part for authenticating a person with an iris recognition module and a part for encrypting iris and retinal image data in order to receive diabetic and prognostic symptom diagnosis services through iris and retinal images. The service unit 204 is capable of diagnosing diabetes and prognostic symptoms through the learned AI-based disease diagnosis neural network, individual constitution diagnosis service comparing the iris constitution features with collected iris data, and probable occurrences based on the extracted prognostic symptoms. Complications diagnostic services, and personalized eating habits management services through the diagnosis of constitution and prognostic symptoms. The data server unit 205 may store iris and retinal image data necessary for the AI neural network to learn. The neural network learning server unit 206 may learn the artificial intelligence neural network through the stored iris and retinal image data.

FIG. 5 is a schematic block diagram of an Image Super Resolution artificial intelligence neural network 300 that raises the iris and retinal images shown in FIG. 3 from low resolution to high resolution. If the iris and retinal images have low resolution, there is less data to refer to when diagnosing a disease, so the accuracy of the iris and retinal images is relatively low. Therefore, it is necessary to raise the low resolution iris and retinal images to high resolution. The low resolution iris and retinal images coming into the input are divided into a specific patch size, and the divided images are input as inputs and the reconstructed output image is extracted through a multiplication neural network. In addition, the use of a generalized composite neural network reduces the size of the output feature map, making it impossible to make full use of information about pixels near the image boundary. Therefore, zero padding is added before the product of multiplication to maintain the size of the feature map in all layers so that pixels near the image boundary can be accurately predicted. In the course of learning, we use dense blocks that allow us to relearn the features of the previous stages, reducing gradient vanishing, and using the skip connections used in the Resnet neural network. This allows for faster learning and better learning the edges of the input image. Since the output images are divided into patch units, they can be converted into a single high resolution image through a merge process.

FIG. 6 is a block diagram illustrating a structure of a database 400 for storing iris and retinal image data in the neural network learning unit illustrated in FIG. 3 of the present invention. In order to store iris and retinal image data in a database, the following operation may be performed. First, server processes in the database can be allocated to process the SQL. The server process uses the handler to find the handlers in the library cache that contain information about the SQL. If the handler exists, the server parses the SQL statement. Hard Parsing) procedure. Hard parsing includes syntax checking (SQL spelling and SQL grammar checking), semantic checking (table / column existence), query transformation (optimizer rewrites SQL for better performance), authorization checking, and DML (Data) During Manipulation Language), lock type check, execution plan, parsing tree creation, and log record (LOG record generated during DML) are processed. Data is stored in a specific block in the data buffer cache, and when a commit is performed, the data of the block is written to the data file. Through this process, iris and retinal image data can be stored in a database.

7 is a schematic block diagram of the artificial intelligence based disease diagnosis neural network shown in FIG. 4 of the present invention. The AI-based disease diagnosis neural network shown in FIG. 4 of the present invention is composed of a disease classification unit 501 and a disease location detection unit 502, and the disease classification unit 501 is a diabetic precursor symptom (pancreas) from iris and retinal images. A multiplicative neural network can be used to classify site lesions, triglycerides, stress rings, etc.). The disease location detection unit 502 may extract the features for the prognostic symptoms while passing through the composite product neural network of the disease classification unit 501 and process the feature map output from the last output layer. The disease location detecting unit 502 may detect and segment the precursor symptom site based on a feature map from the disease classification unit 501. In order to extract a region of interest for prognostic symptoms on a feature map, the region is subjected to a Region Proposal Network (RPN). The region of interest is extracted according to a predetermined anchor box. Here, the region of interest may represent a detection region for prognostic symptoms in the iris and retinal images. Since the extracted regions of interest differ in size, they cannot be used in convolutional neural networks that require performance at a fixed size. Therefore, ROIs of different sizes are converted into the same size through the ROI pooling layer. After the conversion, the precursor symptom classification / detection / division of the corresponding region of interest may be performed in parallel to output to the iris and retinal images which precursor symptoms are detected at which positions. Based on the prognostic symptoms in the iris and retina of a person with diabetes, the predicted outcome of the neural network can be predicted.

Referring to FIG. 8, the additional service includes an image capturing unit 601, a data server unit 602, a constitution diagnosis unit 603, a disease diagnosis unit 604, a complication diagnosis unit 605, and a eating habit management unit 606. It can be provided using. The image capturing unit 601 may allow the user to photograph the iris and the retina using an iris authentication dedicated lens attached to the front of the smartphone. The data server unit 602 may encrypt and store the photographed iris and retina image data. The constitution diagnosis unit 603 may perform constitution diagnosis by the sun person, Taein person, Soyangin person, and the noise person in comparison with the iris constitution property from the personal iris and retinal image data stored in the database server. The disease diagnosis unit 604 may diagnose diabetes and prognostic symptoms using the captured iris and retinal image data. The complication diagnosis unit 605 may diagnose complications associated with the prognostic symptoms from the disease diagnosis unit 604. The eating habit management unit 606 may provide a personalized diet based on the constitution diagnosed by the constitution diagnosis unit 603 (sun person, Taein person, Soyangin person, and noise person) and the prognostic symptoms diagnosed by the disease diagnosis unit 604.

9 is a schematic diagram of an encryption method for storing data of an iris and retina image data encryption module in the security unit shown in FIG. 4 of the present invention. Encryption methods for storing data include application program encryption 701, DB server encryption 702, DBMS self encryption 703, DBMS encryption function call 704, and operating system encryption 705. Application self-encryption (701) is a method in which the encryption / decryption module is installed in each application server in the form of API library to call the encryption / decryption module within the application, but it does not affect the DB server. Modifications may be necessary. DB server encryption 702 is easy to implement because the encryption / decryption module is installed on the DB server and calls the encryption / decryption module connected to the plug-in from the DBMS. Therefore, it is necessary to create a view corresponding to the existing DB schema and add a table to encrypt, and there may be a load on the DB server. The DBMS self encryption (703) is processed at the DBMS kernel level by performing encryption / decryption processing using the encryption function built in the DBMS, so that modification of the existing application or modification of the DB schema is rarely required. Load can occur. The DBMS encryption function call 704 provides an API for the DBMS to perform its own encryption / decryption function, and the application needs to be modified in such a way that the application executes to use the function, and the DB server may be overloaded. have. The operating system encryption 705 is an encryption / decryption method using input / output system calls generated by the OS, but does not require modification of an application program or a DB schema, but may cause a load on the file server and the DB server to encrypt the entire DB file.

In addition to the above-described embodiment, the present invention accumulates big data indicating the possibility of diabetes and the degree of diabetes progression according to the position and shape of the lesion area associated with diabetes mellitus, and based on the big data, Learning and determining the possibility of diabetes mellitus and the progression of diabetes mellitus, and according to the possibility of diabetes mellitus and the degree of diabetes mellitus, an additional test including a blood glucose test, an oral glucose load test and a glycated hemoglobin (HbA1c) test is required. Notification can be made in real time. In addition, even when the degree of diabetes progressed exponentially in recent years, the progress of the diabetes may be notified in real time through the user terminal. The real time notification through the user terminal may be configured as a pop-up or push alarm.

Meanwhile, the above-described method may be written as a program executable on a computer, and may be implemented in a general-purpose digital computer which operates the program using a computer readable medium. In addition, the structure of the data used in the above-described method can be recorded on the computer-readable medium through various means. Computer-readable media that store executable computer code for carrying out the various methods of the present invention include, but are not limited to, magnetic storage media (eg, ROM, floppy disks, hard disks, etc.), optical reading media (eg, CD-ROMs, DVDs). Storage media).

It will be understood by those skilled in the art that embodiments of the present invention can be implemented in a modified form without departing from the essential characteristics of the above description. Therefore, the disclosed methods should be considered in descriptive sense only and not for purposes of limitation. The scope of the present invention is shown in the claims rather than the detailed description of the invention, and all differences within the equivalent scope should be construed as being included in the scope of the present invention.

Claims

As a method of artificially analyzing iris images and retinal images to diagnose diabetes and prognostic symptoms,

Obtaining, at the user terminal, a captured image of the user's eyes;

Preprocessing the captured image to extract a region of interest at the user terminal;

Expanding the resolution of the extracted region of interest at the user terminal;

Compressing, at the user terminal, the library for reducing the amount of data for the region of interest to be transmitted to a server;

At the user terminal, performing user authentication through the iris image;

Encrypting, at the user terminal, data of the iris image and the retina image to be stored in the server;

Learning, at the server, an artificial neural network to perform the diagnosis of the diabetes and the prognostic symptoms based on the stored data; And

Predicting, at the user terminal, a probability of developing diabetes using the learned artificial neural network;

The preprocessing of the captured image further includes graying out an area of the captured image that is not extracted to the region of interest, and extracting only the region of interest.

Predicting the probability of developing diabetes further comprises the step of predicting the type of diabetes, based on the location and shape of the lesion in the region of interest,

Method of artificially analyzing iris and retinal images.
The method of claim 1,

The acquiring of the captured image may further include extracting the iris image and the retina image by using an iris authentication dedicated lens attachable to the user terminal.

Method of artificially analyzing iris and retinal images.
The method of claim 1,

Preprocessing the captured image,

Extracting a region of interest including an iris region and a retina region from the captured image;

Setting an axis for obtaining rotation rates of the iris region and the retinal region in the region of interest;

Extracting only the iris region and the retina region from the region of interest; And

Further relocating the iris region and the retinal region based on the rotation rate;

Method of artificially analyzing iris and retinal images.
The method of claim 3,

The setting of the axis may further include setting an axis for obtaining the rotation rate after performing eyelid segmentation.

The repositioning of the iris region and the retinal region may include repositioning through rotation and interpolation such that the region of interest is 0 ° with respect to the vertical direction using the rotation rate when the region of interest is inclined with respect to the vertical direction. Further comprising the step of,

Method of artificially analyzing iris and retinal images.
The method of claim 1,

The expanding of the resolution may include: dividing the region of interest into a specific patch size, and then applying a multiplicative neural network to the divided image to extend the resolution of the region of interest to a preset resolution. Including,

Before the convolutional neural network is applied, zero padding is added,

When the convolutional neural network is applied, a dense block for re-learning the features of the previous step is used, and a skip connection is applied.

Method of artificially analyzing iris and retinal images.
The method of claim 1,

The encrypting step may further include encrypting a code value extracted from the data of the iris image and the retina image to be stored in the server as an encryption key.

Method of artificially analyzing iris and retinal images.
The method of claim 1,

Learning the artificial neural network,

Storing the data of the iris image and the retina image in a database;

Learning on the stored data using the artificial neural network in parallel to perform classification, detection and segmentation for diagnosing the diabetes or prognostic symptoms,

The artificial neural network includes a factorization method, a detachable convolution method, and a pointwise convolution method.

Method of artificially analyzing iris and retinal images.
The method of claim 7, wherein

Learning using the artificial neural network,

Classifying the data of the iris image and the retinal image and extracting a feature map for the diabetes or prognostic symptoms;

Extracting a region of interest corresponding to a predetermined anchor box by applying a region proposal network (RPN) to the feature map;

Applying pooling to the learned region to convert regions of different sizes into regions of the same size; And

Further comprising classifying, detecting, and dividing a region corresponding to the diabetes or the prognostic symptoms based on the same size region;

Method of artificially analyzing iris and retinal images.
The method of claim 1,

Predicting the probability of developing diabetes further comprises applying the artificial neural network to the data of the iris image and the retinal image to predict the probability of developing diabetes from the prognostic symptoms,

Method of artificially analyzing iris and retinal images.
The method of claim 1,

After predicting the probability of developing diabetes,

In the user terminal, further comprising the step of providing a personal constitution diagnosis service, complication diagnosis service and eating habits management service,

Method of artificially analyzing iris and retinal images.