CN111820863A

CN111820863A - Method for analyzing iris image and retina image by artificial intelligence technology

Info

Publication number: CN111820863A
Application number: CN202010083980.1A
Authority: CN
Inventors: 南宫钟
Original assignee: Hongbo Co ltd
Current assignee: Hongbo Co ltd
Priority date: 2019-04-11
Filing date: 2020-02-10
Publication date: 2020-10-27

Abstract

A method for analyzing iris images and retina images to diagnose diabetes and precursor symptoms, comprising: using the mobile terminal of the user to shoot the user eye image; a stage of image preprocessing for extracting the attention area; expanding the resolution of the concerned area; in order to reduce the data storage volume, the stage of compressing the concerned area data to be transmitted to the server by using the database form; a stage of user authentication by using the iris image; encrypting the image data; a stage of making artificial neural net learn stored data for diagnosing diabetes and its precursor symptoms; and predicting the diabetes prevalence by using the learned artificial neural network. Meanwhile, the stage of preprocessing the imaging image further comprises the following steps: a stage of extracting only the concerned field after carrying out gray level processing on the part which is not extracted as the concerned field; the stage of predicting the probability of diabetes further comprises: the stage of the diabetes type is predicted based on the location and shape of the lesion.

Description

Method for analyzing iris image and retina image by artificial intelligence technology

Technical Field

The invention relates to a method for analyzing iris and retina images by using an artificial intelligence technology in order to diagnose diabetes and precursor symptoms thereof, namely, the invention can accurately diagnose diabetes after iris and retina images are shot on a smart phone and provides services such as diabetes probability with credibility and the like.

Background

Diabetes mellitus is a metabolic disease characterized by hyperglycemia (elevated blood glucose concentration). Hyperglycemia is caused by a defect in insulin secretion or an impaired biological action thereof, or both. Hyperglycemia causes various symptoms and precursors, and glucose is eliminated by urination.

Diabetes is classified into type 1 diabetes and type 2 diabetes. Type 1 diabetes, the primary insulin-dependent diabetes, is commonly developed because insulin is not produced at all and is frequently found in children and adolescents. Type 2 diabetes develops because the cells cannot burn glucose efficiently due to a decrease in the function of blood glucose-lowering insulin in the body.

Currently, the severity of diabetes is usually determined by the following three examination methods. Namely a blood glucose test, an oral glucose tolerance test, a glycosylated hemoglobin (HbA1c) test. Among them, blood glucose test is the most representative and common diagnostic method for diabetes. The method comprises the following steps: a blood glucose level in the fasting state (blood glucose level measured after 8-hour fasting) of 126mg/dL or higher and a blood glucose level of 200mg/dL or higher, which is examined regardless of whether or not a meal is irregular, are considered to be diabetes. The oral glucose tolerance test is a test method for confirming a diagnosis of a patient whose blood glucose level is not high but falls out of a normal range. The checking method comprises the following steps: venous blood was collected after an empty stomach for 8 hours or more, then 300ml of water mixed with 75g of glucose was orally taken, and blood was collected for examination every time period after 30 minutes, 60 minutes, and 120 minutes. Then, when each blood glucose level is compared with the standard value and the blood glucose level measured 2 hours later is higher than 200mg/dL, it is judged that the disease is diabetes. The glycosylated hemoglobin (HbA1c) test is a standard for a diabetes diagnosis method and an evaluation index for blood glucose regulation, which are most recently used in the medical field. The examination result reflects the average value of blood sugar of 2-3 months, namely the blood sugar regulation degree. The range is between 4 and 6 percent, and more than 6.5 percent of the patients are diagnosed as diabetes. The blood sugar test results are classified into abnormal glucose tolerance, abnormal fasting blood sugar and diabetes.

Recently, there are a lot of services for diagnosing various diseases by using artificial intelligence technology, and the most common artificial intelligence disease diagnosis service in the past has the following structure: server-client architecture and the images employed for learning or diagnosing disease are MRI, PET, retinal images, etc. The service can not only find out the problem part, but also diagnose the relevant diseases with high accuracy after extracting the characteristics of each disease from the image through a Convolutional Neural Network (CNN).

In the conventional technology, although the accuracy of disease differentiation is high, a hospital needs to introduce expensive medical equipment such as MRI, PET, a special retina imager, and the like, and thus a patient needs to bear high medical diagnosis cost, and a relative visits the hospital to receive diagnosis. A disadvantage of this server-client architecture is that laggard countries such as africa, which lack medical equipment and doctors, are difficult to introduce and use. Statistically, 75% of diabetics are distributed in developing countries, and if inexpensive and simple to use medical devices can be developed, more patients can be helped to prevent diabetes.

In view of the above, there is a need to invent a diagnosis service that allows a patient to receive diabetes and its precursor symptoms at any time and any place without a specialized physician, and that is inexpensive but has high accuracy and reliability.

Disclosure of Invention

Problems to be solved by the invention

The invention aims to solve the defects of the existing artificial neural network disease diagnosis technology that the cost is high, the space restriction is caused, the convenience is lacked and the like, and realize the technology that the diabetes and the precursor symptoms can be predicted and diagnosed only by using a smart phone. If a special lens for iris recognition can be sleeved on the front lens of the smart phone to shoot the iris and retina of the user, so as to diagnose and predict diabetes and premonitory symptoms, the medical diagnosis cost can be greatly reduced, and the advantages of convenience in use, no space, time restriction and the like are achieved. However, due to the restriction of the hardware of the smart phone, the deep neural network cannot be applied to the smart phone, i.e. the accuracy of the neural network is not good. The invention aims to solve the problems and provides platform service for users in areas with backward medical conditions or unsmooth networks, and the platform service can accurately predict and diagnose diabetes and precursor symptoms thereof only by taking iris and retina pictures by using a mobile phone.

The technical problem to be achieved by the present invention is not limited to the above. Therefore, it is believed that those skilled in the art to which the present invention pertains will appreciate technical problems not related or related to the present invention from the following description.

Means for solving the problems

To solve the above problems, according to an embodiment of the present invention, a method for diagnosing diabetes and its precursor symptoms after analyzing iris and retina images using an artificial intelligence technique includes the following stages: an image receiving stage for obtaining an eye imaging photo shot by a user by using a mobile terminal; a stage of preprocessing the shot image for extracting a Region of interest (Region of interest) on the mobile terminal; expanding the resolution of the concerned area on the mobile terminal; on the mobile terminal, in order to reduce the data volume of the concerned area needing to be uploaded to the server, the stage of compressing through a compression library; a stage of using the iris image to authenticate the user on the mobile terminal; on the mobile terminal, encrypting the iris and retina image data stored in the server; and learning the artificial intelligent neural network stage capable of diagnosing diabetes and precursor symptoms on the server based on the stored data. The pre-processing stage of capturing the image further includes a stage of performing gray scale processing on an unnecessary region in the image and extracting only a region of interest, and the stage of predicting the incidence probability of diabetes further includes a stage of predicting the type of diabetes based on the position and shape of a lesion in the region of interest.

According to the embodiment of the invention, the method for analyzing the iris and retina images by the artificial intelligence technology has the following characteristics:

the stage of obtaining the imaging picture also comprises the stage of obtaining the iris and retina images by utilizing the special camera which can be sleeved on the front lens of the user mobile terminal.

The pre-processing stage of the imaging image also comprises a stage of extracting the attention area, namely extracting only iris and retina parts from the imaging image; a step of setting a standard axis for obtaining the rotation rate of the iris and retina areas from the attention area; and repositioning the iris and retina areas.

The above stage of setting the standard axis further comprises: after the eye curtain Segmentation (Segmentation), the stage of setting a standard axis for obtaining the rotation rate further comprises the following stages of repositioning the iris and retina areas: if the image of the iris and retina area is inclined, the iris and retina area is readjusted to 0 degree according to the rotation rate, and then the image is repositioned according to the interpolation method.

The step of expanding the resolution may further include a step of dividing the region of interest by a size of a specific small Block (Patch), applying an artificial neural network (CNN) to the divided image, and expanding the resolution of the region of interest based on a previously set resolution, and adding Zero padding (Zero padding) before applying the neural network (CNN), and when applying the neural network (CNN), using a Dense Block (Dense Block) and a Skip Connection (Skip Connection) method in order to relearn a feature of a previous stage.

The encryption phase also includes the phase of extracting codes from the iris and retina data stored in the server as encryption KEY and encrypting the codes.

The above stage of learning the artificial neural network further comprises: storing the data of the iris and the retina in a database; a stage of learning the above-mentioned stored data by using artificial neural network in order to diagnose diabetes and precursor symptoms and implement classification, detection and division functions at the same time; the artificial neural network comprises a Factorization (Factorization) method, a depth Separable Convolution (Depthwise Separable Convolvation) method and a point-by-point Convolution (Depthewise Separable Convolvation) method, and a stage of extracting a data Feature Map (Feature Map) from the data of the iris and retina images after classifying the data; a stage of extracting the attention field of the corresponding Anchor Box (Anchor Box) appointed in advance after applying RPN (region ProposalNet) to the Feature Map (Feature Map); after Pooling (Pooling) is carried out on the fields which are completed with learning, the fields with different sizes are uniformly adjusted to be the same size; classifying, detecting and classifying the positions of the diabetes mellitus and the precursor symptoms based on the same-size field.

The stage of predicting the incidence probability of diabetes further comprises the stage of predicting the incidence probability of diabetes according to the precursor symptoms after the artificial intelligence neural network is applied to the iris image and the retina image data.

According to the embodiment of the present invention, the method for analyzing iris and retina images by artificial intelligence technology further includes a step of providing services such as personal physique analysis, disease comprehensive diagnosis, diet management, etc. on the user terminal device after the step of predicting the incidence probability of diabetes.

Effects of the invention

As described above, several effects can be expected as follows. However, the effects obtainable by the present invention are not limited thereto.

Firstly, through the invention, a user can be free from the limitation of a place, and can accurately diagnose diabetes and premonitory symptoms on the smart phone through a well-learned neural network after the special iris camera is connected to the front camera of the smart phone to shoot the iris and retina images (high image quality is guaranteed). Since the diagnosis service is not a server-client network, it is possible to help patients in regions where there is no network or infrastructure lag to receive an inexpensive and accurate diagnosis service for diabetes and to prevent diabetes.

Second, according to the statistics of the International Diabetes Association (IDF) in 2015, 4 million 1500 million adults are diagnosed with Diabetes globally, and 3 million 1800 million adults have higher risk of developing Diabetes. In addition, 5-20% of the medical welfare budget of major countries around the world use the prevention and treatment of diseases related to diabetes. Since 75% of diabetics in the world live in developing countries, the diabetes prevention work in such laggard areas is very important. If the present invention successfully launches the diagnosis platform service, people can easily receive the diagnosis of diabetes at a low price anytime and anywhere. The method can greatly reduce the incidence of diabetes, reduce the medical welfare budget, use the medical welfare budget to the public welfare of residents and improve the life quality of the residents.

The technical problems to be solved by the present invention are not limited to the above-mentioned matters, and it is believed that those who have ordinary knowledge in the art related to the present invention will clearly understand the technical problems not mentioned herein or other related matters through the description.

Drawings

Fig. 1 is a conceptual diagram showing an overall system according to an embodiment of the present invention.

Fig. 2 is a sequence diagram of a method for analyzing iris and retina images by artificial intelligence techniques, according to an embodiment of the present invention.

Fig. 3 is a conceptual diagram of an artificial intelligence diagnostic service for analyzing iris and retina images and diagnosing diabetes and its precursor symptoms on a smartphone according to an embodiment of the present invention.

Fig. 4 is a block diagram of an overall system for artificial intelligence diagnosis for analyzing iris and retina images and diagnosing diabetes and its precursor symptoms on a smartphone according to an embodiment of the present invention.

Fig. 5 is a schematic block diagram of an artificial intelligence neural network for converting the iris and retina images of low Resolution in fig. 3 into Super Resolution images (Image Super Resolution) of high Resolution.

Fig. 6 is a block diagram of a database structure for storing the learning unit data in fig. 3.

Fig. 7 is a schematic block diagram of the artificial intelligent disease diagnosis neural network of fig. 4.

Fig. 8 is a structural diagram of additional functions in addition to the function of diagnosing diabetes and its precursor symptoms in the diagnosis stage of fig. 4.

Fig. 9 is a schematic diagram of the encryption of iris and retina image data in the encryption stage of fig. 4.

Detailed Description

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The following description, which is set forth in connection with the accompanying drawings, is intended as an illustration of an embodiment of the invention and is not intended to represent the only embodiment of the invention. All embodiments are to be construed in order to more fully convey to those skilled in the art all of the pertinent scope and definition of the invention.

However, in order to avoid the problem of concept ambiguity, a part of the configuration and the device will not be explained, and the core functions of the configuration and the device will be mainly explained. Meanwhile, in all the following, the same constituent elements will be described with reference to the same drawing symbols.

Throughout the following description, if a description of a certain part "to include" or "comprising" appears, unless otherwise noted, it means "to add," that is, to add the aforementioned constituent element without excluding the former constituent element.

The term "…" as used herein means a structural unit including at least one or more functions or actions, and may be one piece of hardware or one piece of software, or a combination of one piece of hardware and one piece of software. Unless expressly noted otherwise, all references herein to "a" or "an", "one" and the like are intended to include all singular and negative meanings.

Unless otherwise noted, specific terms used herein are words used for the convenience of the reader's understanding of the contents of the description, and therefore, all technical and scientific terms have the same or similar meaning as that of terms used in ordinary times, that is, the meaning of the terms is not difficult for a person having common knowledge in the field relevant to the present invention to understand. Meanwhile, the specific terms described above may be used interchangeably with other expressions without departing from the scope of the present invention.

Hereinafter, the most suitable embodiment of the present invention will be described in detail with reference to the accompanying drawings. The following description, which is set forth in connection with the accompanying drawings, is intended as an illustration of an embodiment of the invention and is not intended to represent the only embodiment of the invention.

Fig. 1 is a conceptual diagram showing the entire system of the real-time scenario of the present invention.

In fig. 1, the mobile terminal of the user may be an electronic device such as a smart phone, a tablet computer, or a notebook computer. The mobile terminal comprises a camera, and the camera can comprise a special lens for iris recognition or be externally connected with the special lens for iris recognition. That is, when the user takes a picture of the eye using the mobile terminal, the user can obtain an image or video of a specific part of the eye including the iris, the retina and the like with high resolution. The high-definition eye images shot by the mobile terminal device are used for diagnosing diabetes, precursor symptoms of diabetes and the like. Hereinafter, the diagnosis process of diabetes will be described in detail with reference to the accompanying drawings.

Fig. 2 is a sequence diagram of a method for analyzing iris and retina images by artificial intelligence technology for diagnosing diabetes and its precursor symptoms according to an embodiment of the present invention.

Referring to fig. 2, the method for analyzing iris and retina images by artificial intelligence technique for diagnosing diabetes and its precursor symptoms according to the embodiment of the present invention includes a stage of acquiring (S11) an eye photograph through a mobile terminal of a user; an image preprocessing process (S12) is performed on the user' S mobile terminal to extract a Region of Interest (Region of Interest). When extracting the attention area, only the necessary attention area can be extracted after performing gray scale processing on unnecessary parts in the image captured in advance, so that the calculation amount is reduced and the diagnosis accuracy is improved. Furthermore, the method will further comprise: expanding the resolution of the extracted attention area on the mobile terminal of the user (S13); a step (S14) of compressing data by using a program library mode in order to reduce the data volume of the concerned area transmitted to the server on the mobile terminal of the user; a step (S15) of authenticating the user on the mobile terminal of the user through the iris image; a step (S16) of encrypting the data of the iris image and the retina image which need to be stored in the server on the mobile terminal of the user; performing, on the server, a neural network self-learning process for diagnosing diabetes and precursor symptoms based on the stored data (S17); and a stage of predicting the incidence probability of diabetes by using the neural network which is completed with the learning on the mobile terminal of the user (S18). In addition, the type of diabetes can be predicted based on the location and shape characteristics of the lesion in the area of interest. Diabetes can be classified into type 1 diabetes and type 2 diabetes according to the difference in insulin function, and can be classified into impaired glucose tolerance, impaired fasting glucose, and the like according to the blood glucose concentration. Hereinafter, the proceeding process of the above-mentioned stages S11 to S18 will be described in detail with reference to fig. 3.

Fig. 3 is a technical conceptual diagram of iris and retina image analysis and artificial intelligence diagnosis of diabetes and its precursor symptoms on a smartphone according to an embodiment of the present invention.

Referring to fig. 3, an artificial intelligence diagnosis service 100 for diabetes and its precursor symptoms by analyzing iris and retina images on a smartphone according to an embodiment of the present invention will pass through an image capturing part 101; an image preprocessing unit 102; a neural net learning unit 103; the disease diagnosis unit 104 provides.

The image capturing unit 101 is a part for capturing an eye photograph with a smartphone, and needs to capture an image by applying an iris recognition camera to a front camera of the smartphone and acquire the captured image (S11). Although the camera built in the smart phone can be used for shooting the photos of the eyes, the external iris recognition special lens can shoot more vivid and clear iris and retina images through a single focus and an infrared LED function, and the images of the surrounding environment cannot be received. The clearer the shot image is, the better the iris recognition and the artificial intelligent diagnosis of diabetes and its precursor symptoms are, without being affected by light. The captured image differs depending on the color of the iris.

The image preprocessing unit 102 extracts only iris and retina regions necessary for diagnosis from the captured eye image. First, only the Region of Interest (RoI: Region-of-Interest) needs to be extracted from the eye image using the JPEG2000 library. The attention area here means an area of a minimum range extracted for extracting the iris and retina areas. The reason for extracting the attention area is to reduce the amount of calculation required for extraction and replacement of the iris and retina. Then, after the extracted attention area is divided into eye curtains (Segmentation), the iris and retina areas are extracted from the extracted attention area, and then the external part of the eye curtains is processed in a gray scale mode. In the image subjected to the gradation processing, the front and rear edges of the eye curtain are connected by straight lines, and then the rotation rate is obtained with 0 degrees as a reference by setting a vertical straight line intersecting the straight line as a standard axis. Subsequently, the extracted iris and retina images are reset by a method such as a rotation (rotation) interpolation method with the rotation rate as a reference. The reason why the iris and the retina are repositioned is that the location of lesions such as coloration, pits, cracks, lines, etc. of the iris is very important information in diagnosing the premonitory symptoms of diabetes through the iris and the retina. That is, since the position and the feature of the eye mean that the specific organ is a problem, the iris and retina images must be accurately adjusted with 0 degrees as a reference in order to accurately discriminate the precursor symptom (S12). In addition, if the resolution of the iris and retina images is too low, the amount of data required for diagnosis is reduced, thereby causing the accuracy of disease diagnosis to become low. At this time, the low-resolution image may be adjusted to the high-resolution image using the resolution-adjustable artificial neural network (S13). That is, the Neural Network may divide the iris and retina images into specific patches (Patch) and then expand the resolution of the separated images by a Convolutional Neural Network (CNN). Due to the expansion of the original image in specific small blocks, the image can be restored by a method of merging the output images. In the above-mentioned neural network for converting the low resolution into the high resolution, in order to enhance the problems of feature propagation and reuse, a high density Dense Block (Dense Block) method may be applied at a previous stage of extracting the feature points. Meanwhile, in order to enable the neural network to more correctly learn the information of the image boundary part, a Skip Connection method for connecting the input image with the output pre-stage image is applied. By this stage, images required for diagnosing diabetes and its precursor symptoms can be obtained from the iris and retina images. Meanwhile, the data of the iris and retina images can be remotely transmitted through a network. However, if the original file is transmitted in the process, the original file is too large in capacity, and a JPEG2000 library is used to compress the image (S14) and transmit the image to the network. Meanwhile, user authentication may be performed through the iris image (S15).

The neural network learning unit (103) is composed of a database for storing iris and retina image data and a neural network model for learning the data. Although there are many types of databases, an Oracle database having high security is used. According to the personal information protection method, information such as biometric information, identification information (identification number, passport number, driver's license number, foreign person login number), and password stored in the database must be encrypted. The iris and retina data belong to the biological information, and must be encrypted (S16) and stored. The encryption method for storing information includes various methods such as an application self-encryption method, a DB server encryption method, a DBMS self-encryption method, a DBMS encryption function calling method, and an operation system encryption method. Since each encryption method has different influences on its performance, it is necessary to select a DB encryption scheme in consideration of encryption characteristics, advantages and disadvantages, restriction conditions, and the like, in accordance with a construction environment. The process of setting the encryption KEY is very important when encrypting. The encryption KEY uses codes extracted from iris and retina image data, and is encrypted by the encryption KEY and then stored in a database. Next, an artificial intelligence neural network that can diagnose diabetes and its precursor symptoms can be learned using the saved iris and retina image data (S17). The artificial intelligence neural network for diagnosing diabetes and its precursor symptoms through iris and retina images needs to classify (Classification)/detect (Detection)/classify (Segmentation) the diabetes precursor symptoms after workAnd then the study is carried out. Therefore, the artificial neural network of the present invention applies a Pre-learning (Pre-training) method to enable more accurate classification in the learning process. The artificial neural network in the classification stage occupies most of the field of the whole neural network, so that the artificial neural network needs to be lightened as much as possible and can be used on a mobile terminal. There are various methods for reducing the weight of the artificial intelligence neural network, for example, a factorization method in which the convolution operation is performed by dividing a large convolution kernel of 5 × 5 into two convolution kernels of small size of 3x3 and 3x3, that is, the amount of operation can be reduced by 20 to 30%; in order to effectively process the process, the correlation and the spatial correlation between channels can be processed separately by combining and adopting a method for reducing the correlation and the spatial correlation in comparison with the conventional Convolution method (Convolution)

In the discrete Convolution (DepthwiseSeparateable Convolition) method of the operand and the Point-by-Point Convolution (Pointwise Convolition) method of processing only the channel correlation without considering the spatial correlation in the general Convolution (Convolition) hierarchy, and expanding or compressing the feature map, the artificial neural network is reduced in weight. After the classification method is learned first (Pre-training), a method of finding a premonitory symptom position from the extracted feature map and representing it using a Bounding Box (Bounding Box) can be learned. At this time, since the borders (Bounding boxes) are different in size, it is necessary to uniformly adjust the borders to the same size by a rounding (RoI firing) method, and then learn a method of Classification (Classification)/Detection (Detection)/division (Segmentation) of the diabetic precursor symptoms through a Convolutional Neural Network (CNN). For Classification that is not a precursor symptom, a more accurate method of Classification (Classification)/Detection (Detection)/division (Segmentation) can be learned by giving a return to the "Penalty" process.

The disease diagnosis unit (104) can detect diabetes and precursor symptoms using the previously processed iris and retina images, using the artificial neural network learned by the neural network learning unit (103), and finally diagnose diabetes (S18).

Fig. 4 is a block diagram of an artificial intelligence diagnostic system for diagnosing diabetes and its precursor symptoms after analyzing iris and retina images on a smart phone according to an embodiment of the present invention.

Referring to fig. 4, an artificial intelligence diagnosis system 200 for diagnosing diabetes and its precursor symptoms after analyzing iris and retina images on a smart phone according to an embodiment of the present invention is composed of a user side 201, a platform side 202, a security side 203, a server side 204, a data server side 205, and a neural network learning server side 206. The user terminal 201 may be an individual or a hospital or a medical institution capable of providing iris and retinal disease diagnosis services. The platform 202 needs to provide services on the mobile phone side, and therefore adopts systems such as android, IOS, WINDOWS, and the like. The secure terminal 203 will be divided into an iris recognition part for authenticating the user who needs to receive the diagnosis service and a part for encrypting iris and retina image data. The server 204 is divided into a diagnosis service for diabetes and precursor symptoms, a service for analyzing the individual constitution according to iris features, a service for predicting the incidence probability of future complications according to extracted precursor symptoms, a service for providing individual healthy diet and exercise according to the diagnosis results of constitution and precursor symptoms, and the like, based on the artificial intelligence disease diagnosis neural network for completing learning. The data server 205 may store iris and retina image data required for artificial neural network learning. On the neural network learning server 206, the artificial intelligence neural network can be learned according to the stored iris and retina image data.

Fig. 5 is a schematic diagram of an artificial intelligence neural network 300 for enhancing the iris and retina images of fig. 3 from low resolution to high resolution (Image super resolution). If the resolution of the iris and retina images is low, the data basis that can be referred to is too small, and the diagnosis accuracy is reduced. In order to increase the low resolution to the high resolution at this time, an artificial intelligence neural network is used. That is, the input low-resolution iris and retina images are divided into specific Patch (Patch) sizes, and only the divided small images are input and reconstructed by a convolutional neural network to output an image. At this time, if a general convolutional neural network is used, the pixel information of the edge portion of the image cannot be fully utilized due to the reduction in the size of the feature map of the output. Therefore, it is necessary to accurately predict pixel information of an edge portion of an image while maintaining a constant size of feature maps of all layers by a Zero Padding (Zero Padding) method before performing convolution operation. Then, in the learning process, a Dense Block (Dense Block) method for re-learning the features of the previous stage can be adopted to reduce the gradient disappearance (gradientdisappearance) phenomenon, and a skip connection (skip connection) method used in a residual neural network (Resnet) is used to enable the learning to be completed more quickly and data of the input image edge part to be processed better. At this time, since the output image is divided into small blocks of a specific size, it can be converted into a high-resolution picture through a Merge (Merge) process.

Fig. 6 is a diagram showing a configuration of a database 400 for storing iris and retina image data in the neural network learning unit shown in fig. 3. In order to store iris and retina image data in the database, the following work is required. First, the server processor in the database makes room for the corresponding SQL to work. The processor will search the library cache for multiple handlers (handlers) with SQL related information, use the handlers if their programs are found, and perform a Hard Parsing of the SQL syntax if not found (Hard matching). The Hard Parsing (Hard Parsing) process is divided into syntax analysis (SQL word validation, SQL syntax validation), meaning check (checking whether an icon/field exists or not), query conversion (the optimizer redoes SQL to improve performance), authority validation, Lock Type validation in DML (data management language) work, generation of an execution plan and syntax analysis tree, and a log-in record (record of log-in information occurring in DML). Then, after storing the relevant data in a specific buffer, recording the data in the buffer in the relevant file after executing Commit (Commit) work. The data of the iris and retina images can be stored in the database through the above process.

Fig. 7 is a schematic diagram of the disease diagnosis neural network based on the artificial intelligence technique illustrated in fig. 4. The artificial intelligence disease diagnosis neural network illustrated in fig. 4 of the present invention is configured by a disease classification unit 501 and a disease position detection unit 502. The disease classification unit 501 may apply a convolutional neural network for classifying symptoms of diabetic precursors (pancreatic lesions, neutral fat, pressure rings, etc.) from iris and retina images. That is, after various features of the precursor symptom are extracted by the convolutional neural network of the disease classification unit 501, a Feature Map (Feature Map) output from the last output layer is processed by the disease position detection unit 502. The disease position detection unit 502 may perform the detection and division of the precursor symptom part based on the Feature Map (Feature Map) extracted by the disease classification unit 501. In order to extract the attention area part related to the precursor symptom in the feature map, a candidate area generation Network (RPNRegion pro-active Network) is needed, and at this time, the attention area can be extracted according to an anchor box (anchor box) which is specified in advance. Here, the attention area may be a detection area for marking a precursor symptom from an iris or retina image. Since the extracted attention areas all vary in size, they cannot be used in convolutional neural networks that require execution at a fixed size. In this regard, various sizes of fields of interest will be converted to the same size by a field of interest (RoI Pooling) layer. After that, after performing operations such as classification, detection, and division of the precursor symptoms in parallel in the converted attention area, it is output where the precursor symptoms are expressed in the iris and retina images. Then, the probability of onset of diabetes can be predicted from the result value finally output from the neural network based on various precursor symptoms appearing in the iris and retina of the diabetic patient.

Fig. 8 is a structural diagram of a platform part, i.e., additional service functions in addition to the diagnosis service function of diabetes and precursor symptoms in the artificial intelligence diagnosis system illustrated in fig. 4 of the present invention.

As shown in fig. 8, the additional service includes an image capturing unit 601, a data server unit 602, a physical diagnosis unit 603, a disease diagnosis unit 604, a complication diagnosis unit 605, and a diet management unit 606. In the image capturing unit 601, the user can capture the iris and retina photographs by applying the special iris authentication camera to the front camera of the smartphone. The data server 602 may store data for encrypting the captured iris and retina images. The physical constitution diagnosing unit 603 may analyze and compare the individual iris and retina information stored in the data server, and then determine and classify the individual physical constitution attributes of the sun, taiyin, shaoyang, shaoyin, and the like. The disease diagnosis section 604 can diagnose diabetes and its precursor symptoms from the captured iris and retina image data. The complication diagnosing unit 605 can predict and diagnose the related complications based on the precursor symptom information given by the disease diagnosing unit 604. The diet management unit 606 provides an individual best-tailored diet menu based on the physical attributes classified by the individual physical diagnosis unit 603 and the various precursor symptoms diagnosed by the disease diagnosis unit 604.

Fig. 9 is a schematic diagram of the encryption and storage method of iris and retina image data in the security unit shown in fig. 4 according to the present invention. When storing data, the encryption method used is as follows: an encryption method 701 for the application itself, a DB server encryption method 702, a DBMS encryption method 703, a call 704 for the DBMS encryption function, an operation system encryption method 705, and the like. The encryption system 701 of the application itself is set as an encryption/decryption module in the API library state on the server of each application, and operates by calling the encryption/decryption module from inside the application, and therefore, there is no influence on the DB server. In the DB server encryption method 702, since the encryption/decryption module is installed In the DB server, that is, since the encryption/decryption module connected to the DBMS is called by Plug-In (Plug-In), there is an advantage that it is easy to use since it is almost unnecessary to modify an application program, but there is a possibility that an overload state of the DB server occurs because it is necessary to create a View (View) corresponding to an original DB Schema (Schema) and add a Table work requiring encryption. The DBMS encryption method 703 is a method for performing encryption and decryption operations using an encryption function of a memory in the DBMS. Since this approach only requires processing at the core (kernel) level of the DBMS itself, there is little need to change the original application or DB Schema (Schema), but server overload may also occur. The call 704 for the encryption function of the DBMS is a way for the DBMS itself to provide an API that can perform the encryption and decryption functions and perform this work on the application using this function. This approach requires modification of the application and therefore the DB server may be overloaded. The operation system encryption system 705 is a system for performing encryption and decryption by using a call function of the input/output system generated from the OS. Although there is no need to change an application or a DB Schema (Schema), in order to encrypt the entire DB file, an overload phenomenon of the file server and the DB server may occur.

In addition to the above embodiments, the present invention will provide the following services: accumulating big data capable of judging the incidence possibility and the progression degree of diabetes according to the expression position and the shape of the diabetes and the related pathological changes on the eyes; learning and determining the incidence probability and the progression degree of diabetes according to the position and the shape of the lesion field based on the accumulated big data; the patient is informed of the need for additional blood glucose level tests, oral glucose tolerance tests, glycated hemoglobin (HbA1c), and the like in real time by a mobile phone terminal based on the probability of onset of diabetes and the degree of progression of diabetes. Meanwhile, if the diabetes degree is abnormally increased in the near future, the mobile phone terminal informs the patient of the need of paying attention and suggesting to go to a hospital for examination in real time. The method for notifying the patient in real time through the mobile phone terminal of the user can use modes of floating window, short message sending and the like.

In addition, the method can be written into a program capable of running on a computer, namely the program can run on a common model computer which can drive the program by a medium readable by the computer. Also, the data structures used in the above schemes may be recorded on the computer-readable medium in a variety of ways. Computer-code-storing, computer-readable media that can implement the methods of the present invention include: magnetic storage media (e.g., ROM, floppy disks, hard disks, etc.), optical read-only media (e.g., compact disks, DVD disks), and so forth.

Those skilled in the art who have the knowledge of the technology relating to the embodiments of the present invention will understand that the objects can be achieved by changing the form without departing from the essential characteristics. The various aspects described above are therefore not to be interpreted in a limiting sense, but are to be interpreted and considered in an illustrative sense. The scope of the invention is indicated in the appended claims rather than in the detailed description of the invention, and all differences within the scope will be construed as being included in the present invention.

Industrial applicability

The invention is to diagnose diabetes and its precursor symptom, and the method for analyzing iris and retina image by artificial intelligence technique is used as the medical means for applying iris and retina image to diagnose diabetes and its precursor symptom, belonging to the invention scheme applied to the related industry field as medical diagnosis means.

Claims

1. A method for analyzing iris and retina images by artificial intelligence technique for diagnosing diabetes and its precursor symptoms, which comprises the following steps:

shooting the user eye image on the user mobile terminal;

on the mobile terminal of the user, in order to extract the concerned area, the stage of pre-processing the shot image;

expanding the resolution of the extracted attention area on the user mobile terminal;

on the user mobile terminal, in order to reduce the data quantity of the concerned area to be transmitted to the server, the stage of compressing by using the database;

a stage of user identity authentication by using the iris image on the user mobile terminal;

on the mobile terminal of the user, encrypting the iris and retina image data to be stored in the server;

learning the artificial neural network for diagnosing the diabetes and its precursor symptoms on the server based on the stored data; and

on the mobile terminal of the user, predicting the incidence probability of diabetes by utilizing the learned artificial neural network;

the aforementioned stage of preprocessing the captured image further includes: a stage of extracting the concerned area only after carrying out gray scale processing on the part which is not extracted into the concerned area in the shot image;

the stage of predicting the incidence of diabetes further comprises: and predicting the stage of the diabetes according to the position and the shape of the lesion in the attention area.

2. The method for analyzing iris and retina images using artificial intelligence technique according to claim 1,

in the stage of obtaining imaging image, the special lens for iris identification which can be used on the mobile terminal of user is used to extract the iris and retina image.

3. The method for analyzing iris and retina images using artificial intelligence technique according to claim 1,

the preprocessing stage of the imaging image further comprises:

extracting attention areas including iris and retina areas from the imaging image;

setting reference axis for obtaining rotation rate of iris and retina area in the attention area;

extracting only the iris region and the retina region in the attention region; and

rearranging the iris and retina field stages based on the rotation rate.

4. The method for analyzing iris and retina images using artificial intelligence technique according to claim 3,

the step of setting the reference axis further comprises: a step of setting a reference axis for obtaining the rotation rate after dividing the eyelid;

the stage of rearranging the above mentioned iris and retina fields will also comprise: and a rearrangement step of rotating the attention area in the vertical direction to a tilt of 0 ° or by interpolation using the reference axis.

5. The method for analyzing iris and retina images using artificial intelligence technique according to claim 1,

the stage of expanding the resolution additionally comprises: a stage of dividing the concerned area into small blocks with specific sizes, applying a neural network to the divided images, and expanding the resolution of the concerned area to a preset resolution;

before the convolution neural network is applied, a zero filling method is added;

when the convolutional neural network is applied, in order to relearn various feature information of the previous stage, a dense block method is used, and a step of a jump connection method is applied.

6. The method for analyzing iris and retina images using artificial intelligence technique according to claim 1,

the encryption stage additionally includes: and encrypting the code value extracted from the iris and retina image data stored in the server by using the encryption key.

7. The method for analyzing iris and retina images using artificial intelligence technique according to claim 1,

the learning phase of the artificial neural network further comprises:

storing the data of the iris and retina images into a database;

a stage of learning the stored data by using an artificial neural network in order to synchronously complete the diagnosis work of classifying, detecting, dividing and the like of the diabetes and the precursor symptoms thereof;

the artificial neural network includes: a factorization method, a depth separable convolution structure, and a point-by-point convolution method.

8. The method for analyzing iris and retina images using artificial intelligence technique according to claim 7,

the above-mentioned stage of learning with an artificial neural network will also include,

classifying the iris and retina image data, and extracting the characteristic diagram of the diabetes and the precursor symptoms;

extracting the attention area corresponding to the anchor box set in advance after applying RPN on the characteristic diagram;

after the field which has been learned is applied with the pooling function, uniformly converting the fields with different sizes into the fields with the same size; and

classifying, detecting and classifying the diabetes and the precursor symptom part based on the same size field.

9. The method for analyzing iris and retina images using artificial intelligence technique according to claim 1,

the stage of predicting the incidence probability of diabetes further comprises:

and applying the artificial neural network to the iris and retina image data to predict the probability of the diabetes mellitus patient according to the precursor symptoms.

10. The method for analyzing iris and retina images using artificial intelligence technique according to claim 1,

after the stage of predicting the incidence probability of diabetes, the method additionally comprises the following steps:

providing personal customized constitution diagnosis service, complication diagnosis service, diet management service, etc. on the mobile terminal of the user.