WO2018222136A1

WO2018222136A1 - Image processing method and system

Info

Publication number: WO2018222136A1
Application number: PCT/SG2018/050262
Authority: WO
Inventors: 库兰塔扬⋅巴瓦尼; 聂文伟; 亚申⋅古米朗⋅亨德拉⋅斯蒂亚万⋅穆罕默德; 王嘉慧; 王顺吉; 范虎登; 黄诗敏; 王俊文
Original assignee: 正凯人工智能私人有限公司
Priority date: 2017-05-30
Filing date: 2018-05-28
Publication date: 2018-12-06
Also published as: CN109348732A

Abstract

The present invention relates to an image processing method and system. The method comprises: receiving an initial user file from a user terminal, the initial user file comprising user data and a user image; loading the initial user file to a server, the server having stored therein reference images and a calculation model, the reference images comprising a plurality of reference images having classification code 1 and a plurality of reference images having classification code 2; comparing, by means of the calculation model, the user image and the reference images to determine whether the classification code of the user image is 1 or 2; storing the classification code of the user image in the initial user file to generate an updated user file; and, sending the updated user file to the user terminal.

Description

Image processing method and system

The present invention relates to an image processing method and system. In particular, the present invention relates to a retinal fundus image classification method and system. Background technique

[0002] The main causes of eye diseases, blindness and visual impairment include cataracts (47.9 %), glaucoma (12.3%), age-related macular degeneration (A bowel) (8.7%), corneal opacity (5.1%), Diabetic retinopathy (4.8%). The prevalence of these diseases is on the rise, partly due to sedentary lifestyles and an aging population, which brings many metabolic diseases associated with these eye diseases (such as diabetes, high blood pressure, high cholesterol (hyperlipidemia) and age Related diseases. If detected and treated in time, the development of the above eye diseases to blindness is mostly preventable.

[0003] A cataract is an eye disease that causes blurred vision or fog-like vision due to opacity of the lens in the patient's eye. Cataracts mainly occur in the eyes of the elderly because it is believed that the formation of cataracts is due to the deterioration of protein fibers in the lens. This results in the formation of a block which creates a cloud zone in the lens. If left untreated, cataracts can cause permanent vision loss.

[0004] According to a 2010 survey, 45.9 million people worldwide affected their vision due to cataracts. Of the 4,5.9 million people, 10.8 million were due to cataract blindness. In Asia alone, 3,5 million people suffer from cataracts and 7.27 million people lose vision due to cataracts. In 2010, 2.5 million people in China suffered from cataracts and are expected to increase by 400,000 a year. This is mainly due to the large number of elderly people aged 65 and over. [0005] Glaucoma refers to a group of eye diseases that refer to a slow deterioration of the optic nerve located in the posterior part of the eye. This is often due to the accumulation of fluid pressure in the eye. This causes a circulatory blockage of the fluid called the aqueous humor, which usually flows naturally out of the eye. This blockage may be due to genetic factors or chemical damage to the eye.

[0006] There are several types of glaucoma: open angle glaucoma, angle closure glaucoma, and congenital glaucoma. [0007] Open-angle glaucoma (0AG) is a common type of glaucoma that occurs when the angle of the eye (where the iris meets the cornea) is normal, however, damage to the ability of the eye to drain causes fluid to accumulate, which causes internal pressure The increase, while causing optic nerve damage.

[0008] On the other hand, angle-closure glaucoma (ACG) is not common. This occurs when the pressure inside the eye is suddenly increased due to the angle between the iris and the cornea being too narrow, thus interrupting the drainage of the liquid.

[0009] Congenital glaucoma is a rare form of glaucoma that results from poor or incomplete development of the patient's fetal ocular drainage tube.

[0010] As is apparent from its name, diabetic retinopathy (DR) is the only condition that occurs only in diabetic patients. This disease causes progressive damage to blood vessels in the retina over time. This is mainly due to the high amount of sugar (which is present in the blood of diabetic patients), which causes tiny blood vessels in the retina to leak liquid or bleed, as well as causing progressive damage to the blood vessels of the eye. This causes visual damage such as turbid or blurred vision. In the advanced phase of the disease, neovascularization occurs, which further damages the retinal cells. If left untreated, it can lead to blindness. [0011] Disease progression in diabetic retinopathy is classified into four distinct phases: mild, moderate, severe, and proliferative. In the first stage (mild), there is swelling of tiny blood vessels in the retina. In the second phase (medium), the blood vessels in the retina continue to swell, destroying its structure, causing the blood vessels to lose their ability to transport blood. During this phase, it causes a change in the shape of the retina, which may lead to diabetic macular edema (Li E). In the third phase (severe), most of the blood vessels are blocked, resulting in a decrease in blood supply to the retina. When the retina is deprived of blood supply, growth factors are released into the formation of new blood vessels. In the final stage (proliferation), the sustained release of growth factors causes new fragile blood vessels to grow, causing easy bleeding and leakage, which will eventually lead to retinal detachment.

[0012] According to a 2010 clinical study, more than 371 million adults worldwide are estimated to be affected by diabetes [9, 14]. In Asia alone, an estimated 226 million adults are affected by diabetes [8, 9, 14]. Countries such as India and China have the largest number of patients, with 65.1 million and 113.9 million people affected by diabetes. In addition, 1,200,600,000 of the 703 million adults have DR [9, 14]. In China alone, 5,615 million people suffer from DR [9, 14]. This is a global epidemic and the number of patients is increasing year by year.

[0013] In 2010, an estimated 60.4 million people worldwide suffered from glaucoma, of which 4,4.7 million had 0AG and 1,5.7 million had ACG [5]. In Asian countries, including China, India, Japan and Southeast Asia, 34.4 million people have glaucoma, of which 20.9 million have 0AG and 13.5 million suffer from ACG.

[0014] Under the prior art, timely or early detection of many eye diseases has proven to be difficult to achieve, especially in developing countries with large rural populations scattered throughout regions such as China, Russia and India. [0015] In 2010, the International Council of Ophthalmology noted that there were only 32 million ophthalmologists worldwide. This is equivalent to a ratio of 1 million people per 6,400 ophthalmologists. However, based on a forecast study for 2020, Southeast Asia will have a population of 1.63 billion, but only 13 3,300 ophthalmologists, or 1 ophthalmologist to 122,000 people. These data suggest that more ophthalmologists are needed, especially in developing countries.

[0016] The following is a 2012 specific data according to the International Council of Ophthalmology:

(source:

Http: //www.icoph.org/ophthalmologists-worldwide, html )

[0017] It can be seen from the above data that the number of ophthalmologists in Indonesia and Thailand is between 1000 and 1300. Compared with more than 60 million people, their ophthalmologists are very scarce. Therefore, people who need eye diseases are needed. Faced with a higher risk of unconditional medical treatment.

[0018] At the same time as the ophthalmologist's lack, populations in rural and remote areas are also facing difficulties in accessing eye care services because ophthalmologists are concentrated in large cities. This has made it necessary to go to big city hospitals and ophthalmologists who queue up for treatment in big city hospitals.

[0019] According to statistics from the National Eye Centre of Singapore (SNEC), even in diabetic patients, only one-third of those who do eye exams actually need an ophthalmologist in a timely manner. Heal. This means that ophthalmologists spend a lot of time and resources checking patients who do not actually need emergency medical treatment, and time and resources could have been better used to examine patients who really need treatment.

[0020] Therefore, there is a need for an economical and more convenient solution, especially for users in rural or remote areas, so that users with healthy eyes can save unnecessary travel to an ophthalmologist. At the same time, users with latent eye diseases can know their eye health in time to further screen or treat the eyes. Summary of the invention

[0021] An embodiment of the present invention provides an image processing method, the method comprising: receiving an initial user file from a user end, the initial user file including user data and a user image; loading the initial user file into a server The server stores a reference image and a calculation model, the reference image includes a plurality of reference images having a classification code of 1 and a plurality of reference images having a classification code of 2; using the calculation model, the user image and the The reference images are compared to determine the classification code of the user image as one of 1 and 2; the classification code of the user image is stored in the initial user file to generate an updated user file; and the update user file is sent to the client.

[0022] Preferably, the update user file includes a color mark, and the color mark includes a green mark corresponding to the classification code 1 and a red mark corresponding to the classification code 2.

[0023] Preferably, if the classification code of the user image is determined to be 1, the method further comprises storing the first follow-up code in an initial user file to generate the updated user file. [0024] Preferably, if the classification code of the user image is determined to be 2, the method further comprises storing the second follow-up code in an initial user file to generate the updated user file. [0025] Preferably, the method further comprises storing the user image of the determined classification code as a reference image in the server.

[0026] Preferably, the method further includes: before receiving the initial user file from the user end, loading the reference image in the server, training the artificial intelligence engine based on the reference image, and using the artificial intelligence engine The calculation model is constructed.

[0027] Preferably, the artificial intelligence engine comprises at least one of a machine learning algorithm and a deep learning algorithm or a combination of algorithms.

[0028] Preferably, the artificial intelligence engine comprises at least one of a support vector machine (SVM), a gradient booster (GBM), a random forest, and a convolutional neural network.

[0029] Preferably, the method further comprises training the artificial intelligence engine based on the user image and the determined classification code.

[0030] Preferably, the user image is a retinal fundus image of the user, comprising at least 3000*2000 pixels, a fundus region having at least 45 degrees, and a pixel resolution of at least 150 dp i.

[0031] Preferably, the user image is a retinal fundus image of the user, wherein comparing the user image with the reference image further comprises comparing using at least one of the following eye state determination elements:

(a) multiple retinal vessels present in the image;

(b) the cup-to-disk ratio is less than 0.3;

(c) Missing at least one of the following elements:

(i) visible media turbidity; (ii) a diabetic retinopathy indicator comprising at least one of blotting-like bleeding, microaneurysms, and hard exudates;

(iii) diabetic macular degeneration;

(iv) macular edema;

(v) exudates near the macula;

(vi) exudate on the macula;

(vii) laser scars;

(iii) cataract;

(ix) glaucoma;

(χ) diabetic retinopathy; and

(xi) Age-related macular degeneration comprising at least one of a plurality of large drusen, a marked atrophy with a significant area of hypopigmentation, and a choroidal neovascular membrane, wherein age-related macular degeneration is indicative of atrophic At least one of neovascularization and exudation;

At least one of the eye state judging elements may not be a judging element of the classification.

[0032] Another embodiment of the present invention is an image processing system, the system includes: a server and a client connected to the server, the server stores a reference image and a calculation model, where the reference image includes multiple A reference image with a classification code of 1 and a plurality of reference images with a classification code of 2; a client for generating an initial user file, the initial user file including user data and a user image. After receiving the user file, the server starts the computing model, compares the user image with the reference image, and determines the classification code of the user image as one of 1 and 2, and classifies the user image. The initial user file is saved to generate an updated user file, and the updated user file is sent to the client.

[0033] Preferably, the update user file includes a color mark, and the color mark package A green mark corresponding to the classification code 1 and a red mark corresponding to the classification code 2 are included.

[0034] Preferably, the system further comprises an artificial intelligence engine trained based on the reference image, the artificial intelligence engine for constructing the calculation model.

[0035] Preferably, the artificial intelligence engine comprises at least one of a machine learning algorithm and a deep learning algorithm or a combination of algorithms.

[0036] Preferably, the artificial intelligence engine comprises at least one of a support vector machine (SVM), a gradient elevator (GBM), a random forest and a convolutional neural network.

[0037] By providing an economical and convenient solution for classifying a user's retinal fundus image, the present invention has the potential to significantly reduce preventable blindness and visual impairment worldwide in developing and developed countries. According to an embodiment of the present invention, a user with a healthy eye can save unnecessary time and resources for consulting an ophthalmologist, and a user with a potential eye disease can know his or her eye condition in time to arrange an appointment with an ophthalmologist in time. Ophthalmologists can also schedule their limited time and resources to check who really needs medical attention. DRAWINGS

[0038] Hereinafter, embodiments of the present invention will be described in detail by way of examples, with reference to the accompanying drawings in which:

1 is a schematic diagram showing a construction model of an image processing method and system according to an embodiment of the present invention.

2 is a retinal fundus of an image processing method and system according to an embodiment of the present invention; Image loading diagram.

3 is a schematic diagram of retinal fundus image classification according to an image processing method and system according to an embodiment of the present invention.

4 is a flow chart showing the steps of constructing a calculation model of an image processing method and system according to an embodiment of the present invention.

5 is a flow chart showing the steps of retinal fundus image loading in an image processing method and system according to an embodiment of the present invention.

6 is a flow chart showing the steps of retinal fundus image classification according to an image processing method and system according to an embodiment of the present invention. 7 is a schematic diagram of a method and system for processing an image through a communication path portal according to an embodiment of the present invention.

[0046] FIG. 8A is a retinal fundus image of a healthy eye. 8B to 8E are retinal fundus images of several eye diseases.

9 and FIG. 10 are schematic diagrams of an image processing method and system according to an embodiment of the present invention.

11 is a schematic diagram of a retinal fundus image classification system of an image processing method and system according to another embodiment of the present invention.

12 is a flow chart of a method for constructing a calculation model of the embodiment shown in FIG. 11. 13A is a flowchart of a retinal fundus image loading method of the embodiment shown in FIG. 11 [0052] FIG. 13B is a flowchart of a retinal fundus image classification method of the embodiment shown in FIG.

[0053] FIG. 14A is an example of a retinal fundus image of a healthy eye.

14B to 14E are examples of retinal fundus images having several eye diseases.

15A is an example of a symbol mark used to represent a classification code of 1, FIG. 15B is an example of a symbol mark used to represent a classification code of 2 and a sub-category code of 2-1, and FIG. 15C is for representing a classification code of 2 and the sub-classification code is an example of a symbol mark of 2-2.

detailed description

[0056] In the present disclosure, the description of a given element and technical feature, or the consideration or use of a particular component number in a particular drawing, or a reference thereto in a corresponding description material, may be encompassed with another The same or equivalent elements and technical features or reference numerals are used in the drawings or in the description of the materials. While the aspects of the present disclosure are described in conjunction with the embodiments of the present invention, it is understood that the detailed description of the embodiments is not intended to limit the invention. To the contrary, the disclosure is intended to cover the embodiments of the invention, In addition, the detailed description is set forth to provide a thorough understanding of the disclosure. However, the present disclosure will be recognized by those skilled in the art, that is, those skilled in the art, the present disclosure may be described in detail, and/or have a combination of aspects from the specific embodiments. Implemented in multiple details. In a In other instances, well-known systems, methods, steps, and components have not been described in detail. The terms "comprising", "including" (compri se), "including", "include", and their grammatical variants, as used herein, unless otherwise indicated, The language is intended to be "open" or "inclusive", such that the methods and systems defined therein include the elements defined in the claims, and also include additional, non-limiting elements. The terms "transfer", "receive" or "load" and their grammatical variants as used herein are intended to mean the connection of two objects, elements or devices (either directly connected together or indirectly connected Together, electrically or wirelessly, through the connection of other components (such as routers, the Internet, networks, and servers).

[0057] As shown in FIG. 1, a calculation model construction 100 of an image processing method and system according to an embodiment of the present invention includes loading an expert retinal fundus image 101 into a database 102 for storing an expert retinal fundus image, using an expert. The level retinal fundus image 101 is trained on the artificial intelligence (AI) engine 103, and the model 104 is constructed using the AI engine 103.

2 is a schematic diagram of a retinal fundus image loading 200 of an image processing method and system in accordance with an embodiment of the present invention. A schematic diagram of retinal fundus image loading 200 shows: a portable fundus camera takes a user retinal fundus image 201 and combines the user's retinal fundus image with user data to create an initial user file with the user's retinal fundus image and user data 202, transmitted over the cell phone The initial user file 203 is connected to the handset tower connected to the network 204, the initial user file is received at the web server 205, and the initial user file is loaded to the server database 206 for storing the user files.

3 is a schematic diagram of a retinal fundus image classification 300 of an image processing method and system in accordance with an embodiment of the present invention. A schematic diagram of the retinal fundus image classification 300 shows: a database 206 storing an initial user file, the initial user file being compared using a plurality of reference images having a classification code of 1 or 2 stored in the database 206 using the calculation model 104, Take The classification code of the user image is determined to be one of 1 and 2, wherein 301 represents a user file including a user image whose classification code is determined to be 1, and 302 represents a user file including a user image whose classification code is determined to be 2. After the classification code of the user image is determined, the classification code of the user image is stored in the initial user file to generate an updated user file. The update user file is sent to the client, and the classification code and related information of the user image are transmitted to the user.

4 is a flow chart showing the steps of a computing model construction 400 of an image processing method and system in accordance with an embodiment of the present invention. The flow chart of the steps of the computational model building 400 shows the following steps:

• 401 loads multiple refraction fundus images of the expert classification into the database.

· 402 Training The AI engine operates on retinal fundus images classified by experts.

• 403 uses the AI engine to build a computational model based on expertly classified retinal fundus images.

• 404 further training of the AI engine based on retinal fundus images with classification codes 1 and 2 with ophthalmologist classification.

[0061] FIG. 5 is a flow chart showing the steps of a retinal fundus image loading 500 of an image processing method and system in accordance with an embodiment of the present invention. The step flow of the retinal fundus image loading 500 is shown in the steps:

• 501 indicates that a retinal fundus image of the user's retina is taken with a portable fundus camera in the area.

• 502 represents the creation of an initial user file with user data and a user's retinal fundus image.

• 503 indicates that initial user files are transferred to the server via a wireless data transmitter via a national portal or a world-class portal.

• 504 indicates that the initial user file was received by the server.

• 505 means loading the initial user file into the database. 6 is a flow chart showing the steps of a retinal fundus image classification 600 of an image processing method and system in accordance with an embodiment of the present invention. The step flow chart of classification 600 shows the steps:

• 601 indicates that the user's retinal fundus image is compared with the reference image using the calculation model created by the AI engine to determine the classification code of the user image as 1 and 2.

• 602 indicates that the classification code of the user image is stored in the initial user file to generate a newer user file.

• 603 indicates that the user file is determined to be 1 for the classification code. The method and system generate a first follow-up code and add an update user file to remind the user to periodically send the image of the retina to the system for subsequent classification. For user files whose classification code is determined to be 2, the method and system generate a second follow-up code and add an updated user file for suggesting that the user consult an ophthalmologist for further screening and necessary treatment.

7 is a schematic diagram of a communication path 700 through a portal 702 of an image processing method and system in accordance with an embodiment of the present invention. As shown in FIG. 7, communication path 700 through the portal includes communication to and from the portal through laptop 704, smart phone 706, tablet 708, computer 710, and optical center 712. The optical center receives communications from users in country 714 and village 716. One embodiment of the present invention provides a method for classifying a retinal fundus image for classifying a user's retinal fundus image to determine whether the user has an eye disease risk. The method comprises the steps of: (a) loading a plurality of experts to determine the classified reference retinal fundus image into the server database; (b) training the AI engine to operate on the reference retinal fundus image determined by the expert; (c) using the AI engine Calculating a model based on the reference retinal fundus image determined by the expert to assign a classification code to each reference retinal fundus image, wherein the classification code 1 indicates that the corresponding retinal fundus image is classified as "normal" or "low eye disease risk"; A "normal" retinal fundus image as shown in Fig. 8A. Classification code 2 indicates Corresponding retinal fundus images are classified as "abnormal" or "high eye disease risk", including, for example, a retinal fundus image having a "diabetic retinopathy" image feature as shown in FIG. 8B, having "glaucoma" as shown in FIG. 8C. Retinal fundus image of the image features (where 802 represents the cup-to-disk ratio (CDR) greater than or equal to 0.75), a retinal fundus image with "cataract" image features as shown in Figure 8D, and as shown in Figure 8E. Age-related macular degeneration (AMD) "image features of the retinal fundus image; (d) receiving an initial user file from a web server, the initial user file including user data and user images; (e) loading the initial user file into the server database (f) using the calculation model, comparing the retinal fundus image in the initial user file with the reference retinal fundus image in the server to determine the classification code of the user image as one of 1 and 2; And (g) the classification code of the user image is stored in the initial user file to generate an updated user file to record the user's retina eye Image classification, and update the user sends a file to the client.

[0065] The algorithm of the AI engine includes at least one algorithm or combination of algorithms selected from a machine learning algorithm and a deep learning algorithm. The algorithms of the AI engine include at least one of a support vector machine (SVM), a gradient hoist (GBM), a random forest, and a convolutional neural network. User files include user data and unrated user retinal fundus images.

[0066] In the alignment analysis and classification code determining step, the number of retinal fundus images stored in the initial user file may be two to four.

[0067] According to an alternative to the first embodiment, comparing the user image with the reference image comprises performing an alignment analysis based on at least one of the following eye state determination elements: (a) presenting in the image (b) cup-to-disk ratio less than 0.3; and (c) lacking at least one of: (i) visible media turbidity; (ϋ) diabetic retinopathy indicators including blotting-like hemorrhage, arterioles At least one of tumor and hard exudate; (iii) diabetic macular degeneration; (iv) macular edema; (V) exudate near the macula; (vi) in yellow Exudates on the plaque; (vii) laser scar; (viii) cataract; (ix) glaucoma; (X) diabetic retinopathy; and (xi) age-related macular degeneration, including multiple large drusen, At least one of a landmark atrophy of the marked area of hypopigmentation and a choroidal neovascular membrane, wherein age-related macular degeneration is at least one indicative of atrophic, neovascular, and exudative. At least one of the eye state determination elements may be excluded.

[0068] Excluding one or more eye state determination elements may adapt the classification process of the retinal fundus image to the needs of a given country and its available resources. [0069] According to another alternative of the first embodiment, (i) the expert determines that the classified retinal fundus image may be an ophthalmologist determines the classification retinal fundus image; (ii) the method may further include adding the first or Second follow up code. The first follow-up code corresponds to the user's retinal fundus image classified as 1 and may indicate that the user is prompted to transmit the user's retinal fundus image to the system for comparison within a specified time period, for example, within 6 to 12 months. The second follow-up code corresponds to a user retinal fundus image classified as 2, which may indicate that the user is scheduled to submit an updated user retinal fundus image for verification, and advises the user to consult an ophthalmologist; (iii) the method may further include, The updated user file of the first or second follow-up code is sent to the client; (iv) the method may further comprise: training the AI engine based on the classification of the ophthalmologist of the retinal fundus image of the at least one user file of the classification code of 1; V) The method may further comprise: training the AI engine based on the classification of the ophthalmologist of the retinal fundus image of the at least one user file of the classification code of 2; (vi) each retinal fundus image may comprise at least 3000*2000 pixels, having at least a 45 degree fundus area, and having a pixel resolution of at least 150 dpi; (vii) a retinal fundus image of at least one user file can be captured with a portable fundus camera, the step of receiving from the server can include via wireless data connectable to the portable fundus camera At least one of the transmitters Transfer of user files; retinal fundus image (viii) at least one user file may be captured by a portable fundus camera, and the step of receiving from the Web server comprises transmitting at least one file of the user via the portable fundus camera, Wherein the fundus camera may comprise a wireless data transmitter; (ix) the web server may host at least one national portal and at least one worldwide portal; or (X) at least one user file may be uploaded to the web server via at least one portable application. Communication can also be accomplished through wired connections such as telephone, cable, DSL, and fiber optics.

[0070] For a user file including a retinal fundus image with a classification code of 1, the first follow-up code may indicate that the user is reminded to retake the retinal fundus image of the user within a predetermined time, for example, 6 to 12 months. And transmitting to the server for comparison analysis to determine the classification code of the retinal fundus image of the user who is photographed again.

[0071] For a user file containing a user retinal fundus image with a classification code of 2, the second follow-up code may indicate that the user is invited to an ophthalmologist, and may further include scheduling a medical facility through a web server interface to schedule and confirm an appointment. date. [0072] The image processing method of the embodiment of the present invention further includes training the AI engine based on the retinal fundus image of the at least one user file having the classification code of 2. Retinal fundus images that are classified as abnormal by the system can be sent to an expert for classification. Once classified, the expert classified retinal fundus image can be used to further train the AI engine; (vi) each retinal fundus image can include at least 3000*2000 pixels, a fundus area of at least 45 degrees, and a pixel of at least 150 dpi Resolution (vi i ) The retinal fundus image of at least one user file may be captured with a portable fundus camera, and the step of receiving from the web server may include transmission of at least one user file via a wireless data transmitter connectable to the portable fundus camera (vi ii) a retinal fundus image of at least one user file may be captured with a portable fundus camera, and the step of receiving from the web server may include transmission of at least one user file via the portable fundus camera, wherein the fundus camera may comprise a wireless data transmitter ( ix ) the web server may host at least one national portal and at least one worldwide portal; or (X) at least one user file may be uploaded to the web service via at least one portable application Device.

[0073] By connecting a wireless fund transmitter (such as a cell phone) of the portable fundus camera, the user's retinal fundus image and user data can be transmitted to the data center or the lab hosting the AI engine 103 and the model 104, and quickly sorted. Subsequent treatments such as scheduling routine reclassifications or appointments with clinic ophthalmologists can be recommended from the classification of the user's retinal fundus images. In this way, a user's retinal fundus image can be taken using a portable fundus camera in rural or remote areas. User data can be entered by the portable application and then transmitted over the local mobile data network. Portable fundus cameras with wireless data transmitters and/or user data can also be used for user data entry and wireless data transmission. Through the portable application, the user can obtain the classification code of the retinal fundus image and the first or second follow-up code to take corresponding actions, such as obtaining a referral to a public hospital and accessing a health information service, the content of which can be based on his/her user His case records contained in the document were customized. [0074] Server 205 can host multiple portals for uploading user files. Portals can be organized by country, region, language or worldwide. At least one or more portals may be accessible via the portable application.

[0075] A second embodiment of the invention relates to an image processing method and system for running on a high speed computing system (which may be implemented in a public or private cloud or on a dedicated enterprise computing resource) for the user's retinal fundus The images are sorted.

As shown in FIGS. 9 and 10, a user living in a user terminal, such as a village 910 or a small city 920, photographs a retinal fundus image of the user by a camera 912, 922 disposed nearby (FIG. 10, step 1012), generating an initial User files, and the initial user files are transmitted through the communication network 930 to a server provided with the image processing system of the present invention, such as the system server 942 disposed in the metropolitan area 940, for performing a comparison analysis to obtain a classification code 1016 of the user's retinal fundus image. , 1018 (Fig. 10, step 1014). The classification code 1016 is a classification of the code "1" indicating that the user's retinal fundus image belongs to "normal" or "low eye disease risk"; the classification code 1018 is a classification of the code "2" indicating that the user's retinal fundus image belongs to "abnormal" or "High eye disease risk". The method and system can further include generating a first follow-up code 1026 and a second follow-up code 1028, corresponding to the classification code 1 and the classification code 2, respectively. The classification codes 1016, 1018, the first and second follow-up codes 1026, 1028 are stored in the updated user files 1036, 1038, respectively, and sent to the clients 910, 920.

[0077] The image processing method and system provided by the embodiment develops and trains an artificial intelligence ("Art") engine based on a reference image with a classification code authenticated by an expert or an ophthalmologist, and builds the engine using the UI engine. Computational model, and using the calculation model to compare and predict the retinal fundus image of the user with the reference image with the classification code, thereby obtaining the classification of the retinal fundus image of the user, and feeding back the classification result to the user. Classified as the first category, that is, the user corresponding to the classification code 1, is determined to belong to the low-risk eye disease population, and may not need to seek medical treatment at present, and may be routinely performed after a certain period of time, for example, 6 months to 12 months later. The retinal fundus image is classified into the second category, that is, the user corresponding to the classification code 2, and is determined to belong to a population of high-risk eye diseases and/or related diseases. The image processing method and system of the embodiment of the present invention will further Including generating the first or second follow-up code, and following the first or second follow-up The code is stored in the update user file and the update user file is sent to the client.

According to the present embodiment, the reference image to which the classification code is added, the 引擎 engine and the calculation model can effectively identify and classify the retinal fundus image corresponding to the main disease associated with the eye. Such diseases include diabetic retinopathy (DR), age-related macular degeneration (AMD), glaucoma, and cataracts.

[0079] According to a calculation model using an embodiment of the present invention, a user's retina fundus image is used Compared with the reference image, if the degree of similarity between the retinal fundus image and the reference image classified as the first type in the reference image is higher than the determination threshold in the calculation model, the retinal fundus image of the user is divided into the same category. That is the first category. If the degree of similarity between the retinal fundus image and the reference image classified as the second type in the reference image is higher than the determination threshold in the calculation model, the retinal fundus image of the user is classified into the same category, that is, the second category.

[0080] According to an embodiment of the present invention, the reference image is manually screened by a qualified ophthalmologist, and each reference image is classified into one of the first category and the second category based on the discrimination result, and the corresponding classification code is assigned one by one.

[0081] Based on a plurality of reference images having classification codes, the AI engine is trained to construct a calculation model to perform an alignment analysis on the retinal fundus image of the user, and a classification code of the retinal fundus image of the user is obtained. [0082] The server storing the plurality of reference images and the calculation model having the classification code may be implemented in the form of deep learning ("DL") or deep neural network ("DNN"). The process of supervising learning machine learning (Machine Learning, "ML") or artificial intelligence (AI) frameworks. The obtained DNN algorithm and the calculation model can be used for comparing and analyzing the retinal fundus image of the user with the reference image in the server to obtain the classification code of the retinal fundus image of the user.

[0083] The computational model can be implemented, but is not limited to, a desktop workstation (with or without a GPU). Operating system (OS) used, including but not limited to, Windows®, iOS®, Android, and Linux-based systems. The computing model can also be hosted on a cloud-based service provided by a third-party vendor's platform. Image processing systems and/or platforms may include, but are not limited to, Nvidia CUDA® or open source platforms such as OpenGL, OpenCV and OpenCL. The computational model can choose a high-level programming language and platform (including but not limited to Matlab®, Python, C++ and R, etc., as well as the packaging of these platforms) implementation.

An image processing method according to an embodiment of the present invention includes accessing a database including digitized retinal fundus images, and storing information for processing. The original image is mapped to a 3-tuple consisting of 3 independent matrices, each of which represents a color in RGB (red, green, and blue). If desired, RGB-based images can be converted or reduced to grayscale images.

[0085] The image processing method according to an embodiment of the present invention further includes reshaping all images having the same spatial dimension in the data set, although this is not absolutely necessary. The number of pixels in the width and height of the image can be optimized to the amount of time required to train the classification model. Image processing methods such as image enhancement, image noise reduction, image restoration and deblurring, scaling, panning, rotation, and edge detection can be applied depending on the quality of the image. The mapped image will form a data set that will be used as input to train the classification and calculation models. [0086] These images may also be via, but not limited to, principal component analysis (also known as

The KarhuenLoeve transform) and other transform methods such as dynamic mode decompression are further processed, in which the singular value decomposition of the matrix is performed. By using transformation techniques, alternative and supplemental DNNs can be developed to train on these transformed images to correlate with the primary D-L model to compare, analyze, and accurately classify retinal fundus images.

[0087] The architecture of the classification model is employed, but is not limited to the use of Convolutional Neural Networks (CNN). The nominal data set is imported into the CNN architecture, and the inferred function is used to train the classification model to predict new unseen images. The accuracy of the CNN architecture depends on a range of parameters, such as the number of nodes on each layer, the choice of activation function, the loss function, the percentage of loss, the number of epochs, and so on.

[0088] Classification and computational models can be further enhanced with k-fold cross-validation techniques. Other Possible statistical techniques can be implemented to improve the accuracy of the classification model, and are not limited to the above techniques. The k-fold cross-validation technique is a model validation that evaluates the statistical performance of a trained classification model. The nominal data set is divided into training data sets and test data sets with different percentage weights. [0089] A possible example of a classification model is described below to illustrate this process:

(1) The digitized image and its label are mapped to the data set. The scaling of the image can be performed by first calculating the average and standard deviation of the respective pixel densities of each of the red, green, and blue channels. The results are stored in 3-tuples, which correspond to the average of the red, green, and blue channels, respectively. The image in the dataset is scaled by subtracting the corresponding average of each of the red, green, and blue channels and dividing by the corresponding standard deviation for each of the red, green, and blue channels.

(2) The C architecture can be written in a high-level language such as R, and has the help of a package such as Keras. The CNN can be designed to have a specific number of layers consisting of interconnected nodes. The link between each node in a different function is defined by a function consisting of weights and deviations. Activation functions such as RELU are often used to update the weights and deviations of functions. Adding a pooling function to extract a subset on the CNN layer may not be necessary. A percentage loss of, for example, 20% is added after each CiW layer. At the last level, use the activation function S0FTMAX.

(3) The classification model goes through multiple periods (epoch) to update its accuracy. The optimizer in the model is not limited to ADAM, there are other optimizers, such as RMSPR0P. At each epoch, the nominal data set can be split into training data sets and test data sets, not limited to 4:1 segmentation. This ratio can be further subdivided into other ratios that are considered to be the most suitable for training the classification model. [0090] A possible verification step of the trained classification model is described below to illustrate the process.

(1) After completing the training of the classification model, a new set of unseen retinal fundus images that are not used in the training or test are presented to the trained model. New unseen retinal fundus images are manually identified by a qualified ophthalmologist, and the classification code is determined and marked.

(2) The probability of the user's eye health is based on the retinal fundus image. The next step is to verify the probability of the generation by a qualified ophthalmologist by screening the unseen images and their respective labeled categories (category 1 or 2).

(3) Further verification steps can be applied to identify major potential eye diseases including, but not limited to, DR, A, glaucoma, and cataract. A qualified ophthalmologist may have difficulty obtaining all the subtle differences in visual impairment and conclude the user's eye health. The trained computational model can correlate the conclusions drawn by a qualified ophthalmologist with the probabilities generated by the trained classification calculation model.

[0091] The automated process of the artificial intelligence portal can generate a concise and easy-to-read retinal fundus image classification and analysis report that will be sent to users from surrounding communities to inform the user of the classification code and follow-up code of the retinal fundus image, and Based on the information recorded in the classification code and follow-up code, does the user need to consult an ophthalmologist for further examination. The classification report compares the retinal fundus image classification code and the corresponding, easily identifiable color marker by the calculation model. For example, a green mark and/or (-) may be used to represent the classification code 1, indicating that the user's retinal fundus image is classified into the first category; using the red mark and/or (+) representing the classification code 2, indicating the user's retinal fundus The images are classified into the second category.

[0092] An example report according to the above description is as follows: Classification color marker symbol report meaning first category (green) (-) User will not be advised to consult an ophthalmologist for further examination

The second category (red) ( +) users will be advised to consult an ophthalmologist

For further inspection

[0093] The calculation model construction of the image processing method and system according to another embodiment of the present invention may include: loading an expert retinal fundus image into a database storing an expert retinal fundus image, using an expert retinal fundus image For the training of the AI engine, build the model using the AI engine.

[0094] A retinal fundus image loading system of an image processing method and system according to another embodiment of the present invention may include: (i) a portable fundus camera for capturing a retinal fundus image of a user and imaging the user's retina fundus with user data Combining to create an initial user file with a user retinal fundus image and user data; (ii) a mobile phone for transmitting the initial user file to a mobile phone tower connected to the network; (iii) a web server for receiving the initial user file, And load the initial user file into the server database used to store the user files.

11 is a schematic diagram of a retinal fundus fundus image classification system 1100 of an image processing method and system according to another embodiment of the present invention.

[0096] The retinal fundus image classification system 1100 may include a server database 1106 in which a calculation model 1104 and a reference image including a plurality of reference images having a classification code of 1 and a plurality of reference images having a classification code of 2 are stored. After receiving the initial user file (including the user data and the user image), the server starts the computing model 1104, and the user is The image is compared with a plurality of reference images having a classification code of 1 or 2 stored in the database 1106 to determine the classification code of the user image as one of 1 and 2. The reference image having the classification code of 2 includes a plurality of reference images having a sub-category code of 2-1 and a reference image having a plurality of sub-category codes of 2-2. Further, the server starts the calculation model 1104, and compares the user image whose classification code is determined to 2 with a reference image with multiple sub-category codes of 2-1 and a reference image with multiple sub-category codes of 2-2. The sub-category code of the user image whose classification code is determined to be 2 is further determined to be one of 2-1 and 2-2. 1111 denotes a user file including a user image whose classification code is determined to be 1, 1112 denotes a user file including a user image whose classification code is determined to be 2, and 1121 denotes a user file including a user image whose sub-category code is determined to be 2-1, 1122 Represents a user file including a user image whose subcategory code is determined to be 2-2. After the classification code of the user image and the sub-category code are determined, the classification code and sub-category code of the user image are stored in the initial user file to generate an updated user file. The update user file is sent to the client, and the classification code, sub-category code and related information of the user image are transmitted to the client. [0097] Preferably, FIG. 12 is a flow chart showing the steps of the calculation model construction method 1200 of the image processing method and system according to the embodiment shown in FIG. The calculation model construction method 1200 includes:

• Load multiple refraction fundus images of the expert classification into the database, as shown in block 1202;

• Train the AI engine to operate on retinal fundus images classified by an expert, as shown in block 1204;

• Use the AI engine to build a computational model of the retinal fundus image based on expert classification, as shown in block 1206;

• Further training of the AI engine based on retinal fundus images with classification codes 1 and 2 with an ophthalmologist classification, as shown in block 1208;

• Further training of the AI engine based on retinal fundus images with sub-classification codes 2-1 and 2-2 with ophthalmologist classification, as shown in block 1210. [0098] Preferably, FIG. 13A is a flow chart showing the steps of the retinal fundus image loading method 1300 according to the image processing method and system of the embodiment shown in FIG. The retinal fundus image loading method 1300 can include:

• Taking a retinal fundus image of the user's retina with a portable fundus camera in the area, as shown in block 1302;

• Create an initial user file with user data and a user retinal fundus image, as shown in block 1304;

• Transfer the initial user files to the server via a wireless data transmitter via a national portal or a world-class portal, as shown in block 1306;

· The initial user file is received by the server, as shown in block 1308;

• Load the initial user file into the database, as shown in box 1310.

13B is a flow chart showing the steps of the retinal fundus image classification method 1350 according to the image processing method and system of the embodiment shown in FIG. 11. Classification method 1350 includes:

· Aligning the user's retina fundus image with a plurality of reference images with a classification code of 1 and a plurality of reference images with a classification code of 2 using a calculation model created by the AI engine to determine the classification code of the user image as 1 and 2 One, as shown in block 1352;

• Using the calculation model created by the AI engine, the user's retina fundus image with the classification code determined as 2 is compared with a reference image with multiple subcategory codes of 2-1 and a reference image with multiple subcategory codes of 2-2 to The sub-category code of the user retinal fundus image whose classification code is determined to be 2 is determined to be one of 2-1 and 2-2, as shown in block 1354;

• storing the classification code and sub-category code of the user image into the initial user file to generate an updated user file, as shown in block 1356; [0100] The classification method 1350 can further include:

• For user files whose classification code is determined to 1, the method and system generate a third follow-up code and add an update user file to remind the user to periodically retina the fundus The image is sent to the system for subsequent classification (for example, it is recommended that the user take a retinal fundus image again within 6 to 12 months and transmit it to the server for comparison analysis to determine the classification code of the retinal fundus image of the user who is photographed again) As shown in block 1358; • for a user file whose sub-category code is determined to be 2-1, the method and system generate a fourth follow-up code and add an update user file for suggesting that the user consult an ophthalmologist, but not an emergency , or for suggesting that the medical institution/ophthalmologist does not need to give the user immediate medical attention (ie: non-emergency). For user files whose sub-category code is determined to be 2-2, the method and system generate a fifth follow-up code and add an updated user file for suggesting that the user immediately consult an ophthalmologist or for suggesting a medical institution/ophthalmologist immediately The user is given emergency medical treatment for further screening and necessary treatment (i.e., emergency), as shown in block 1360.

[0101] In particular, the communication path of the image processing method and system through the portal may include communication to and from the portal through a laptop, a smart phone, a tablet, a computer, and an optical center. The optical center receives communications from users in countries and villages.

[0102] Specifically, the classification code 1 may indicate that the corresponding retinal fundus image is classified as "normal" or "low eye disease risk", for example, an example of a "normal" retinal fundus image as shown in FIG. 14A. Classification code 2 may indicate that the corresponding retinal fundus image is classified as "abnormal" or "high eye disease risk". The sub-category code 2-1 may indicate that the corresponding retinal fundus image is classified as "abnormal but not urgent". The sub-category code 2-2 may indicate that the corresponding retinal fundus image is classified as "an abnormal and urgent situation" including, for example, a retinal fundus image having a "cataract" image feature as shown in FIG. 14B, as shown in FIG. 14C Retinal fundus image of "diabetic retinopathy" image features, retinal fundus image with "glaucoma" image features as shown in Figure 14D, and "age-related macular degeneration (AMD) as shown in Figures 14E-14F) "Image features of the retinal fundus image. Wherein 1402 in Figure 14C represents pre-retinal hemorrhage, 1404 and 1406 represent hard exudate, 1408 represents cotton wool, 1410 And 1412 indicates bleeding, and 1414 in Fig. 14D indicates an increased cup-to-disk ratio (cup-to-disk ratio greater than or equal to 0.75), 1416 in Fig. 14E indicates a map-like atrophy (late A-intestine), and 1418 in Fig. 14F indicates a drusen. [0103] According to the embodiment shown in FIGS. 11-13B, the user image and the reference image are compared and analyzed to determine the classification code of the user image as one of 1 and 2, including performing comparison based on at least one of the following eye state determination elements. For analysis: (a) multiple retinal vessels present in the image; (b) cup-to-disk ratio less than 0.3; and (c) lack of at least one of: (i) visible media turbidity; ( ϋ ) diabetic retinopathy The indicator includes at least one of blotting-like bleeding, microaneurysms, and hard exudate; (iii) diabetic macular degeneration; (iv) macular water; (v) exudate near the macula; (vi) Exudate on the macula; (vii) laser scar; (viii) cataract; (ix) glaucoma; (X) diabetic retinopathy; and (xi) age-related macular degeneration, including multiple large drusen, At least one of a landmark atrophy with a significant area of hypopigmentation and a choroidal neovascular membrane, wherein age-related macular degeneration is indicative of atrophic, neovascular, and exudative One. At least one of the eye state determination elements may be excluded. Excluding one or more eye state determination elements can tailor the classification process of the retinal fundus image to the needs of a given country and its available resources.

[0104] According to the embodiment shown in FIGS. 11-13B, the comparison of the user image whose classification code is determined to 2 and the reference image whose sub-category code is 2-1 further includes comparison using at least one of the following eye state determination elements :

(a-i) microaneurysm / blotting;

(a-ii) hard exudate not in the macula;

(b-i) a slightly dense cataract (the macula and blood vessels are visible);

(c-i) the presence of a glass (hard or soft) in the surrounding area away from the macula (outside the three disc diameters from the center of the macula);

(c-ii) Pigment. [0105] Aligning the user image whose classification code is determined to 2 with the reference image whose sub-category code is 2-2 further includes comparing using at least one of the following eye state determination elements:

(a-i) more than three blots;

(a-ii) flame bleeding;

(a-iii) cotton wool spots with hemorrhage and microaneurysms;

(a-iv) hard exudates with microaneurysms and blotting in the macula;

(a-v) venous beaded changes;

(a-vi) 2 or more quadrants of bleeding;

(a-vii) microvascular abnormalities in the retina;

(a-viii) venous loop;

(a-ix) the presence of new blood vessels on the disc or any other new blood vessel;

(a-x) retinal detachment;

(a-xi) preretinal hemorrhage;

(a-xii) vitreous hemorrhage;

(a-xiii) fibrosis;

(b-i) Partial or complete densification of the cataract that obscures the macula and blood vessels;

(b-ii) the macula and blood vessels are invisible;

(c-i) the presence of drusen (hard or soft) within the macula (within the diameter of the two discs from the center of the macula);

(c-ii) map-like atrophy;

(c-iii) fibrous scar under the retina;

(c-iv) pigment epithelial detachment;

(c-v) lesions in the subretinal fibrovascular (hemorrhage under the retina);

(c-vi) choroidal neovascular membrane;

(c-vii) exudative age-related macular degeneration; (di) the cup-to-disk ratio of any eye is greater than or equal to 0.3;

(d-ii) the asymmetry of the disk is greater than or equal to 0.2;

(d-iii) bleeding from the disk;

(d-iv) the occurrence of any grooving or edge thinning;

(e-i) medium turbidity;

(e-ii) a blurred macula (which may cause a contracted pupil);

(e-iii) invisible blood vessels and macula caused by underexposure or overexposure;

(e-iv) Insufficient focus and/or incorrect positioning of the optic disc and macula. [0106] Alternatively, (i) the expert determines that the classified retinal fundus image may be an ophthalmologist determines the classification retinal fundus image; (ii) the method may further include adding an update of the third, fourth or fifth follow-up code The user file is sent to the client; (iii) the method may further comprise: training the AI engine based on the classification of the ophthalmologist of the retinal fundus image of the at least one user file of the classification code of 1; (iv) the method may further comprise, based on the classification code The classification of the ophthalmologist of the retinal fundus image of at least one user file, the training AI engine; (v) the method may further comprise an ophthalmologist of the retinal fundus image based on at least one user file of the sub-classification code 2-1 Classification, training the AI engine; (vi) the method may further comprise, based on the classification of the ophthalmologist of the retinal fundus image of the at least one user file of the sub-classification code 2-2, training the AI engine; (vii) each retinal fundus image may Including at least 3000*2000 pixels, having a fundus area of at least 45 degrees, And having a pixel resolution of at least 150 dpi; (viii) the retinal fundus image of the at least one user file can be captured with a portable fundus camera, the step of receiving from the server can include at least one user via a wireless data transmitter connectable to the portable fundus camera (ix) Retinal fundus image of at least one user file may be captured with a portable fundus camera, and the step of receiving from the web server includes transmission of at least one user file via a portable fundus camera, wherein the fundus camera may include wireless data transmission (X) a web server can host at least one national portal and at least one worldwide portal; or (xi) at least one user file can be At least one portable application is uploaded to the web server. Communication can also be accomplished through wired connections such as telephone, cable, DSL, and fiber optics.

[0107] By connecting a wireless fund transmitter (such as a cell phone) of the portable fundus camera, the user's retinal fundus image and user data can be transmitted to a data center or a lab hosting the AI engine and computing model 1104, and quickly sorted. Subsequent treatments such as scheduling routine reclassifications or appointments with an ophthalmologist at the clinic (urgent or non-emergency) can be recommended from the classification of the user's retinal fundus image. Thus, a portable fundus camera can be used to capture a user's retinal fundus image in rural or remote areas. User data can be entered by the portable application and then transmitted over the local mobile data network. Portable fundus cameras with wireless data transmitters and/or user data can also be used for user data entry and wireless data transmission. Through the portable application, the user can obtain the classification code of the retinal fundus image and the third, fourth or fifth follow-up code to take corresponding actions, such as obtaining a referral to a public hospital and accessing a health information service, the content of which can be based on him/ Her case records contained in her user files are customized.

[0108] The server can host multiple portals for uploading user files. Portals can be organized by country, region, language or the world. At least one or more portals may be accessible via the portable application.

[0109] Alternatively, the comparison between the user retinal fundus image and the reference image is analyzed, and if the degree of similarity of the retinal fundus image and the reference image classified into the first category in the reference image is higher than the determination threshold in the calculation model, This user's retinal fundus image is divided into the same category, the first category. If the degree of similarity between the retinal fundus image and the reference image classified as the second type in the reference image is higher than the determination threshold in the calculation model, the user's retinal fundus image is classified into the same category, that is, the second category.

[0110] Alternatively, the alignment of the user's retinal fundus image with the reference image is analyzed, such as If the degree of similarity between the user retinal fundus image and the reference image of the sub-category code 2-1 in the reference image is higher than the determination threshold in the calculation model, the sub-category code of the user's retinal fundus image is determined to be 2-1 (ie, classification) For the second category, the subcategory is the first category). If the similarity between the retinal fundus image and the reference image of the sub-category code 2-2 in the reference image is higher than the determination threshold in the calculation model, the sub-category code of the retinal fundus image of the user is determined to be 2-2 (ie, classification) For the second category, the subcategory is the second category).

[0111] The automated process of the artificial intelligence portal can generate a concise and easy-to-read retinal fundus image classification and analysis report that will be sent to users or medical institutions/doctors from surrounding communities to inform them about the user's retinal fundus image. Classification code and subcategory code

(if any), and based on the information recorded by the classification code and sub-category code (if any), does the user need to consult an ophthalmologist for further examination and whether the user needs to consult an ophthalmologist or medical institution/ophthalmologist immediately? The user needs to be given emergency medical treatment immediately (ie: is the situation urgent). The classification report compares the retinal fundus image classification code/sub-category code obtained by the comparative analysis of the model, and the corresponding, easily recognized color marker/symbol marker. For example, "(-)" or a symbol mark as shown in Fig. 15-A may be used to represent the classification code 1, indicating that the user's retinal fundus image is classified into the first category. The symbol mark can further select a green color mark. You can use "(non-emergency)", "(+ 30% non-emergency)" or a symbol mark as shown in Figure 15-B to represent the classification code 2 and the sub-category code 2-1, indicating the user's retinal fundus The image is classified into the second category and the subcategory is the first category. The symbol mark can further select a red color mark. You can use "(emergency)", "(+90% urgent)" or the symbol mark shown in Figure 15-C to represent the classification code 2 and the sub-category code is 2-2, indicating that the user's retinal fundus image is classified as The second category and subcategory are the second category. The symbol mark can further select a red color mark. Where 30%, 90% of the symbolic markers can alternatively select other percentage values that represent the likelihood that the user's eyes are at risk for the disease. In Fig. 15B, 1502 indicates "non-emergency"; in Fig. 15C, 1504 indicates "emergency". [0112] An example report according to the above description is as follows:

[0113] In the foregoing detailed description, embodiments of the invention are described with reference to the accompanying drawings. The description of the various embodiments herein is not intended to be limited or limited to the particular embodiments of the disclosure.

[0114] The present disclosure is used to address at least some of the above problems and problems associated with the prior art. Although only some embodiments of the present disclosure are disclosed herein, in view of the present disclosure, Various changes and/or modifications may be made to the disclosed embodiments without departing from the scope of the disclosure. The scope of the disclosure and the scope of the appended claims are not limited to the embodiments described herein.

Claims

Rights request

An image processing method, the method comprising:

Receiving an initial user file from a user end, the initial user file including user data and a user image;

Loading the initial user file into a server, where the server stores a reference image and a calculation model, the reference image includes a plurality of reference images with a classification code of 1 and a plurality of reference images with a classification code of 2;

Using the calculation model, comparing the user image with the reference image to determine the classification code of the user image as one of 1 and 2; storing the classification code of the user image into the initial user file to generate Update user files; send update user files to the client.

2. The method according to claim 1, wherein the updating a user file package

A color mark is included, the color mark including a green mark corresponding to the classification code 1 and a red mark corresponding to the classification code 2.

3. The method according to claim 1, wherein if the user image is classified

The code is determined to 1, the method further comprising storing the first follow-up code in an initial user file to generate the updated user file.

4. The method according to claim 1, wherein if the user image is classified

The code is determined to be 2, the method further comprising storing the second follow-up code in an initial user file to generate the updated user file.

5. The method according to claim 1, further comprising: determining the score

The user image of the class code is stored as a reference image in the server.

The method according to claim 1, further comprising: before receiving the initial user file from the user end, loading the reference image in the server, training the artificial intelligence engine based on the reference image, and The computational model is constructed using the artificial intelligence engine.

The method according to claim 6, wherein the artificial intelligence engine comprises a combination of at least one of a machine learning algorithm and a deep learning algorithm.

8. The method of claim 7, wherein the artificial intelligence engine comprises at least one of a support vector machine (SVM), a gradient hoist (GBM), a random forest, and a convolutional neural network.

9. The method of claim 6, further comprising: training the artificial intelligence engine based on the user image and the determined classification code.

The method according to claim 1, wherein the user image is a retinal fundus image of a user, comprising at least 3000*2000 pixels, a fundus region having at least 45 degrees, and a pixel resolution of at least 150 dpi. .

The method according to claim 1, wherein the user image is a retinal fundus image of a user, wherein comparing the user image with the reference image further comprises using an eye state determining element At least one comparison:

(a) multiple retinal vessels present in the image;

(b) cup to plate ratio is less than 0.3; and (c) Lack of at least one of the following elements:

(i) visible media turbidity;

( ϋ ) a diabetic retinopathy indicator comprising at least one of a blotting-like hemorrhage, a microaneurysm, and a hard exudate;

(iii) diabetic macular degeneration;

(iv) macular edema;

(v) exudates near the macula;

(vi) exudate on the macula;

(vii) laser scars;

(viii) cataract;

(ix) glaucoma;

(χ) diabetic retinopathy; and

(xi) Age-related macular degeneration, which includes at least one of a plurality of large drusen, a marked atrophy with a significant area of hypopigmentation, and a choroidal neovascular membrane, wherein age-related macular degeneration is indicative of atrophic At least one of neovascularization and exudation;

At least one of the eye state judgment elements may not be a judgment element of the classification. The method of claim 1, wherein the user file is uploaded to the server by a wireless data transmitter.

13. The method of claim 1 wherein the server hosts at least one national data transmission portal and at least one world data transmission portal.

14. The method of claim 1 wherein a user file is uploaded to the server via at least one portable application.

The method according to claim 1, wherein the reference image having the classification code of 2 comprises a plurality of reference images having a sub-category code of 2-1 and a plurality of reference images having a sub-category code of 2-2. The method includes comparing a user image whose classification code is determined to 2 with a reference image of the plurality of sub-category codes of 2-1 and a reference image of a plurality of sub-category codes of 2-2 to determine the classification code as The sub-category code of the user image of 2 is further determined to be one of 2-1 and 2-2, and the sub-category code of the user image is stored in the initial user file to generate an updated user file. The method according to claim 15, wherein the update user file comprises a text mark, the text mark includes a "non-emergency" mark corresponding to the sub-category code 2-1 and a sub-category code 2 - 2 corresponding "emergency" mark.

17. The method of claim 15, wherein if the sub-category code of the user image is determined to be 2-1, the method further comprises storing the fourth follow-up code in an initial user file to generate the Update user files.

The method according to claim 15, wherein if the sub-category code of the user image is determined to be 2-2, the method further comprises: storing the fifth follow-up code in the initial user file to generate the Update user files.

The method according to claim 15, wherein the user image is a retinal fundus image of a user, wherein the user code whose classification code is determined to be 2 and the reference of the sub-category code is 2-1 Aligning the images further includes using at least one of the following eye state determination elements for comparison:

( a-i ) microaneurysm / blotting;

( ai i ) a hard exudate that is not in the macula; (bi) a slightly dense cataract (the macula and blood vessels are visible);

(c-i) the presence of drusen (hard or soft) in the surrounding area away from the macula (outside the three disc diameters from the center of the macula);

(c-ii) Pigment.

The method according to claim 15, wherein the user image is a retinal fundus image of a user, wherein the user code whose classification code is determined to be 2 and the reference of the sub-category code is 2-2 Aligning the images further includes using at least one of the following eye state determination elements for comparison:

(a-i) more than three blots;

(a-ii) flame bleeding;

(a-iii) cotton wool spots with hemorrhage and microaneurysms;

(a-iv) hard exudates with microaneurysms and blotting in the macula;

(a-v) venous beaded changes;

(a-vi) 2 or more quadrants of bleeding;

(a-vii) microvascular abnormalities in the retina;

(a-viii) venous loop;

(a-ix) the presence of new blood vessels on the disc or any other place; (a-x) retinal detachment;

(a-xi) preretinal hemorrhage;

(a-xii) vitreous hemorrhage;

(a-xiii) fibrosis;

(b-ii) the macula and blood vessels are invisible;

(c-ii) map-like atrophy;

(c-iii) fibrous scars under the retina; (c-iv) pigment epithelial detachment;

(c-v) lesions in the subretinal fibrovascular (hemorrhage under the retina);

(c-vi) choroidal neovascular membrane;

(c-vii) exudative age-related macular degeneration;

(d-i) the cup-to-disk ratio of any eye is greater than or equal to 0.3;

(d-ii) the asymmetry of the disk is greater than or equal to 0.2;

(d-iii) bleeding from the disk;

(d-iv) the occurrence of any grooving or edge thinning;

(e-i) medium turbidity;

(e-ii) a blurred macula (which may cause a contracted pupil);

(e-iii) invisible blood vessels and macula caused by underexposure or overexposure; and

(e-iv) Insufficient focus and/or incorrect positioning of the optic disc and macula. 21. An image processing system, wherein the system comprises:

a server, wherein a reference image and a calculation model are stored, the reference image includes a plurality of reference images having a classification code of 1 and a plurality of reference images having a classification code of 2; and a user end, configured to generate an initial user file, the initial user The file includes user data and a user image; the client is in communication with the server, and after receiving the user file, the server starts the computing model, and compares the user image with the reference image to The classification code of the user image is determined to be one of 1 and 2, the classification code of the user image is stored in the initial user file to generate an updated user file, and the updated user file is transmitted to the client. The system according to claim 21, wherein said updating a user file package

23. The system of claim 21, wherein the system further comprises an artificial intelligence engine trained based on the reference image, the artificial intelligence engine to construct the computing model.

The system according to claim 22, wherein the artificial intelligence engine comprises a group of at least one of a machine learning algorithm and a deep learning algorithm

25. The system of claim 23, wherein the artificial intelligence engine comprises at least one of a support vector machine (SVM), a gradient hoist (GBM), a random forest, and a convolutional neural network.

The system according to claim 21, wherein the user image is a retinal fundus image of the user, the server stores the eye state determination element information, and the eye state determination element information comprises:

(a) multiple retinal vessels present in the image;

(b) the cup-to-disk ratio is less than 0.3;

(c) at least one of the following elements:

(i) visible media turbidity;

( ϋ) a diabetic retinopathy indicator comprising at least one of a blotting-like hemorrhage, a microaneurysm, and a hard exudate;

(iii) diabetic macular degeneration;

(iv) macular edema;

(v) exudates near the macula;

(vi) exudate on the macula;

(vii) laser scars; (iii) cataract;

(ix) glaucoma;

(χ) diabetic retinopathy; and

(xi) Age-related macular degeneration, including at least one of a plurality of large drusen, with a hypotrophic and map-like atrophy of a significant area of the choroidal neovascular membrane, wherein age-related macular degeneration is indicative of atrophy, At least one of neovascularization and exudation.

27. The system according to claim 21, wherein the reference image having the classification code of 2 comprises a plurality of reference images having a sub-category code of 2-1 and a plurality of reference images having a sub-category code of 2-2. The server starts the calculation model, and compares the user image whose classification code is determined to 2 with the reference image of the plurality of sub-category codes of 2-1 and the reference image of the plurality of sub-category codes of 2-2 to The sub-category code of the user image whose classification code is determined to be 2 is further determined to be one of 2-1 and 2-2, and the sub-category code of the user image is stored in the initial user file to generate an updated user file.

28. The system according to claim 27, wherein the update user file comprises a text mark comprising a "non-emergency" mark corresponding to the sub-category code 2-1 and a sub-category code 2 - 2 corresponding "emergency" mark.

29. The system of claim 27, wherein: the user map

For example, the retinal fundus image of the user, wherein the comparison of the user image whose classification code is determined to 2 with the reference image of the sub-category code of 2-1 further comprises using at least one of the following eye state determination elements for comparison:

(a-i) microaneurysm / blotting;

(a-ii) hard exudate not in the macula;

(bi) a slightly dense cataract (the macula and blood vessels are visible); (ci) the presence of drusen (hard or soft) in the surrounding area away from the macula (outside the diameter of the three discs from the center of the macula);

(c-ii) Pigment. 30. The system of claim 27, wherein the user map

For example, the retinal fundus image of the user, wherein the comparison of the user image whose classification code is determined to 2 with the reference image of the sub-category code of 2-2 further comprises using at least one of the following eye state determination elements for comparison:

(a-i) more than three blots;

(a-ii) flame bleeding;

(a-iii) cotton wool spots with hemorrhage and microaneurysms;

(a-iv) hard exudate with microaneurysm and blot-like hemorrhage in the macula; (a-v) venous bead-like changes;

(a-vi) 2 or more quadrants of bleeding;

(a-vii) microvascular abnormalities in the retina;

(a-viii) venous loop;

(a-xi) preretinal hemorrhage;

(a-xii) vitreous hemorrhage;

(a-xiii) fibrosis;

(b-ii) the macula and blood vessels are invisible;

(c-i) the presence of a glass (hard or soft) within the macula (within the diameter of the two discs from the center of the macula);

(c-ii) map-like atrophy;

(c-iii) fibrous scars under the retina; (c-iv) pigment epithelial detachment;

(c-v) lesions in the subretinal fibrovascular (hemorrhage under the retina);

(c-vi) choroidal neovascular membrane;

(c-vii) exudative age-related macular degeneration;

(d-i) the cup-to-disk ratio of any eye is greater than or equal to 0.3;

(d-ii) the asymmetry of the disk is greater than or equal to 0.2;

(d-iii) bleeding from the disk;

(d-iv) the occurrence of any grooving or edge thinning;

(e-i) medium turbidity;

(e-ii) a blurred macula (which may cause a contracted pupil);

(e-iv) Insufficient focus and/or incorrect positioning of the optic disc and macula.