CN115496700A

CN115496700A - Disease detection system and method based on eye image

Info

Publication number: CN115496700A
Application number: CN202110607244.6A
Authority: CN
Inventors: 不公告发明人
Original assignee: Yanling Shanghai Intelligent Technology Co ltd
Current assignee: Yanling Shanghai Intelligent Technology Co ltd
Priority date: 2021-06-01
Filing date: 2021-06-01
Publication date: 2022-12-20

Abstract

The invention provides a disease detection system based on an eye image, which comprises: the human face image preprocessing module is used for preprocessing a human face image and obtaining an eye region image; the eye characteristic pathology database is used for storing an eye pathology characteristic knowledge map constructed according to Chinese traditional medical clinical experience knowledge and Western medicine pathology knowledge; the eye image feature extraction module is used for extracting basic features of the eye region picture according to the trained eye image feature extraction model; the eye image feature filtering module is used for filtering the basic features based on the eye pathological feature knowledge graph; and the disease semantic attribute analysis module is provided with a high-dimensional disease semantic attribute space constructed based on the eye pathological feature knowledge graph and used for detecting the disease according to the filtered basic features. The invention can conveniently and rapidly detect the diseases of the patient by shooting the eye region in the face picture.

Description

Disease detection system and method based on eye image

Technical Field

The invention relates to the field of medical image processing, in particular to a disease detection system and method based on an eye image.

Background

In the past decades, artificial intelligence techniques based on Deep Learning (DL) have made significant advances in various computer vision tasks, such as object detection, image classification, instance segmentation, and object recognition, among others. The advantages of deep learning make it widely used in medical image analysis, for example, to classify different diseases according to medical images. Known fields of application are not already autistic spectrum disorders or alzheimer's disease in the brain, breast cancer, diabetic retinopathy and glaucoma, and common conditions such as lung cancer or pneumonia.

At present, some work adopts a deep learning technology to learn and extract medical image characteristics to identify and screen disease patients, and good effects are achieved. However, the medical image capturing needs to be performed by using professional medical imaging equipment and needs to be operated by a professional, and meanwhile, since the capturing and imaging process takes a long time, it is impossible to rapidly extract the features of the medical image and complete the identification and screening of the disease patient. That is to say, the existing disease patient screening technology based on medical image features has the defects of poor timeliness, high equipment requirement, dependence on professional personnel and the like, and is difficult to meet the requirement of large-scale and rapid screening.

The eye is used as an observation window for the health condition of multiple organs, is the only organ of a human living body, and can observe blood vessels by naked eyes without wound, and by virtue of the advantages of anatomy and imaging, the pathological changes of internal organs of a human body are represented in the image of the eye, so that the health condition of organs such as an endocrine system, a heart blood vessel, a liver and the like can be reflected. However, image detection is mainly performed through fundus camera equipment at present, the cost is high, the operation difficulty is high, no eye image detection-based technology exists at present, and the accuracy and the stability of the eye image detection-based algorithm technology are not mature.

Disclosure of Invention

In order to solve the above problems, the present invention provides a disease detection system and method based on eye images, which can detect diseases of patients by photographing eye regions in a face picture. According to a first aspect of embodiments of the present invention, a disease detection system based on an eye image is provided, including:

the face image preprocessing module is used for cutting and preprocessing the face image to obtain an eye region picture;

the eye characteristic pathology database is used for storing an eye pathology characteristic knowledge map constructed according to experience knowledge of clinical observation and western medicine pathology knowledge;

the eye image feature extraction module is used for extracting basic features of the eye region picture according to the trained eye image feature extraction model;

the eye image feature filtering module is used for filtering the basic features based on the eye pathological feature knowledge graph;

and the disease semantic attribute analysis module is provided with a high-dimensional disease semantic attribute space constructed based on the eye pathological feature knowledge graph and is used for detecting the disease according to the filtered basic features.

In a possible implementation manner of the first aspect, the system further includes an image acquisition module, configured to acquire a face image.

In a possible implementation manner of the first aspect, the image capturing module is a terminal with a shooting function, and includes a mobile phone, a television, a digital camera, a personal computer, or a portable medical device.

In one possible implementation manner of the first aspect, the face image preprocessing module further includes a face position recognition module, a face image capturing module, and an eye image filtering module,

the face position identification module is used for acquiring a position area of a face in the face image and coordinates of key points of the face through a face detection model;

the face image intercepting module intercepts the eye regions of the face according to the face key point coordinates to obtain an eye intercepting picture;

the eye image screening module is used for screening the eye captured picture to obtain the eye region picture.

In a possible implementation manner of the first aspect, the coordinates of the facial key points include key points of left and right eyes of a human face, and the human face detection model obtains a maximum value and a minimum value of horizontal and vertical coordinates of the eye region of the human face according to the key points of the left and right eyes of the human face, and then cuts the eye region of the human face.

In a possible embodiment of the first aspect, the step of constructing the eye pathology knowledge base according to clinical experience of a specialist in traditional Chinese medicine further comprises:

a: labeling a knowledge graph training set: predefining 5 areas covering the eyes, labeling the eye features according to the 5 areas and forming vector codes of specific diseases;

b: attribute machine learning model procedure: and B, according to the knowledge graph training set formed by labeling in the step A, learning by using a machine learning model and taking the eye-based image as input, and predicting the attribute labeling of the 5 regions in the corresponding eye image.

Generating vector codes of specific diseases: and B, generating corresponding vector codes according to the step B, and further calculating distances of different disease vector codes by using attribute space clustering results to form a knowledge graph with disease attributes.

In a possible implementation manner of the first aspect, for a specific disease, a certain number of eye photos are collected, corresponding attribute labeling is performed according to the step B, then, an unsupervised clustering algorithm is used to cluster all data in an attribute space, and an arithmetic average value or a geometric average value of all sample data of a category cluster where the specific disease is located is used as a vector code of the specific disease.

In one possible embodiment of the first aspect, the specific disease includes, but is not limited to, new coronary pneumonia, diabetes, viral influenza, lung diseases, liver diseases, eye diseases.

In one possible embodiment of the first aspect, the eye features marked in step a include morphology, blood streak, turbid ring, color, speckle, vault size, dynamic changes in vault position structure, and collateral veins of the eye.

In a possible implementation manner of the first aspect, the machine learning model in step B includes a support vector machine, a neural network, a random tree forest, a logistic regression, and a linear regression.

In one possible implementation manner of the first aspect, the knowledge-graph of the ocular pathological features includes specific disease semantics, specific disease attribute definitions, and a corresponding relationship between a specific disease and ocular image features.

In a possible implementation manner of the first aspect, the high-dimensional disease semantic attribute space is embedded with a vector code and a knowledge graph thereof for defining the characterization of a specific disease by western medicine, and is embedded with a semantic attribute definition for defining the specific disease according to the expert experience description of the disease by traditional chinese medicine.

In a possible implementation manner of the first aspect, the eye image feature extraction model is obtained by training through a Hopfield network and a multi-layer perceptron.

In a possible implementation manner of the first aspect, the system further includes a visual aid decision module, configured to generate a corresponding thermodynamic diagram according to the model attention distribution of the ocular image feature filtering module.

In a possible implementation manner of the first aspect, the system further includes a system control module, configured to perform state control on the modules and perform message passing between the modules.

According to a second aspect of the embodiments of the present invention, there is provided an eye image-based disease detection method, including the steps of:

face image preprocessing, namely acquiring a face image, and cutting and preprocessing the face image to obtain an eye region image;

establishing an eye pathological characteristic knowledge map, namely establishing the eye pathological characteristic knowledge map according to the clinical observation experience of Chinese traditional medical experts and western medicine pathological knowledge, and storing to form an eye characteristic pathological database;

extracting eye image features, namely extracting basic features of the eye region picture according to the trained eye image feature extraction model;

filtering eye image features, and filtering the basic features according to the eye pathological feature knowledge graph;

and disease semantic attribute analysis, which is provided with a high-dimensional disease semantic attribute space constructed based on the eye pathological feature knowledge graph and detects the disease according to the filtered basic features.

In one possible embodiment of the second aspect, the method comprises:

and generating a visual thermodynamic diagram, and generating a corresponding thermodynamic diagram according to the model attention distribution in the eye image feature filtering step.

In one possible embodiment of the second aspect, the face image is acquired before the face image is preprocessed.

In a possible implementation manner of the second aspect, the face image preprocessing step further includes:

acquiring a position area of a human face in a human face image and coordinates of key points of the human face through a human face detection model;

obtaining the maximum value and the minimum value of horizontal and vertical coordinates of the human face eye region according to the human face left and right eye key points in the face key point coordinates, simultaneously carrying out outward expansion on the horizontal and vertical coordinates by a certain numerical value to ensure that the human face eye region is completely included in the feature extraction range, and intercepting the human face eye region to obtain an eye intercepting picture;

and screening the eye captured picture to obtain an eye region picture.

In a possible implementation manner of the second aspect, the coordinates of the face key points include key points of left and right eyes of the human face, and the human face detection model obtains a maximum value and a minimum value of horizontal and vertical coordinates of the eye regions of the human face according to the key points of the left and right eyes of the human face, so as to intercept the eye regions of the human face.

In a possible implementation manner of the second aspect, the knowledge-graph of the ocular pathological features includes specific disease semantics, specific disease attribute definitions and correspondence between specific diseases and ocular image features.

In a possible implementation manner of the second aspect, the high-dimensional disease semantic attribute space is embedded with the representation definition of western medicine for a specific disease to form a vector code of the specific disease and a knowledge graph thereof, and is embedded with the semantic attribute definition of the specific disease formed according to the expert experience description of Chinese traditional medicine for the disease.

In a possible implementation manner of the second aspect, the eye image feature filtering step further includes:

based on the extracted basic features, a linear classifier is adopted to predict the result;

performing supervised learning on an eye image feature extraction model and classification features according to eye image feature filtering results and real diseased categories of eye region pictures by constructing a loss function;

and performing iterative training on the eye image feature extraction model by adopting an SGD (generalized Gaussian decomposition) optimizer.

According to a third aspect of the embodiments of the present invention, an electronic device is further provided, where the electronic device includes a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus; a memory for storing a computer program; a processor for implementing the method of the second aspect of the above embodiments when executing the program stored on the memory.

According to a third aspect of the embodiments of the present invention, a computer-readable storage medium is further proposed, in which a computer program is stored, and the computer program, when executed by a processor, implements the method steps in the second aspect of the embodiments.

By adopting the technical scheme of the embodiment of the invention, the common camera can be used for collecting images without using professional medical imaging equipment and operating without depending on professionals, and the disease detection of patients with various diseases (such as non-new coronary pneumonia, liver diseases, diabetes internal metabolism and visceral diseases and the like) based on the eye characteristics can be conveniently, quickly and accurately realized.

Drawings

The foregoing summary, as well as the following detailed description, will be better understood when read in conjunction with the appended drawings. For the purpose of illustration, certain embodiments of the disclosure are shown in the drawings. It should be understood, however, that the invention is not limited to the precise arrangements and instrumentalities shown. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate implementations of systems and apparatus according to the invention and, together with the description, serve to explain the advantages and principles of the invention.

FIG. 1 is a functional block diagram of a disease detection system according to one embodiment of the present invention;

FIG. 2 is a functional block diagram of a disease detection system according to one embodiment of the present invention;

FIG. 3 is a flow diagram of a disease detection method according to one embodiment of the invention;

FIG. 4 is a flow diagram of a method of pre-processing a portrait according to one embodiment of the present invention;

FIG. 5 is a flow diagram of a high-dimensional disease semantic attribute space construction according to one embodiment of the invention;

FIG. 6 is a schematic illustration of the five regions A-E in an ocular approach according to one embodiment of the present invention;

FIG. 7 is a hardware system diagram according to one embodiment of the invention;

FIG. 8 is an electronic device according to one embodiment of the invention;

FIG. 9 is a schematic diagram of a disease detection process according to an embodiment of the present invention.

Detailed Description

The present invention will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

Fig. 1 is a functional block diagram of a disease detection system 100 according to an embodiment of the present invention. As shown in fig. 1, the disease detection system of the present embodiment is used for screening disease risks of a patient by taking an eye region in a face picture, and includes a face image preprocessing module 101, an eye feature and pathology database 102, an eye image feature extraction module 103, an eye image feature filtering module 104, a disease semantic attribute analysis module 105, a visualization assistant decision module 106, and a system control module 107. The face image preprocessing module 101 is configured to cut and preprocess the face image to obtain an eye region picture; as shown in fig. 2, the face image preprocessing module further includes a face position recognition module 1011, a face image capturing module 1012 and an eye image screening module 1013, where the face position recognition module 1011 is configured to obtain a position area of a face in the face image and coordinates of key facial points through a face detection model; the face image capture module 1012 captures the eye region of the face according to the coordinates of the key points of the face to obtain an eye captured image; the face detection model obtains the maximum value and the minimum value of the horizontal and vertical coordinates of the eye region of the face according to the key points of the left eye and the right eye of the face, and intercepts the eye region of the face. The eye image screening module 1013 is configured to screen the eye capture picture to obtain the eye region picture.

The eye characteristic pathology database 102 is used for storing an eye pathology characteristic knowledge map constructed according to experience knowledge of traditional Chinese medicine clinical observation and western medicine pathology knowledge. According to the traditional Chinese medicine, the eye diagnosis method is referenced, and the method is recorded in the Taiping Shenghui Fang eye treatise: "liver patients, corresponding to windwheel, heart patients, blood wheel, spleen patients, meat wheel, lung patients, qi wheel, kidney patients, and water wheel". 8978 Zxft 8978 Renzhai Zhi Fang Lun points out: eyes belong to five zang organs, red heads and tails belong to heart, eyes are full of white eyes and lung, upper and lower meatballs belong to spleen, and the middle part of the eyes is black pupil like paint, and kidney excess is dominant. By combining the traditional Chinese medical experience, the eye part of the five rounds is definitely divided into 5 main parts. According to the above eye diagnosis method, in this embodiment, 5 different eye regions are respectively adopted for feature extraction, and the 5 regions are respectively defined as: the area A, the area B, the area C, the area D and the area E, wherein the area C is an area containing the black eyes and the white eyes, and the rest areas of the area A, the area D, the area E and the area B are respectively positioned in the upper area, the lower area, the left area and the right area of the area C and can be overlapped with the area C to a certain extent, as shown in fig. 6.

According to our empirical data analysis, the extracted features of the above 5 ocular regions reflect the body of different human internal organs: the region a mainly reflects the condition of the spleen, the region B mainly reflects the condition of the heart, the region C mainly reflects the condition of the liver and the kidney, the region D mainly reflects the overall mental condition of the human body, and the region E mainly reflects the condition of the lung and the liver.

The objective diagnosis method according to the present embodiment is a holographic diagnosis method for diagnosing lesions, injuries and disorders of various parts of a body where diseases are located, based on the morphology of each part of the eye of a patient, the size of blood streak, turbid annulus, color, spots, fornices, dynamic changes in the position structure, collateral fine collaterals of the eye, and the like. The method for constructing the knowledge graph aiming at the Chinese traditional medical theory in the embodiment specifically comprises the following steps:

(1) Labeling a knowledge graph training data set: firstly, the Chinese medicine experts label according to the five-round area in the figure 6 according to the clinical experience of the Chinese medicine experts, and form vector codes aiming at specific diseases according to the labels of the shape, blood silk, turbid ring, color, spot, the size of fornix, dynamic change of position structure, collateral fine collaterals of eyes and the like.

TABLE 1 labeling five regions A-E to form a vector code for a particular disease

(2) Attribute machine learning model procedure: and predicting the attribute labels of the five regions A-E of the corresponding eyes by using the five region images A-E, the whole eye image and the corresponding attribute labels as sample labels and using the machine learning model learning and the eye image as input. The machine learning model adopted by the method comprises machine learning algorithms such as a support vector machine, a neural network, a random tree forest, a logistic regression and a linear regression.

(3) Clustering of specific diseases: for some specific diseases, such as but not limited to new coronary pneumonia, diabetes, viral influenza, lung diseases, liver diseases, eye diseases, etc., by collecting eye photographs of a small sample of a patient, predicting each eye photograph according to step (2), and performing corresponding attribute labeling. Further using unsupervised clustering algorithm (such as K-means, etc.), clustering all data in attribute space. Generally, each disease is grouped into a class cluster, and the arithmetic mean or geometric mean of all sample data is used as the vector code for that particular disease.

(4) Generating a disease-specific vector code: for each disease, corresponding vector codes can be generated according to the step (3), and the distance (such as Euclidean distance, angle distance and the like) of different disease vector codes is calculated by further utilizing the attribute space clustering result, so that the knowledge graph with the disease attribute is finally formed.

The eye pathology feature knowledge graph comprises specific disease semantics, specific disease attribute definition and a corresponding relation between specific diseases and eye image features.

The high-dimensional disease semantic attribute space is embedded into the vector code and the knowledge graph of the western medicine for the characterization definition of the specific disease to form the specific disease, specifically, pathological feature description in the modern western medicine bulbar conjunctiva microcirculation theory, clinical observation and research statistics of eyes of specific disease patients, enlightenment and correction annotation of eye features by medical experts, eye images are used as input, and machine learning models (such as a support vector machine, a deep neural network and the like) are used for learning and outputting the medical expert annotation.

The eye image feature extraction module 103 is configured to perform basic feature extraction on the eye region picture according to a trained eye image feature extraction model, where the eye image feature extraction model is obtained by training through a Hopfield network and a multi-layer sensor. The eye image feature filtering module 104 is configured to filter the basic features based on the eye pathology feature knowledge graph; the disease semantic attribute analysis module is provided with a high-dimensional disease semantic attribute space constructed based on the eye pathological feature knowledge graph and is used for detecting the disease according to the filtered basic features.

In another embodiment, the disease detection system further includes an image acquisition module for acquiring a face image. The image acquisition module is a terminal with a shooting function, and comprises a mobile phone, a television, a digital camera, a personal computer or portable medical equipment.

In another embodiment, the system further comprises a visual aid decision module for generating a corresponding thermodynamic diagram according to the model attention distribution of the ocular image feature filtering module.

In another embodiment, a system control module 107 is further included for performing state control and message passing between the modules.

Another embodiment of the present invention provides a disease detection method based on eye images, and the specific steps are shown in fig. 3. The disease detection method comprises the following steps:

s1, preprocessing a face image, acquiring the face image, cutting and preprocessing the face image to obtain an eye region image;

first, to ensure the uniformity of model input, the input pictures must first be aligned through preprocessing. In this embodiment, the face image preprocessing unit 1 is configured to preprocess a face image and obtain an eye region image.

The face image preprocessing unit 1 is configured with a face detection model, and the face image preprocessing unit 1 preprocesses a face image based on the face detection model, as shown in the preprocessing unit in fig. 4, the preprocessing includes the following steps:

s11: and acquiring the position area of the face and the coordinates of key points of the face in the face picture through the face detection model. It should be appreciated that due to the non-standard nature of data acquisition, the original face picture typically contains not only the eye region, but may also include other regions of the background or face, such as the nose, ears, and mouth. If the original face picture is directly used for classification, noise or irrelevant information such as background noise or nose features are necessarily introduced, which may cause the extraction result of the eye image features to be inaccurate and unreliable. Therefore, in order to focus on extracting eye image features, the face detection model is used in the present embodiment to obtain the location area of the face and the coordinates of the key points of the face in the picture. The coordinates of the key points of the face comprise the position coordinates of the left eye and the right eye of the face and other preferred organs of the face.

S12: and calculating to obtain the maximum value and the minimum value of the horizontal and vertical coordinates of the human face eye region according to the coordinates of the human face left and right eye key points in the coordinates of the facial key points. In this embodiment, it is preferable that the horizontal and vertical coordinates are extended outward by a certain value to ensure that the eye region of the human face is all included in the feature extraction range, and then the eye region of the human face is captured to obtain an eye capture picture.

S13: and screening the eye captured picture to obtain an eye region picture. When the eye-captured images are screened, preferably, the images with the longitudinal length longer than the transverse length in the eye-captured images are removed, and then the eye region images are obtained. It should be understood that, considering that the images of the face angle, the background noise, etc. exist, the coordinates of the key points of the face obtained by the face detection model are not completely accurate, so that the interception of the eye region of the face has corresponding deviation; considering that the eye region of the human face should be a rectangular region with a long transverse direction, it is preferable that the eye-captured picture should be removed if the longitudinal length of the eye-captured picture is longer than the transverse length of the eye-captured picture.

S2, constructing an eye pathological characteristic knowledge map, constructing the eye pathological characteristic knowledge map according to Chinese traditional medical expert experience and Western medicine pathological knowledge, and storing to form an eye characteristic pathological database; the eye pathological feature knowledge graph comprises specific disease semantics, specific disease attribute definition and a corresponding relation between specific diseases and eye image features.

The high-dimensional disease semantic attribute space is embedded with vector codes and knowledge maps thereof for characterizing and defining specific diseases by western medicine, and is embedded with semantic attribute definitions for forming specific diseases according to expert experience description of Chinese traditional medicine on the diseases.

S3, extracting eye image features, namely extracting basic features of the eye region picture according to the trained eye image feature extraction model; in this embodiment, the eye image feature extraction step is used to input the eye region picture into the eye image feature extraction model to extract the basic features. Preferably, a neural network is used to extract the higher order features and used as input to the classifier.

S4, filtering the eye image features, and filtering the basic features according to the eye pathological feature knowledge graph; the image-level classification unit 3 predicts whether the corresponding patient in the eye region image has multiple types of diseases according to the basic features and outputs the image-level classification result, and the data processing flow comprises the following parts:

s41: and inputting the eye region picture into an eye image feature extraction model to extract basic features, predicting multiple types of diseased classes of corresponding patients in the eye region picture according to the basic features by a classification part, and outputting a picture-level classification result. In this embodiment, the classification result is to determine whether the patient corresponding to the eye region image has a certain disease (e.g., hepatitis, new coronary pneumonia, non-new coronary pneumonia, diabetes, etc.). It should be understood that this determination is not a diagnosis of the disease, but rather a labeling of the input image of the eye region based on the eye features, e.g., features of the eye, that the patient may have.

S42: and (3) supervising the basic features extracted by the eye image feature extraction model according to the picture-level classification result and the real diseased category of the patient corresponding to the eye region picture by constructing a loss function (such as cross entropy).

S43: and (3) iteratively training the eye image feature extraction model by adopting an SGD (stochastic gradient descent) optimizer. For example, a momentum-carrying SGD optimizer is employed to increase the training speed.

It should be understood that in other embodiments of the present invention, the eye image feature filtering may not include two parts of feature supervision (S41) and iterative training (S42), but uses a fully trained eye image feature extraction model.

And S5, analyzing the semantic attributes of the diseases, wherein the semantic attributes of the diseases have a high-dimensional semantic attribute space of the diseases constructed based on the knowledge graph of the pathological features of the eyes, and the diseases are detected according to the filtered basic features. The high-dimensional disease semantic attribute space can carry out causal inference and relevance analysis according to Chinese traditional medical experience knowledge, so that Chinese traditional medical explanation of specific diseases and related disease description are obtained. Meanwhile, according to the specific disease vector code and the knowledge graph obtained by western medical characterization definition, a western medical explanation for the specific disease can be obtained, and is specifically shown in fig. 5. The specific construction process of the knowledge graph is described in detail in the previous section, and is not described herein again.

With continued reference to fig. 3, the disease semantic attribute analysis step is used to perform patient-level classification, which employs highest-priority voting decisions. Specifically, the most urgent disease among the disease categories is set as the highest priority disease category, and when the number of the highest priority disease categories among the predicted picture-level classification results is 1 or more among a plurality of eye region pictures of one patient, it is determined that the patient has a high possibility of suffering from the disease, and a disease-level prediction result is obtained.

It should be understood that in other embodiments of the present invention, the analysis of the semantic attributes of the disease may not be integrated, but the output of the filtering of the ocular image features may be presented to medical personnel for judgment.

And a visualization aid decision step S6, visualizing and displaying the importance of each region preferably in a thermodynamic diagram mode according to the weight and influence of each region of the image participating in score evaluation in the model classification process, so as to enhance the interpretability of the model. Reference may be made to fig. 8 for details of the thermodynamic diagram.

It should be appreciated that the visualization aid decision may also be used to aid medical personnel in making the determination. It should also be understood that in other embodiments of the invention, the visualization aid decision step may not be provided.

Fig. 7 shows a hardware system diagram of the embodiment of fig. 1. As shown in fig. 7, the system is divided into two parts, namely a server and a client. In a preferred embodiment, the computational model is deployed on the server side. The computer equipment of the server side comprises a processor and a memory: the processor is a hardware processor for computing and executing executable codes, such as a Central Processing Unit (CPU) or a graphic computation processor (GPU); the memory is a non-volatile storage device for storing executable code to cause the processor to perform corresponding computing processes. Meanwhile, the memory also stores various intermediate data and parameters. The memory storage content comprises model related parameters and executable codes. The hard disk stores training data required by the model. The service container runs on the computing and storage resources of the server, and provides underlying support for the risk screening deep learning model of the multiple types of disease patients.

In one embodiment, the server side adopts two servers, namely an A server and a B server, which are across countries to process and transmit data. For the A server in one country with a detection model, the high-dimensional analysis data (HDA) is formed mainly by inputting the collected human face data and according to a machine learning model and is transmitted to the B server in the other country with a matrix model. The server B calculates the received HDA, predicts diseases and outputs the results to the detection model. The server a has less demand on computing resources and can be deployed on the terminal device, while the server B needs stronger computing power and should be deployed on a server with a GPU.

In this embodiment, the media data is obtained by shooting through various data acquisition devices, such as a smart phone, and the media data may be video content or image content. Of course, it should be understood that in other embodiments of the present invention, a media data collection device, such as a camera, may be incorporated into the system of the present invention. Preferably, the face image data is derived from media data. More preferably, the face image data comprises face image data of a plurality of patients of different identities and disease types.

Fig. 8 shows a schematic structural diagram of an electronic device according to an embodiment of the invention. The electronic device 200 may include: a processor (processor) 201, a memory (memory) 202 and a communication bus 203, wherein the processor 201 and the memory 202 communicate with each other via the communication bus 203.

The processor 201 may call a computer program in the memory 202 to execute the steps of any one of the eye image-based disease detection methods provided by the above embodiments.

An embodiment of the present invention provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements the steps of any one of the above-described embodiments of the method for detecting a disease based on an eye image.

In addition, the logic instructions in the memory may be implemented in the form of software functional units and may be stored in a computer readable storage medium when sold or used as a stand-alone product. Based on such understanding, the technical solution of the present invention or a part thereof which substantially contributes to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the above methods of the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, and various media capable of storing program codes.

FIG. 9 shows a schematic diagram of a disease detection process according to an embodiment of the invention. As shown, after the pre-processed eye picture is input into the picture-level classification model, a probability prediction distribution for disease classification is generated. In addition, the key area based on which the determination is made is also presented in the form of a thermodynamic diagram. In which the thermodynamic diagram is the highest in the right eye for the eye features in the red region, and is lower in the left eye for the eye features.

According to the disease patient risk screening deep learning system based on the eye features, because the disease category prediction of picture level and disease level can be performed through the basic features extracted by the eye image feature extraction model by means of the feature extraction and classification prediction capabilities of the deep learning network according to the conjunctivitis-like expression of the eyes of the disease patient, including the features of conjunctival congestion, blood stasis, excessive discharge or secretion increase, the learning expression of the features is performed aiming at the eye region of the human face, and the disease patient risk screening based on the eye features can be realized by capturing the features with higher resolution and identification power; the invention can carry out screening work of disease patients by shooting the face picture and according to the eye region picture in the face picture, can improve the rapidness, accuracy and convenience of disease risk screening, can get rid of the limitation of dependence of professionals and the like, can be popularized in a large scale, can realize quantitative detection at any time and any place in an epidemic situation stage, dynamically monitor the degree of virus infection, observe the treatment effect, and carry out epidemic situation tracking and epidemic situation map drawing, thereby realizing high-efficiency epidemic situation prevention and control.

Furthermore, because the eye region pictures are obtained by preprocessing and screening the face pictures and then input into the eye image feature extraction model, the input pictures can be ensured to be the eye region pictures with accurate positioning, the noise and irrelevant information in the face pictures are effectively removed, and the eye region pictures can be correctly processed by the eye image feature extraction model.

It should be noted that the above detailed description is exemplary and is intended to provide further explanation of the disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the exemplary embodiments described above in accordance with the present application. As used herein, the singular is intended to include the plural unless the context clearly dictates otherwise. Furthermore, it will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof.

It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in other sequences than those illustrated or otherwise described herein.

Furthermore, the terms "comprising" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements explicitly listed, but may include other steps or elements not explicitly listed or inherent to such process, method, article, or apparatus.

For ease of description, spatially relative terms such as "over … …", "over … …", "over … …", "over", etc. may be used herein to describe the spatial positional relationship of one device or feature to another device or feature as shown in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if a device in the figures is turned over, devices described as "above" or "on" other devices or configurations would then be oriented "below" or "under" the other devices or configurations. Thus, the exemplary term "above … …" may include both orientations of "above … …" and "below … …". The device may also be oriented in other different ways, such as by rotating it 90 degrees or at other orientations, and the spatially relative descriptors used herein interpreted accordingly.

In the foregoing detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, like numerals typically identify like components, unless context dictates otherwise. The illustrated embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the spirit or scope of the subject matter presented here.

The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A disease detection system based on an eye image, comprising:

the eye characteristic pathology database is used for storing an eye pathology characteristic knowledge map constructed according to the clinical experience knowledge of Chinese medicine experts and the western medicine pathology knowledge;

and the disease semantic attribute analysis module is provided with a high-dimensional disease semantic attribute space constructed based on the eye pathological feature knowledge graph and used for detecting the diseases according to the filtered basic features.

2. The eye image-based disease detection system of claim 1, further comprising an image acquisition module for acquiring a face image.

3. The eye image-based disease detection system of claim 2, wherein the image capturing module is a terminal with a camera function, including a mobile phone, a television, a digital camera, a personal computer or a portable medical device.

4. The eye image-based disease detection system of claim 1, wherein said face image preprocessing module further comprises a face location recognition module, a face image clipping module, and an eye image filtering module,

the eye image screening module is used for screening the eye intercepted picture to obtain the eye region picture.

5. The eye-image-based disease detection system of claim 4, wherein the coordinates of the facial key points comprise human face left and right eye key points, and the human face detection model captures the eye regions of the human face by obtaining the maximum and minimum values of the horizontal and vertical coordinates of the eye regions of the human face according to the human face left and right eye key points.

6. The eye image-based disease detection system of claim 1, wherein the knowledge-map of ocular pathological features comprises disease-specific semantics, disease-specific attribute definitions, and disease-specific to eye image feature correspondences.

7. The eye image-based disease detection system of claim 1, wherein the high-dimensional disease semantic attribute space embeds the characterization definition of western medicine for a specific disease to form a vector code of the specific disease and its knowledge graph, and embeds the semantic attribute definition of the specific disease according to the expert experience description of chinese traditional medicine for the disease.

8. The eye image-based disease detection system of claim 1, wherein the eye image feature extraction model is obtained by a Hopfield network, a multi-layer perceptron.

9. The ocular image-based disease detection system of claim 1, further comprising a visualization aid decision module for generating a corresponding thermodynamic map from the model attention distribution of the ocular image feature filtering module.

10. The eye-image-based disease detection system of claim 1, further comprising a system control module for performing state control and message passing between modules.

11. A disease detection method based on an eye image is characterized by comprising the following steps:

preprocessing a face image, namely acquiring the face image, and cutting and preprocessing the face image to obtain an eye region image;

establishing an eye pathological characteristic knowledge map, namely establishing the eye pathological characteristic knowledge map according to the clinical experience of Chinese medical experts and western medical pathological knowledge, and storing the eye pathological characteristic knowledge map to form an eye characteristic pathological database;

and the disease semantic attribute analysis is provided with a high-dimensional disease semantic attribute space constructed based on the eye pathological feature knowledge graph, and the disease is detected according to the filtered basic features.

12. The method of eye image-based disease detection according to claim 11, comprising:

13. The eye image-based disease detection method of claim 11, wherein the face image is acquired before the face image is preprocessed.

14. The method of claim 11, wherein the disease detection based on eye images,

the face image preprocessing step further comprises:

acquiring a position area of a face and coordinates of key points of the face in a face image through a face detection model;

and screening the eye captured picture to obtain an eye region picture.

15. The eye image-based disease detection method according to claim 14,

the face key point coordinates comprise key points of left and right eyes of the face, and the face detection model obtains the maximum value and the minimum value of horizontal and vertical coordinates of the eye region of the face according to the key points of the left and right eyes of the face, so that the eye region of the face is intercepted.

16. The method of eye image-based disease detection according to claim 11, wherein the step of constructing the eye pathology knowledge base according to clinical experience of a specialist in traditional Chinese medicine further comprises:

b: attribute machine learning model procedure: and B, according to the knowledge graph training set formed by labeling in the step A, learning by using a machine learning model and taking the eye image as input, and predicting the attribute labeling of the 5 regions in the corresponding eye image.

17. The method according to claim 16, wherein for a specific disease, a certain number of eye photographs are collected, corresponding attribute labeling is performed according to the step B, and then all data are clustered in an attribute space by using an unsupervised clustering algorithm, and an arithmetic mean or a geometric mean of all sample data of a category cluster where the specific disease is located is used as a vector code of the specific disease.

18. The method of claim 17, wherein the specific disease includes but is not limited to new crown pneumonia, diabetes, viral influenza, lung diseases, liver diseases, and eye diseases.

19. The method according to claim 16, wherein the eye features labeled in step a include morphology, blood streak, turbid ring, color, speckle, vault size, dynamic changes in vault position structure, and collateral branches of the eye.

20. The method of claim 17, wherein the machine learning model of step B comprises support vector machine, neural network, random tree forest, logistic regression, and linear regression.

21. The method of claim 11, wherein the knowledge-map of ocular pathological features comprises disease-specific semantics, disease-specific attribute definitions, and disease-specific to ocular image features.

22. The eye image-based disease detection method of claim 11, wherein the high-dimensional disease semantic attribute space embeds vector codes and their knowledge maps that define the characterization of certain diseases by western medicine, and embeds semantic attribute definitions that define certain diseases according to the expert experience description of diseases by traditional chinese medicine.

23. The eye image-based disease detection method of claim 11,

the eye image feature filtering step further comprises:

based on the extracted basic features, adopting a linear classifier to predict results;

by constructing a loss function, according to the eye image feature filtering result and the real diseased category of the eye region picture, performing supervised learning on an eye image feature extraction model and classification features;

and performing iterative training on the eye image feature extraction model by adopting an SGD optimizer.

24. An electronic device, characterized in that the electronic device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus; a memory for storing a computer program; a processor for implementing the method of any one of claims 11 to 23 when executing a program stored on a memory.

25. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of the claims 11-23.