WO2022041396A1 - 一种基于眼表特征的新冠肺炎（covid-19）疾病患者风险筛查深度学习系统 - Google Patents

一种基于眼表特征的新冠肺炎（covid-19）疾病患者风险筛查深度学习系统 Download PDF

Info

Publication number
WO2022041396A1
WO2022041396A1 PCT/CN2020/119241 CN2020119241W WO2022041396A1 WO 2022041396 A1 WO2022041396 A1 WO 2022041396A1 CN 2020119241 W CN2020119241 W CN 2020119241W WO 2022041396 A1 WO2022041396 A1 WO 2022041396A1
Authority
WO
WIPO (PCT)
Prior art keywords
picture
eye
disease
face
covid
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/119241
Other languages
English (en)
French (fr)
Inventor
付彦伟
薛向阳
顾梦炜
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fudan University
Original Assignee
Fudan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fudan University filed Critical Fudan University
Publication of WO2022041396A1 publication Critical patent/WO2022041396A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7271—Specific aspects of physiological measurement analysis
    • A61B5/7275—Determining trends in physiological measurement data; Predicting development of a medical condition based on physiological measurements, e.g. determining a risk factor
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems

Definitions

  • the invention belongs to the cross field of medical image and computer image recognition, and in particular relates to a deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface features.
  • COVID-19 new coronary pneumonia
  • DL-based artificial intelligence techniques have made remarkable progress in various computer vision tasks such as object detection, image classification, instance segmentation, and object recognition.
  • the present invention is made to solve the above problems, and aims to provide a deep learning system for risk screening of disease patients based on ocular surface characteristics.
  • the present invention provides a deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface features, which is used to screen the disease risk of patients by using the eye area in a photographed face picture, It has such features, including: a face image preprocessing part, which is used to preprocess the face image and obtain an eye area image; an eye image feature extraction part, which is used for extracting through the trained eye image feature extraction model.
  • the basic features of eye area pictures; the classification part is used to predict the disease category of the new coronary pneumonia (COVID-19) at the picture level according to the basic features and obtain the picture-level classification results, as well as the patient-level classification results based on the picture-level classification results.
  • the patient-level prediction results are obtained by classification, and the training of the eye image feature extraction model includes the following steps:
  • Step S1 input the eye region picture into the eye image feature extraction model to extract basic features, and the classification unit predicts whether the corresponding patient in the eye region picture suffers from new coronary pneumonia (COVID-19) according to the basic features and outputs the picture-level classification result; step S2, by constructing a loss function, supervise the basic features extracted by the eye image feature extraction model according to the picture-level classification results and the actual disease category of the patient corresponding to the eye area image; step S3, using the SGD optimizer to analyze the eye image
  • the feature extraction model is iteratively trained.
  • the face image preprocessing part also has a face detection model
  • the preprocessing of the face image by the face image preprocessing part includes the following steps: step E1, obtaining the position area of the face in the face image and the coordinates of the facial key points through the face detection model; step E2, according to the facial key point coordinates in the The key points of the left and right eyes of the face are obtained, and the maximum and minimum values of the horizontal and vertical coordinates of the face eye area are obtained.
  • the horizontal and vertical coordinates are expanded outward by a certain value to ensure that all the face and eye areas are included in the feature extraction range.
  • the eye intercepted pictures are The pictures with the longitudinal length longer than the horizontal length are removed to obtain the pictures of the eye region.
  • the deep learning system for risk screening of disease patients based on ocular surface features may also have the following feature: when the classification unit performs patient-level classification, the most urgent disease in the disease category is set as the highest Priority disease category, for a patient's multiple eye region pictures, when the number of the highest priority disease category in the predicted picture-level classification results is greater than or equal to 1, it is determined that the patient has the disease. It is more likely to obtain patient-level prediction results.
  • the deep learning system for risk screening of new coronary pneumonia (COVID-19) disease patients based on ocular surface characteristics involved in the present invention because the ocular surface of the disease patients often has conjunctivitis-like manifestations, including conjunctival congestion, stasis removal, overflow
  • the basic features extracted by the eye image feature extraction model are used to predict the disease category at the picture level and the patient level.
  • the learning and expression of features is carried out in the region, and the risk screening of disease patients based on eye features can be realized by capturing more distinguishing and discriminating features; Pictures can be used to screen patients with diseases, which can improve the speed, accuracy and convenience of disease risk screening. At the same time, it can get rid of restrictions such as dependence on professionals. Detection, dynamic monitoring of the degree of virus infection, observation of treatment effects, epidemic tracking and epidemic mapping, so as to achieve efficient epidemic prevention and control.
  • FIG. 1 is a structural block diagram of a deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface features in an embodiment of the present invention
  • FIG. 2 is a system diagram of a deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface characteristics in an embodiment of the present invention
  • FIG. 3 is a flowchart of face picture preprocessing performed by the deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface features in an embodiment of the present invention
  • FIG. 4 is a flowchart of training an eye image feature extraction model in the deep learning system for risk screening of new coronary pneumonia (COVID-19) disease patients based on ocular surface features in an embodiment of the present invention
  • FIG. 5 is a schematic diagram of the disease screening process of the deep learning system for risk screening of new coronary pneumonia (COVID-19) disease patients based on ocular surface characteristics in an embodiment of the present invention
  • FIG. 6 is a block diagram of a disease screening process of the deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface characteristics in an embodiment of the present invention.
  • FIG. 1 is a structural block diagram of a deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface characteristics in an embodiment of the present invention.
  • COVID-19 new coronary pneumonia
  • the deep learning system 100 for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface features is used to screen the disease risk of patients by taking the eye area in the obtained face picture , including a face image preprocessing part 1, an eye image feature extraction part 2, a classification part 3, a disease risk screening evaluation part 4, an output display part 5, a system communication part 6 and a system control part 7 for controlling the above parts .
  • the deep learning system 100 for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface characteristics is composed of a computer device 110 and a result display device 140, and is used for external media data. to be processed.
  • the computer device is composed of a processor 120 and a memory 130: the processor 120 is a hardware processor for computing and running executable codes, such as a central processing unit CPU or a graphics computing processor GPU; the memory 130 is a non-volatile
  • the storage device is used to store executable codes so as to allow the processor 120 to perform corresponding computing processes. Meanwhile, the memory 130 also stores various intermediate data and parameters.
  • the storage content of the memory 130 includes a face data set storing face pictures and related parameters, executable codes, and the executable codes are used to run the face image preprocessing part 1 , the eye image feature extraction part 2 and the classification part 3 .
  • the media data is captured by various data collection devices, such as a smart phone, and the media data may be video content or image content.
  • the media data may also be face image data including multiple different identities and disease types, and the face pictures are obtained from the media data.
  • the face image preprocessing part 1 is used to preprocess the face image and obtain the eye region image.
  • the face image preprocessing part 1 has a face detection model, and the face image preprocessing part 1 preprocesses the face image based on the face detection model, as shown in Figure 3, the preprocessing comprises the following steps:
  • step S1 the position area of the face in the face picture and the coordinates of the key points of the face are obtained through the face detection model.
  • the original face picture usually includes not only the eye area, but also the background or other areas of the face, such as the nose, ears, and mouth. If the original face image is directly used for classification, it will inevitably introduce noise or irrelevant information, such as background noise or nose features, making the performance of the eye image feature extraction model inaccurate and unreliable. Therefore, in order to make the eye image feature extraction model focus on extracting eye surface features, the present embodiment adopts the face detection model to obtain the position area of the face in the picture and the coordinates of the facial key points, and the facial key point coordinates include the left and right eyes of the human face. And the position coordinates of other organs of the face.
  • Step S2 according to the key points of the left and right eyes of the face in the key point coordinates of the face, obtain the maximum value and the minimum value of the horizontal and vertical coordinates of the eye region of the human face, and at the same time, the horizontal and vertical coordinates are expanded outward by a certain value to ensure the face.
  • the eye area is all included in the feature extraction range, and the eye area of the face is intercepted to obtain an eye interception image.
  • Step S3 Screen the eye intercepted pictures to obtain eye region pictures.
  • the pictures of the eye clipped pictures whose vertical length is longer than the horizontal length are eliminated to obtain the eye region pictures.
  • the facial key point coordinates obtained by the face detection model are not completely accurate, so the interception of the face eye region will also have corresponding deviations.
  • the eye area of the human face should be a rectangular area with a longer horizontal length, any vertical length longer than horizontal in the aspect ratio is considered to be the result of the failure to locate the eye area of the human face, so such eyes need to be intercepted. Image culling.
  • the eye image feature extraction part 2 is used for extracting the basic features of the eye region picture through the trained eye image feature extraction model.
  • FIG. 4 is a flowchart of training an eye image feature extraction model in the deep learning system for risk screening of patients with novel coronavirus pneumonia (COVID-19) disease based on ocular surface features in an embodiment of the present invention.
  • COVID-19 novel coronavirus pneumonia
  • the training of the eye image feature extraction model includes the following steps:
  • Step E1 input the eye area image into the eye image feature extraction model to extract basic features, and the classification unit predicts the new coronary pneumonia (COVID-19) disease category of the corresponding patient in the eye area image according to the basic features and outputs the image-level classification result .
  • COVID-19 new coronary pneumonia
  • Step E2 by constructing a loss function, supervise the basic features extracted by the eye image feature extraction model according to the picture-level classification result and the actual disease category of the patient corresponding to the eye region picture.
  • Step E3 using the SGD optimizer to iteratively train the eye image feature extraction model.
  • the classification part 3 is used to perform picture-level prediction of the disease category of the novel coronavirus pneumonia (COVID-19) according to the basic features and obtain picture-level classification results, and to perform patient-level classification according to the picture-level classification results to obtain patient-level prediction results.
  • COVID-19 novel coronavirus pneumonia
  • the picture-level disease category prediction is to determine whether the patient corresponding to the eye region picture has a disease or other disease (for example, common pneumonia or eye disease, etc.).
  • a disease or other disease for example, common pneumonia or eye disease, etc.
  • the classification unit 3 When the classification unit 3 performs patient-level classification, the most urgent disease in the disease category is set as the highest-priority disease category. For a patient's multiple eye region pictures, when the predicted picture-level classification result When the number of the highest priority disease category is greater than or equal to 1, it is judged that the patient is more likely to suffer from the disease, and a patient-level prediction result is obtained.
  • the disease risk screening and evaluation unit 4 is configured to perform disease risk evaluation according to the picture-level classification result and the patient-level prediction result.
  • the result display device includes an output display unit 5 for playing media data and displaying picture-level classification results and patient-level prediction results, which can be a computer, a TV, or a mobile device.
  • FIG. 5 is a schematic diagram of a disease screening process flow of the deep learning system for risk screening of disease patients based on ocular surface features in an embodiment of the present invention
  • FIG. 6 is an embodiment of the present invention. 19) A block diagram of the disease screening process of the deep learning system for risk screening of disease patients.
  • the disease screening process of the deep learning system for risk screening of new coronary pneumonia (COVID-19) disease patients based on ocular surface characteristics of this embodiment includes the following steps:
  • Step T1 the face image preprocessing part 1 preprocesses the captured face image to obtain an eye region image, and then enters step T2;
  • Step T2 the eye image feature extraction part 2 extracts the basic features of the eye region picture according to the eye region picture, and then enters step T3;
  • Step T3 the classification unit 3 performs picture-level disease category prediction according to the basic features, predicts whether the patient has a disease or other disease (such as common pneumonia or eye disease, etc.) corresponding to the picture, obtains the picture-level classification result, and then enters step T4 ;
  • a disease or other disease such as common pneumonia or eye disease, etc.
  • Step T4 the classification unit 3 sets the most urgent disease in the disease category as the highest priority disease class, and when the number of the highest priority disease class in the picture-level classification result is greater than or equal to 1, it is determined that the patient suffers from the disease. The possibility of the disease is high, the patient-level prediction result is obtained, and then the step T5 is entered;
  • step T5 the output display unit 5 displays the picture-level classification result and the patient-level prediction result for the user to view, and then enters the end state.
  • the deep learning system for risk screening of disease patients based on ocular surface characteristics involved in this embodiment, because the ocular surface of disease patients often has conjunctivitis-like manifestations, including conjunctival congestion, stasis removal, discharge or increased secretion, etc.
  • the basic features extracted by the eye image feature extraction model are used to predict the disease category at the picture level and the patient level, and the learning and expression of features for the face and eye area , by capturing more distinguishing and discriminating features, the risk screening of disease patients based on eye features can be realized; and the present invention can carry out disease patients by taking a face picture and according to the eye area picture in the face picture.
  • the screening work can improve the speed, accuracy and convenience of disease risk screening, and at the same time, it can get rid of restrictions such as professional reliance, and can be popularized on a large scale.
  • quantitative detection can be realized anytime and anywhere, and virus infection can be dynamically monitored. To achieve efficient epidemic prevention and control.
  • the image of the eye region is obtained by preprocessing and screening the face image first, and then inputting the eye image feature extraction model, it can be ensured that the input pictures are all the images of the eye region with accurate positioning, and the face image is effectively removed.
  • the noise and irrelevant information in the eye image ensure that the eye image feature extraction model can correctly process the eye region picture.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • General Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Public Health (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Pathology (AREA)
  • Computational Linguistics (AREA)
  • Databases & Information Systems (AREA)
  • Mathematical Physics (AREA)
  • Multimedia (AREA)
  • Epidemiology (AREA)
  • Veterinary Medicine (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Primary Health Care (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Signal Processing (AREA)
  • Physiology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Psychiatry (AREA)

Abstract

一种基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习系统，属于医疗影像和计算机图像识别交叉领域，用于通过拍摄得到的人脸图片中的眼部区域来进行患者的新冠肺炎(COVID-19)疾病风险筛查，包括：人脸图像预处理部（1），用于对人脸图片进行预处理并得到眼部区域图片；眼部图像特征提取部（2），用于通过训练后的眼部图像特征提取模型提取眼部区域图片的基础特征；图片级和病患级分类模块部（3），用于根据基础特征进行图片级的患病类别预测并得到图片级分类结果，以及用于根据图片级分类结果进行病患级分类得到新冠肺炎(COVID-19)病患级预测结果。

Description

一种基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习系统 技术领域
本发明属于医疗影像和计算机图像识别交叉领域，具体涉及一种基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习系统。
背景技术
在过去的几十年里，基于深度学习(DL)的人工智能技术在各种计算机视觉任务如目标检测、图像分类、实例分割和物体识别等方面取得了显著的进展。
近年来，深度学习的优势使其在医学图像分析中也得到了广泛的应用，例如，对不同疾病进行分类，自闭症谱系障碍或大脑中的阿尔茨海默病，乳腺癌，糖尿病性视网膜病变和青光眼，以及肺癌或肺炎等常见病症。
此外，已有一些工作采用深度学习技术学习并提取CT影像特征进行疾病患者的识别与筛查，并取得良好效果。但是现有的疾病患者筛查技术中存在时效性差、设备要求高、依赖专业人员等不足，CT影像的拍摄需要使用专业的CT设备进行拍摄并需要依赖专业人员进行操作，同时由于进行拍摄并成像的耗时较长，所以无法快速对CT影像特征进行提取并完成疾病患者的识别与筛查。
发明内容
本发明是为了解决上述问题而进行的，目的在于提供一种基于眼表特征的疾病患者风险筛查深度学习系统。
本发明提供了一种基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习系统，用于通过拍摄得到的人脸图片中的眼部区域来进行患者的疾病风险筛查，具有这样的特征，包括：人脸图像预处理部，用于对人脸图片进行预处理并得到眼部区域图片；眼部图像特征提取部，用于通过训练后的眼部图像特征提取模型提取眼部区域图片的基础特征；分类部，用于根据基础特征进行图片级的新冠肺炎(COVID-19)患病类别预测并得到图片级分类结果，以及用于根据图片级分类结果进行病患级分类得到病患级预测结果，其中，眼部图像特征提取模型训练时包括以下步骤：
步骤S1，将眼部区域图片输入眼部图像特征提取模型中提取基础特征，分类部根据基础特征预测眼部区域图片中对应患者是否罹患新冠肺炎(COVID-19)并输出图片级分类结果；步骤S2，通过构建损失函数，根据图片级分类结果和眼部区域图片对应患者的真实患病类别，对眼部图像特征提取模型提取的基础特征进行监督；步骤S3，采用SGD优化器对眼部图像特征提取模型进行迭代训练。
在本发明提供的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习系统中，还可以具有这样的特征：其中，人脸图像预处理部中还具有人脸检测模型，人脸图像预处理部对人脸图片的预处 理包括以下步骤：步骤E1，通过人脸检测模型获取人脸图片中人脸的位置区域以及面部关键点坐标；步骤E2，根据面部关键点坐标中的人脸左右眼关键点，得到人脸眼部区域的横纵坐标的最大值和最小值，同时对横纵坐标进行一定数值的向外扩充来确保人脸眼部区域全部纳入特征提取范围内，并对人脸眼部区域进行截取，得到眼部截取图片；步骤E3，对眼部截取图片进行筛选，得到眼部区域图片。
在本发明提供的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习系统中，还可以具有这样的特征：其中，对眼部截取图片进行筛选时，将眼部截取图片中纵向长度长于横向长度的图片进行剔除后得到眼部区域图片。
在本发明提供的基于眼表特征的疾病患者风险筛查深度学习系统中，还可以具有这样的特征：其中，分类部进行病患级分类时，将患病类别中最紧急的疾病设为最高优先级别患病类别，对于一位患者的多张眼部区域图片中，当预测得到的图片级分类结果中的最高优先级别患病类别的个数大于等于1时，判断患者患有该疾病的可能性较大，得到病患级预测结果。
发明的作用与效果
根据本发明所涉及的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习系统，因为能够根据疾病患者眼表往往有着类结膜炎的表现，包括结膜充血、化瘀、溢液或分泌物增多等特征，借助深度学习网络的特征提取和分类预测能力，通过眼部图像特征提取模 型提取的基础特征来进行图片级以及病患级的患病类别预测，针对人脸眼部区域进行特征的学习表达，通过捕获更具分辨力和识别力的特征，能够实现基于眼部特征的疾病患者风险筛查；并且本发明通过拍摄人脸图片并根据人脸图片中的眼部区域图片就能够进行疾病患者的筛查工作，能够提高疾病风险筛查的快捷性、准确性以及方便性，同时能够摆脱专业人员依赖等限制，可大规模普及，在疫情阶段能够实现随时随地的定量检测，动态监测病毒感染的程度，观察治疗效果，进行疫情跟踪和疫情地图绘制，从而实现高效的疫情防控。
附图说明
图1是本发明的实施例中的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习系统的结构框图；
图2是本发明的实施例中的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习系统的系统图；
图3是本发明的实施例中的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习系统进行人脸图片预处理的流程图；
图4是本发明的实施例中的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习系统中训练眼部图像特征提取模型的流程图；
图5是本发明的实施例中的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习系统的疾病筛查流程示意图；
图6是本发明的实施例中的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习系统的疾病筛查流程框图。
具体实施方式
为了使本发明实现的技术手段与功效易于明白了解，以下结合实施例及附图对本发明作具体阐述。
<实施例>
图1是本发明的实施例中的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习系统的结构框图。
如图1所示，基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习系统100，用于通过拍摄得到的人脸图片中的眼部区域来进行患者的疾病风险筛查，包括人脸图像预处理部1、眼部图像特征提取部2、分类部3、疾病风险筛查评估部4、输出显示部5、系统通信部6以及用于控制上述各部的系统控制部7。
本实施例中，如图2所示，基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习系统100由计算机设备110和结果展示设备140组成，用于对外部的媒体数据进行处理。其中，计算机设备由处理器120以及内存130构成：处理器120是一个用于计算以及运行可执行代码的硬件处理器，如中央处理器CPU或图形计算处理器GPU；内存130是一个非易失的存储设备，用于储存可执行代码从而让处理器120执行相应的计算过程。同时，内存130也会存储各类中间数据及参数。内存130存储内容包括存储有人脸图片的人脸数据集及其相关参数、可执行代码，可执行代码用于运行人脸图像预处理部1、眼部图像特征提取部2以及分类部3。
本实施例中，媒体数据通过各类数据采集设备拍摄得到，如智能手机等，媒体数据可以是视频内容，也可以是图像内容。媒体数据还可以是包含多个不同身份以及患病类型的人脸图像数据，人脸图片从媒体数据中得到。
人脸图像预处理部1用于对人脸图片进行预处理并得到眼部区域图片。其中，人脸图像预处理部1具有一个人脸检测模型，人脸图像预处理部1基于人脸检测模型对人脸图片进行预处理，如图3所示，该预处理包括以下步骤:
步骤S1，通过人脸检测模型获取人脸图片中人脸的位置区域以及面部关键点坐标。
本实施例中，由于数据采集的灵活性，原始的人脸图片通常不仅包含眼部区域，还包括背景或面部的其他区域，如鼻子、耳朵和嘴巴。如果直接使用原始的人脸图片进行分类，必然会引入噪声或者不相关的信息，比如背景噪声或鼻子的特征，使得眼部图像特征提取模型的性能不准确、不可靠。因此，为了使眼部图像特征提取模型聚焦于提取眼表特征，本实施例采用人脸检测模型获取图片中人脸的位置区域以及面部关键点坐标，该面部关键点坐标中包括人脸左右眼以及面部其他器官的位置坐标。
步骤S2，根据面部关键点坐标中的人脸左右眼关键点，得到人脸眼部区域的横纵坐标的最大值和最小值，同时对横纵坐标进行一定数值的向外扩充来确保人脸眼部区域全部纳入特征提取范围内，并对人脸眼部区域进行截取，得到眼部截取图片。
步骤S3，对眼部截取图片进行筛选，得到眼部区域图片。
对眼部截取图片进行筛选时，将眼部截取图片中纵向长度长于横向长度的图片进行剔除后得到眼部区域图片。
本实施例中，考虑到因存在人脸角度、背景噪声等影像，人脸检测模型得到的面部关键点坐标并非完全准确无误，因而人脸眼部区域的截取也随之会有相应的偏差，考虑到人脸眼部区域应为横向较长的长方形区域，因而在横纵长度比上凡纵向长度长于横向的均认为是人脸眼部区域定位失败的结果，因此需将此类眼部截取图片剔除。
眼部图像特征提取部2用于通过训练后的眼部图像特征提取模型提取眼部区域图片的基础特征。
图4是本发明的实施例中的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习系统中训练眼部图像特征提取模型的流程图。
如图4所示，眼部图像特征提取模型训练时包括以下步骤：
步骤E1，将眼部区域图片输入眼部图像特征提取模型中提取基础特征，分类部根据基础特征预测眼部区域图片中对应患者的新冠肺炎(COVID-19)患病类别并输出图片级分类结果。
步骤E2，通过构建损失函数，根据图片级分类结果和眼部区域图片对应患者的真实患病类别，对眼部图像特征提取模型提取的基础特征进行监督。
步骤E3，采用SGD优化器对眼部图像特征提取模型进行迭代训练。
分类部3用于根据基础特征进行图片级的新冠肺炎(COVID-19)患病类别预测并得到图片级分类结果，以及用于根据图片级分类结果进行病患级分类得到病患级预测结果。
本实施例中，图片级的患病类别预测为判断眼部区域图片对应的患者是否患有疾病或其他疾病(例如，普通肺炎或者眼部疾病等)。
当分类部3进行病患级分类时，将患病类别中最紧急的疾病设为最高优先级别患病类别，对于一位患者的多张眼部区域图片中，当预测得到的图片级分类结果中的最高优先级别患病类别的个数大于等于1时，判断患者患有该疾病的可能性较大，得到病患级预测结果。
本实施例中，疾病风险筛查评估部4用于根据图片级分类结果和病患级预测结果进行患病风险评估。
结果展示设备包括输出显示部5，用于播放媒体数据和显示图片级分类结果以及病患级预测结果，可以是电脑、电视或者移动设备。
图5是本发明的实施例中的基于眼表特征的疾病患者风险筛查深度学习系统的疾病筛查流程示意图，图6是本发明的实施例中的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习系统的疾病筛查流程框图。
如图5和图6所示，本实施例的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习系统的疾病筛查流程包括以下步骤：
步骤T1，人脸图像预处理部1对拍摄得到的人脸图片进行预处理，得到眼部区域图片，然后进入步骤T2；
步骤T2，眼部图像特征提取部2根据眼部区域图片提取眼部区域图片的基础特征，然后进入步骤T3；
步骤T3，分类部3根据基础特征进行图片级的患病类别预测，预测图片对应患者是否患有疾病或其他疾病(例如普通肺炎或者眼部疾病等)，得到图片级分类结果，然后进入步骤T4；
步骤T4，分类部3将患病类别中最紧急的疾病设为最高优先级别患病类别，当图片级分类结果中的最高优先级别患病类别的个数大于等于1时，判断患者患有该疾病的可能性较大，得到病患级预测结果，然后进入步骤T5；
步骤T5，输出显示部5显示图片级分类结果以及病患级预测结果让用户查看，然后进入结束状态。
实施例的作用与效果
根据本实施例所涉及的基于眼表特征的疾病患者风险筛查深度学习系统，因为能够根据疾病患者眼表往往有着类结膜炎的表现，包括结膜充血、化瘀、溢液或分泌物增多等特征，借助深度学习网络的特征提取和分类预测能力，通过眼部图像特征提取模型提取的基础特征来进行图片级以及病患级的患病类别预测，针对人脸眼部区域进行特征的学习表达，通过捕获更具分辨力和识别力的特征，能够实现基于眼部特征的疾病患者风险筛查；并且本发明通过拍摄人脸图片并根据人脸图片中的眼部区域图片就能够进行疾病患者的筛查工作，能够提高疾病风险筛查的快捷性、准确性以及方便性，同时能够摆脱专业 人员依赖等限制，可大规模普及，在疫情阶段能够实现随时随地的定量检测，动态监测病毒感染的程度，观察治疗效果，进行疫情跟踪和疫情地图绘制，从而实现高效的疫情防控。
进一步地，因为先通过对人脸图片进行预处理并筛选得到眼部区域图片再输入眼部图像特征提取模型，能够确保输入的图片均为定位准确的眼部区域图片，有效去除了人脸图片中的噪声和不相关的信息，保证眼部图像特征提取模型能够正确的处理眼部区域图片。
上述实施方式为本发明的优选案例，并不用来限制本发明的保护范围。

Claims (4)

  1. 一种基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习系统，用于通过拍摄得到的人脸图片中的眼部区域来进行患者的疾病风险筛查，其特征在于，包括：
    人脸图像预处理部，用于对所述人脸图片进行预处理并得到眼部区域图片；
    眼部图像特征提取部，用于通过训练后的眼部图像特征提取模型提取所述眼部区域图片的基础特征；
    分类部，用于根据所述基础特征进行图片级的新冠肺炎(COVID-19)患病类别预测并得到图片级分类结果，以及用于根据所述图片级分类结果进行病患级分类得到新冠肺炎(COVID-19)病患级预测结果，
    其中，所述眼部图像特征提取模型训练时包括以下步骤：
    步骤S1，将所述眼部区域图片输入所述眼部图像特征提取模型中提取所述基础特征，所述分类部根据所述基础特征预测所述眼部区域图片中对应患者的新冠肺炎(COVID-19)患病类别并输出所述图片级分类结果；
    步骤S2，通过构建损失函数，根据所述图片级分类结果和所述眼部区域图片对应患者的真实新冠肺炎(COVID-19)患病类别，对所述眼部图像特征提取模型提取的所述基础特征进行监督；
    步骤S3，采用SGD优化器对所述眼部图像特征提取模型进行迭代训练。
  2. 根据权利要求1所述的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习系统，其特征在于：
    其中，所述人脸图像预处理部中还具有人脸检测模型，所述人脸图像预处理部对所述人脸图片的预处理包括以下步骤：
    步骤E1，通过所述人脸检测模型获取所述人脸图片中人脸的位置区域以及面部关键点坐标；
    步骤E2，根据所述面部关键点坐标中的人脸左右眼关键点，得到人脸眼部区域的横纵坐标的最大值和最小值，同时对横纵坐标进行一定数值的向外扩充来确保所述人脸眼部区域全部纳入特征提取范围内，并对所述人脸眼部区域进行截取，得到眼部截取图片；
    步骤E3，对所述眼部截取图片进行筛选，得到所述眼部区域图片。
  3. 根据权利要求2所述的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习系统，其特征在于：
    其中，对所述眼部截取图片进行筛选时，将所述眼部截取图片中纵向长度长于横向长度的图片进行剔除后得到所述眼部区域图片。
  4. 根据权利要求1所述的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习系统，其特征在于：
    其中，所述分类部进行病患级分类时，将所述患病类别中最紧急的疾病设为最高优先级别患病类别，对于一位患者的多张所述眼部区 域图片中，当预测得到的所述图片级分类结果中的所述最高优先级别患病类别的个数大于等于1时，判断患者患有该疾病的可能性较大，得到所述病患级预测结果。
PCT/CN2020/119241 2020-08-25 2020-09-30 一种基于眼表特征的新冠肺炎（covid-19）疾病患者风险筛查深度学习系统 Ceased WO2022041396A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010862461.5 2020-08-25
CN202010862461.5A CN114187223A (zh) 2020-08-25 2020-08-25 基于眼表特征的新冠肺炎疾病患者风险筛查深度学习系统

Publications (1)

Publication Number Publication Date
WO2022041396A1 true WO2022041396A1 (zh) 2022-03-03

Family

ID=80354467

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/119241 Ceased WO2022041396A1 (zh) 2020-08-25 2020-09-30 一种基于眼表特征的新冠肺炎（covid-19）疾病患者风险筛查深度学习系统

Country Status (2)

Country Link
CN (1) CN114187223A (zh)
WO (1) WO2022041396A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220164568A1 (en) * 2020-11-20 2022-05-26 Xiamoi Technology (Wuhan) Co., Ltd. Method for behaviour recognition, electronic equipment, and storage medium
CN116705325A (zh) * 2023-06-26 2023-09-05 国家康复辅具研究中心 一种伤口感染风险评估方法及其系统
US11816924B2 (en) 2020-11-20 2023-11-14 Xiaomi Technology (Wuhan) Co., Ltd. Method for behaviour recognition based on line-of-sight estimation, electronic equipment, and storage medium
CN117877103A (zh) * 2024-03-13 2024-04-12 宁波市眼科医院 一种基于深度元学习的角膜炎智能筛查方法

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115804571A (zh) * 2022-12-19 2023-03-17 上海昱极细胞生物技术有限公司 移动终端健康数据动态监测方法、系统、设备及介质

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106530295A (zh) * 2016-11-07 2017-03-22 首都医科大学 一种视网膜病变的眼底图像分类方法和装置
CN110111316A (zh) * 2019-04-26 2019-08-09 广东工业大学 基于眼部图像识别弱视的方法及系统

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108230296B (zh) * 2017-11-30 2023-04-07 腾讯科技（深圳）有限公司 图像特征的识别方法和装置、存储介质、电子装置

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106530295A (zh) * 2016-11-07 2017-03-22 首都医科大学 一种视网膜病变的眼底图像分类方法和装置
CN110111316A (zh) * 2019-04-26 2019-08-09 广东工业大学 基于眼部图像识别弱视的方法及系统

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220164568A1 (en) * 2020-11-20 2022-05-26 Xiamoi Technology (Wuhan) Co., Ltd. Method for behaviour recognition, electronic equipment, and storage medium
US11790692B2 (en) * 2020-11-20 2023-10-17 Xiaomi Technology (Wuhan) Co., Ltd. Method for behaviour recognition, electronic equipment, and storage medium
US11816924B2 (en) 2020-11-20 2023-11-14 Xiaomi Technology (Wuhan) Co., Ltd. Method for behaviour recognition based on line-of-sight estimation, electronic equipment, and storage medium
CN116705325A (zh) * 2023-06-26 2023-09-05 国家康复辅具研究中心 一种伤口感染风险评估方法及其系统
CN116705325B (zh) * 2023-06-26 2024-01-19 国家康复辅具研究中心 一种伤口感染风险评估方法及其系统
CN117877103A (zh) * 2024-03-13 2024-04-12 宁波市眼科医院 一种基于深度元学习的角膜炎智能筛查方法
CN117877103B (zh) * 2024-03-13 2024-05-24 宁波市眼科医院 一种基于深度元学习的角膜炎智能筛查方法

Also Published As

Publication number Publication date
CN114187223A (zh) 2022-03-15

Similar Documents

Publication Publication Date Title
WO2022041396A1 (zh) 一种基于眼表特征的新冠肺炎（covid-19）疾病患者风险筛查深度学习系统
CN112287868B (zh) 一种人体动作识别方法及装置
CN108985210A (zh) 一种基于人眼几何特征的视线追踪方法及系统
WO2020215552A1 (zh) 多目标跟踪方法、装置、计算机设备及存储介质
CN107506770A (zh) 糖尿病视网膜病变眼底照相标准图像生成方法
WO2020258978A1 (zh) 对象检测方法和装置
WO2020042542A1 (zh) 眼动控制校准数据获取方法和装置
WO2015070764A1 (zh) 一种人脸定位的方法与装置
WO2021259033A1 (zh) 人脸识别方法、电子设备以及存储介质
WO2020042541A1 (zh) 眼球追踪交互方法和装置
CN109993021A (zh) 人脸正脸检测方法、装置及电子设备
CN116994170A (zh) 在视频会议跟踪和检测期间减少假阳性识别
WO2022252107A1 (zh) 一种基于眼部图像的疾病检测系统及方法
CN115496700A (zh) 一种基于眼部图像的疾病检测系统及方法
CN114360039A (zh) 一种智能眼睑检测方法及系统
CN109241898B (zh) 腔镜视像的目标定位方法和系统、存储介质
CN109214324A (zh) 基于多相机阵列的最正脸图像输出方法及输出系统
WO2022213349A1 (zh) 戴口罩人脸识别方法、装置、计算机存储介质
CN110188630A (zh) 一种人脸识别方法和相机
WO2022213348A1 (zh) 检测口罩人脸识别方法、装置及计算机存储介质
CN113792629B (zh) 一种基于深度神经网络的安全帽佩戴检测方法及系统
CN114680822A (zh) 一种基于眼部图像的快速筛查系统
CN114078271A (zh) 阈值确定方法、目标人员识别方法、装置、设备及介质
CN114821731A (zh) 一种基于国产cpu和os的动态人脸识别优化方法
CN112395922B (zh) 面部动作检测方法、装置及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20951064

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 090823)

122 Ep: pct application non-entry in european phase

Ref document number: 20951064

Country of ref document: EP

Kind code of ref document: A1