CN111938567A - Ophthalmic parameter measurement method, system and equipment based on deep learning - Google Patents

Abstract

The invention provides a method and system for measuring human eye parameters based on deep learning, including: acquiring a face picture, and extracting left and right eye images in the picture; using a deep neural network to identify different positions in the left and right eye images, including cornea, Sclera and medial and lateral canthal positions; calculates multiple ocular parameters at different positions in the left and right eye images. At the same time, a device implemented based on the above-mentioned method and system for measuring human eye parameters is provided. The method, system and device for measuring human eye parameters based on deep learning provided by the present invention can realize automatic identification and positioning of different parts of the human eye; realize automatic measurement of eye parameters that ophthalmologists often need to measure; The analysis of the external disease situation provides parameter assistance and support.

Description

Ophthalmic parameter measurement method, system and equipment based on deep learning

technical field

The present invention relates to a medical image processing technology in the field of artificial intelligence technology, and in particular, to a method, system and device for measuring ophthalmology parameters based on deep learning.

Background technique

At present, my country is facing an imbalance in the supply and demand of high-quality medical resources, with a long training cycle for doctors, a high rate of misdiagnosis, rapid changes in disease spectrum, rapid technological changes, an aging population, and the growth of chronic diseases. As people pay more attention to health, the large demand has spawned the rapid development of medical AI.

So far, AI has made great progress in more segments of my country's medical field. For example, in October 2016, Baidu released "Baidu Medical Brain", which benchmarked similar products from Google and IBM. The specific application of Baidu Medical Brain in the medical field, it collects and analyzes a large number of medical professional literature and medical data, and gives final recommendations for diagnosis and treatment based on user symptoms by simulating the consultation process.

In July 2017, Ali Health released the medical AI system "Doctor You", including a clinical medical research and diagnosis platform, a medical auxiliary detection engine, etc. In addition, AliHealth has also cooperated with the government, hospitals, scientific research institutions and other external institutions to develop intelligent diagnosis engines for 20 common and frequently-occurring diseases, including diabetes, lung cancer prediction, and fundus screening.

In November 2018, the "Digital Diagnosis and Treatment Equipment Research and Development Project" led by Tencent was launched. As one of the first six pilot projects launched in the national key research and development plan, the project is based on "AI+CDSS" (Artificial Intelligence Clinical Assistant Decision Support). technology) to explore and facilitate the upgrading of medical services.

There are many integration points between AI and the medical field. The application of AI in the medical field is summarized and analyzed. Currently, it is mainly used in five fields, namely: medical imaging, auxiliary diagnosis, drug research and development, health management, and disease prediction.

With the development advantages of medical imaging big data and image recognition technology, medical imaging has become one of the most mature fields in China for the combination of artificial intelligence and medical care, which can be applied to medical fields such as tuberculosis, fundus, breast cancer, and cervical cancer. During an eye exam, physicians often require accurate measurements of a patient's eyes for diagnosis and disease status determination. They usually manually measure many ocular parameters, such as the height of the eye cleft, the longitudinal and transverse diameters of the cornea, the distance between the inner and outer canals, and the distance between the pupils. Currently, a common method is for doctors to measure these parameters directly on the patient's eye using a ruler. This is very troublesome for doctors, and the results are unconvincing, at the same time, physical contact puts doctors and patients at risk of infection. In order to solve this problem, those skilled in the art try to automatically calculate these parameters of the eye according to the eye image through a specific algorithm, which can save the ophthalmologist from the time-consuming and laborious measure of the patient's eye with a ruler, and the patient does not have to pose strange postures to facilitate the doctor and Take the risk of being infected. In recent years, with the rapid development of deep learning, CNN-based image object detection has achieved high accuracy, and more and more researchers have applied it to medical image analysis, but the technology used in eye disease measurement and Research is still limited.

At present, there is no description or report of the technology similar to the present invention, and no similar materials at home and abroad have been collected.

SUMMARY OF THE INVENTION

In view of the above deficiencies in the prior art, the purpose of the present invention is to provide a method, system and device for measuring human eye parameters based on deep learning.

The present invention is achieved through the following technical solutions.

According to one aspect of the present invention, there is provided a method for measuring human eye parameters based on deep learning, including:

The acquired face image, and extract the left and right eye images in the face image;

Using a deep neural network, identify different parts in the left and right eye images, including the cornea, sclera and the position of the inner and outer canthus, and get the recognition results;

According to the recognition results, multiple eye parameters in different positions and different states in the left and right eye images are calculated.

Preferably, the extracting the left and right eye images in the face picture includes:

Use open source engineering to process face images and extract multiple eye key points in the face image, and extract the complete left and right eye images according to the positions of the eye key points.

Preferably, the deep neural network adopts a multi-task convolutional neural network, including a corneal block, a sclera block and an inner and outer canthal block; wherein:

The corneal block adopts pixel-based prediction, and for each pixel position, a bounding box and a confidence level are predicted, and the confidence level reflects the probability of whether the pixel is located in the cornea. After non-maximum suppression, the weighted average of all bounding boxes is calculated as the final prediction result;

The sclera block adopts a fully convolutional network, where the input feature map is upsampled by a factor of m to obtain a classification score map;

The inner and outer canthal blocks adopt the design structure of the last three layers of the YOLO (You Only Look Once) target detection neural network, divide the image into n×n grids, and predict the grids where the inner and outer canthals fall and relative to the center of the grid. The coordinate offset of , and then directly predict the coordinates of the key points.

Preferably, the ocular parameters are 10, including: palpebral fissure height, corneal longitudinal diameter, corneal transverse diameter, corneal coverage, upper eyelid retraction, lower eyelid retraction, pupil diameter, limited amount of downward rotation, and upward rotation Limited and outbound restrictions.

Preferably, the eye parameters are calculated by using a quadratic function to fit the eyelid, and combining the method of calculating the scale of the physical world and the image size with the sticker.

Preferably, the method for fitting the eyelid by the quadratic function comprises:

After obtaining the identification result of the sclera, use the quadratic function to fit the edge of the sclera to obtain the position of the eyelid, and set the center of the cornea as the vertex of the quadratic curve. The expression of the quadratic curve is:

p(y-cy)=±(x-cx) ² (1)

where (cx, cy) is the center of the cornea, and (x, y) is the position coordinate of the pixel in the image. When a positive sign is used on the right side of the equation, the equation fits the lower eyelid, and if a negative sign is used, the equation fits the upper eyelid. The value of p is determined by:

p=d ₁ +d ₂ (2)

Among them, d ₁ and d ₂ respectively represent the pixel distance of the sclera width and the maximum opening of the sclera that cannot be connected due to the position of the cornea or the pixel distance of the corneal diameter; among them, for the sclera recognition result of the upper and lower eyelids unilaterally blocking the cornea, d ₁ and d ₂ are taken according to the pixel distance of the sclera width and the pixel distance of the maximum opening of the sclera that cannot be connected due to the position of the cornea; for the sclera recognition result in which the upper and lower eyelids jointly block the cornea, d ₂ is based on the pixel distance of the upper and lower opening of the sclera. For the sclera recognition result in which the upper and lower eyelids do not block the cornea, d ₂ takes the value according to the pixel distance of the corneal diameter.

Preferably, the method of calculating the scale of the physical world and the image size using stickers includes:

Apply two identical standard circular stickers on the patient's forehead and identify this sticker to obtain the scale S _scale between the picture size and the real world physical size, specifically:

First, use the Hough transform to identify the two stickers in the picture, and extract the diameters of the two stickers. Assuming that the diameters of the two stickers obtained from the picture are D _A and D _B , respectively, the average diameter of the two stickers is recorded as Make D _image :

The physical size of the sticker in the real world is D _real , then the scale S _scale between the picture size and the physical size in the real world is:

Preferably, the method for calculating the eye parameters is:

Eyelid fissure height: After fitting the edge of the eyelid, calculate the distance between the highest point and the lowest point of the eyelid;

Corneal longitudinal diameter and corneal transverse diameter: directly calculated according to the recognition result;

Corneal coverage: After fitting the edge of the eyelid, calculate the distance between the highest point of the eyelid and the highest point of the cornea;

Upper eyelid retraction: After fitting the eyelid edge, calculate the distance between the highest point of the eyelid and the highest point of the cornea;

The amount of retraction under the eyelid: after fitting the edge of the eyelid, calculate the distance between the lowest point of the eyelid and the lowest point of the cornea;

Pupil diameter: After the cornea is identified, it is directly calculated after binarizing the cornea;

Downward rotation limitation: Calculate the distance between the line connecting the inner and outer canthus and the lowest point of the cornea;

Upward rotation limit: Calculate the distance between the line connecting the inner and outer canthus and the lowest point of the cornea;

External rotation limit: Calculate the distance between the lateral canthus and the outermost point of the cornea.

According to a second aspect of the present invention, a deep learning-based human eye parameter measurement system is provided, including:

The eye image extraction module extracts the left and right eye images in the face image according to the acquired face image;

The eye part recognition module uses a deep neural network to recognize different parts in the left and right eye images, including the positions of the cornea, sclera, and inner and outer canthus, and obtain the recognition results;

The eye parameter calculation module calculates a plurality of eye parameters in different positions and different states in the left and right eye images according to the recognition results obtained by the eye part recognition module.

Preferably, the eye image extraction module adopts open source engineering to process the face image and extracts multiple key points of the eye in the face image, and extracts the complete left and right eye images according to the positions of the key points of the eye.

Preferably, in the eye part identification module, the deep neural network adopts a multi-task convolutional neural network, including a corneal block, a sclera block and an inner and outer canthal block; wherein:

The corneal block adopts pixel-based prediction, predicts a bounding box and a confidence level for each pixel position, and obtains a weighted average of all bounding boxes as the final prediction result after non-maximum suppression;

The sclera block adopts a fully convolutional network, where the input feature map is upsampled by a factor of m to obtain a classification score map;

The inner and outer canthus block adopts the design structure of the last three layers of the YOLO target detection neural network, divides the image into n × n grids, predicts the grid where the inner and outer canthus falls and the coordinate offset relative to the center of the grid, and then Directly predict the coordinates of key points.

According to a third aspect of the present invention, there is provided a device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor can be used to execute the above-mentioned computer program when the processor executes the computer program The method of any one.

Due to the adoption of the above-mentioned technical solutions, the present invention has at least one of the following beneficial effects:

The method, system and device for measuring human eye parameters based on deep learning provided by the present invention can realize automatic identification and positioning of different parts of the human eye.

The method, system and device for measuring human eye parameters based on deep learning provided by the present invention realize automatic measurement of eye parameters that ophthalmologists often need to measure.

The method, system and device for measuring human eye parameters based on deep learning provided by the present invention provide parameter assistance and support for ophthalmologists to realize the analysis of eye diseases.

Description of drawings

Other features, objects and advantages of the present invention will become more apparent by reading the detailed description of non-limiting embodiments with reference to the following drawings:

FIG. 1 is a description of different parts of a human eye and an example of a picture of the human eye according to an embodiment of the present invention.

FIG. 2 is a schematic structural diagram of a multi-task neural network in an embodiment of the present invention.

FIG. 3 is a standard sticker in an embodiment of the present invention.

FIG. 4 is a schematic diagram of a post-processing picture for calculating eye parameters in an embodiment of the present invention.

FIG. 5 is a flowchart of a method for measuring human eye parameters based on deep learning provided by an embodiment of the present invention.

Detailed ways

The embodiments of the present invention are described in detail below: This embodiment is implemented on the premise of the technical solution of the present invention, and provides detailed implementation modes and specific operation processes. It should be pointed out that for those skilled in the art, without departing from the concept of the present invention, several modifications and improvements can be made, which all belong to the protection scope of the present invention.

An embodiment of the present invention provides an ophthalmological parameter measurement method based on deep learning. The method uses deep learning technology and image processing algorithm to process human eye images, thereby identifying different parts of the human eye and completing the measurement of ophthalmic parameters . As shown in Figure 5, the method includes the following steps:

Step 1: Collect high-definition photos of the patient's face, and automatically extract the photos of the patient's left and right eyes;

As a preferred embodiment, the left and right eye photos are processed separately;

Step 2: Use the deep neural network to identify different parts of the eyes in the left and right eye photos, including the cornea, sclera, and inner and outer canthus (ie, canthus);

Step 3: Use the result processing algorithm and the eye parameter calculation method to calculate 10 common parameters of the eyes in the left and right eye photos. Among them, these parameters are the basis for the ophthalmologist to analyze the eye condition.

The eye parameters measured above can assist the ophthalmologist to analyze the eye condition.

As a preferred embodiment, in step 1, an existing open source project (such as the dlib open source library, which can extract 68 key points in the photo according to requirements) is used to process the human face and extract multiple facial key points. The points also include key parts such as the human eye, and then images of the human eye can be extracted based on these key points.

In some embodiments of the present invention, the pictures of the human eye to be identified are shown in Figure 1. In Figure 1, (a) is the description of different parts of the human eye, and (b) and (c) are pictures of several different eye diseases, respectively. Example.

As a preferred embodiment, step 2 includes the following sub-steps:

The cornea, sclera, and inner and outer canthus have different characteristics, so different methods need to be used to identify them separately. The shape of the sclera is irregular, and there are great differences between different patients, so it is more suitable for pixel-level semantic segmentation, which can more accurately identify the scope of the sclera. The shape of the cornea is more regular, roughly circular or elliptical, and can be positioned and bounded with a rectangular box, and then extracted with an ellipse. The medial and lateral canthus are two points located on both sides of the eye, so it is desirable to locate their coordinates directly. In addition, the medial and lateral canthus are located on both sides of the sclera and are adjacent to the sclera, which is a relatively easy to find constraint relationship. This is the basic idea of establishing a deep neural network in the embodiment of the present invention.

A multi-task convolutional neural network (CNN), as shown in Fig. 2, is used to identify the above-mentioned cornea, sclera, and inner and outer canthal parts, and the backbone of the multi-task convolutional neural network is based on VGG16. Figure 3 is the structure of the three blocks of CNN, including: Cornea (cornea) block, Sclera (sclera) block and Canthus (inner and outer canthus) block, where:

The corneal block adopts pixel-based prediction, and predicts a bounding box and confidence for each pixel position. After non-maximum suppression, the weighted average of all bounding boxes is calculated as the final prediction result;

The sclera block adopts a fully convolutional network, in which the input feature map is upsampled by m times to obtain a classification score map; as a preferred embodiment, m takes a value of 8;

The coordinates of the key points are directly predicted by the inner and outer canthal blocks, and the YOLO (You Only Look Once) target detection neural network is used (refer to Redmon, Joseph, Santosh Divvala, Ross Girshick, and Ali Farhadi. "You only look once: Unified, real -time object detection."In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.779-788.2016) The design structure of the last three layers divides the image into n×n grids, and predicts which grid the inner and outer canthus falls on and the coordinate offset relative to the center of the grid. In this way, all the key parts of the eye, sclera, cornea, and inner and outer canthus are obtained, and the position of the eyelid can be inferred; as a preferred embodiment, n is 7.

As a preferred embodiment, step 3 includes the following steps:

Select 10 parameters that ophthalmologists often need to measure, these parameters provide objective basis for doctors to analyze the eye condition; these 10 parameters include: palpebral fissure height (PFH), corneal longitudinal diameter, corneal transverse diameter, corneal coverage, sclera The amount of superior retraction, the amount of subscleral retraction, the pupil diameter, the limited amount of downward rotation, the amount of limited upward rotation, and the amount of limited external rotation.

In order to obtain the ratio between the picture size and the physical size in the real world, a circular sticker with a diameter of D _real = 10.0mm is placed on the patient's forehead, as shown in Figure 3, where (a) and (b) are two An example of a standard sticker. When placing the sticker, try to keep the sticker flat and parallel to the camera lens to minimize errors caused by tilt and wrinkles. Denote the average diameter of the two stickers as D _image pixels, then the scale S _scale between the image size and the physical world size is:

First, use the Hough transform to identify the two stickers in the picture, and extract the diameters of the two stickers. Assuming that the diameters of the two stickers obtained from the picture are D _A and D _B , respectively, the average diameter of the two stickers is recorded as Make D _image :

The physical size of the sticker in the real world is D _real , then the scale between the picture size and the physical size in the real world is denoted as S _scale , which can be calculated from formula (10):

A quadratic function is used to fit the edge of the sclera to obtain the position of the eyelid, as shown in Figure 4. In the figure, (a) is an example of several eye pictures, (b) is an illustration of the parameters for fitting the edge of the sclera, ( c) Schematic illustration of the method of fitting the scleral edge. Set the center of the cornea as the vertex of the quadratic curve, and the expression of the quadratic curve is:

p(y-cy)=±(x-cx) ² (5)

where (cx, cy) is the center of the cornea, and (x, y) is the position coordinate of the pixel in the image. When a positive sign is used on the right side of the equation, the equation fits the lower eyelid, and if a negative sign is used, the equation fits the upper eyelid. The value of p is determined by:

p=d ₁ +d ₂ (6)

where d ₁ and d ₂ are shown in Fig. 4(b), d ₁ and d ₂ respectively represent the pixel distance of the sclera width and the maximum opening of the sclera that cannot be connected due to the corneal position or the pixel distance of the corneal diameter; for In the first and second pictures of Fig. 4(b), the upper and lower eyelids unilaterally block the cornea. The sclera identification results, d ₁ and d ₂ are based on the pixel distance of the sclera width and the maximum opening of the sclera that cannot be connected due to the position of the cornea. The value of the pixel distance; for the sclera recognition result of the upper and lower eyelids covering the cornea in the third picture, d ₂ is taken according to the average value of the pixel distances of the sclera open up and down; for the fourth picture, the upper and lower eyelids do not block the cornea, which is the sclera In the recognition result, d ₂ takes the value according to the pixel distance of the corneal diameter.

As a preferred embodiment, the method for calculating eye parameters is:

Eyelid fissure height: After fitting the edge of the eyelid, calculate the distance between the highest point and the lowest point of the eyelid;

Corneal longitudinal diameter and corneal transverse diameter: directly calculated according to the recognition result;

Corneal coverage: After fitting the edge of the eyelid, calculate the distance between the highest point of the eyelid and the highest point of the cornea;

Upper eyelid retraction: After fitting the eyelid edge, calculate the distance between the highest point of the eyelid and the highest point of the cornea;

The amount of retraction under the eyelid: after fitting the edge of the eyelid, calculate the distance between the lowest point of the eyelid and the lowest point of the cornea;

Pupil diameter: After the cornea is identified, it is directly calculated after binarizing the cornea;

Downward rotation limitation: Calculate the distance between the line connecting the inner and outer canthus and the lowest point of the cornea;

Upward rotation limit: Calculate the distance between the line connecting the inner and outer canthus and the lowest point of the cornea;

External rotation limit: Calculate the distance between the lateral canthus and the outermost point of the cornea.

Another embodiment of the present invention provides a deep learning-based human eye parameter measurement system, including:

The eye image extraction module extracts the left and right eye images in the image according to the acquired face image;

The eye part recognition module adopts deep neural network to recognize different parts in the left and right eye images, including the cornea, sclera, and inner and outer canthal positions;

The eye parameter calculation module calculates a plurality of eye parameters in different positions and different states in the human eye image according to the recognition result.

As a preferred embodiment, the eye image extraction module uses an existing open source project (this patent jointly uses the Dlib open source library and the Haar detection algorithm of OpenCV, the dlib open source library is used to process the complete face image, and 68 key points can be extracted ; Haar detection algorithm can process part of facial pictures and locate human eyes) is used to process human faces and extract multiple facial key points, these facial key points include key parts such as human eyes, and then extract human eye images according to these key points.

As a preferred embodiment, in the eye part identification module, the deep neural network adopts a multi-task neural network, including the corneal block, the sclera block and the inner and outer canthal blocks, which can identify all key parts of the eye according to the characteristics of different eye parts, including the sclera. , cornea, inner and outer canthus. in:

Corneal blocks use pixel-based prediction, predicting a bounding box and confidence for each pixel location, which reflects the probability of whether the pixel is located in the cornea. After non-maximum suppression, the weighted average of all bounding boxes is calculated as the final prediction result;

The sclera block adopts a fully convolutional network, where the input feature map is upsampled by a factor of m to obtain a classification score map;

The inner and outer canthal block adopts the YOLO (You Only Look Once) target detection neural network design structure after three layers, divides the image into n×n grid, predicts the grid where the inner and outer canthus falls and the coordinates relative to the center of the grid offset, and then directly predict the coordinates of the key points.

As a preferred embodiment, the eye parameter calculation module first selects 10 parameters that ophthalmologists often need to measure, including palpebral fissure height (PFH), corneal longitudinal diameter, corneal transverse diameter, corneal coverage, and scleral retraction, Subscleral retraction, pupil diameter, limited downward rotation, limited upward rotation, and limited external rotation. As a preferred embodiment, the module can stick a standard circular sticker on the patient's forehead and automatically recognize the sticker to obtain the ratio between the size of the picture and the physical size of the real world; then use a quadratic function to fit the eyelid and calculate the eye parameter.

A third embodiment of the present invention provides a device including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor can be used to execute any item when executing the computer program A deep learning-based method for measuring human eye parameters.

Optionally, the memory is used to store the program; the memory may include volatile memory (English: volatile memory), such as random-access memory (English: random-access memory, abbreviation: RAM), such as static random-access memory ( English: static random-access memory, abbreviation: SRAM), double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic Random Access Memory, abbreviation: DDR SDRAM), etc.; memory can also include non-volatile Memory (English: non-volatile memory), such as flash memory (English: flash memory). The memory 62 is used to store computer programs (such as application programs, functional modules, etc. for implementing the above-mentioned methods), computer instructions, etc., and the above-mentioned computer programs, computer instructions, etc. may be stored in one or more memories in partitions. And the above-mentioned computer programs, computer instructions, data, etc. can be called by the processor.

The computer programs, computer instructions, etc. described above may be partitioned and stored in one or more memories. And the above-mentioned computer programs, computer instructions, data, etc. can be called by the processor.

The processor is configured to execute the computer program stored in the memory, so as to implement each step in the method involved in the above embodiments. For details, refer to the relevant descriptions in the foregoing method embodiments.

The processor and memory can be separate structures or integrated structures that are integrated together. When the processor and the memory are independent structures, the memory and the processor can be coupled and connected through a bus.

The method, system and device for measuring human eye parameters based on deep learning provided by the above-mentioned embodiments of the present invention extract the left and right eye images in the picture by acquiring a face picture; and use a deep neural network to identify different positions in the left and right eye images, including the cornea. , sclera and inner and outer canthal positions; calculate multiple eye parameters at different positions in the left and right eye images; can realize automatic identification and positioning of different parts of the human eye; realize automatic measurement of eye parameters that ophthalmologists often need to measure; for ophthalmology Doctors provide parameter assistance and support for the analysis of eye conditions.

It should be noted that the steps in the method provided by the present invention can be implemented by using corresponding modules, devices, units, etc. in the system, and those skilled in the art can refer to the technical solutions of the system to implement the steps of the method, that is, the steps in the system The embodiment may be understood as a preferred example of the implementation method, which will not be repeated here.

Those skilled in the art know that, in addition to implementing the system provided by the present invention and its respective devices in the form of pure computer-readable program codes, the system provided by the present invention and its respective devices can be made by logic gates, Switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers are used to achieve the same function. Therefore, the system and its various devices provided by the present invention can be regarded as a kind of hardware components, and the devices for realizing various functions included in the system can also be regarded as structures in the hardware components; The means for implementing various functions can be regarded as either a software module implementing a method or a structure within a hardware component.

Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above-mentioned specific embodiments, and those skilled in the art can make various variations or modifications within the scope of the claims, which do not affect the essential content of the present invention.

Claims (10)

1. a human eye parameter measurement method based on deep learning, is characterized in that, comprises: Obtain a face image, and extract the left and right eye images in the face image; Using a deep neural network, identify different parts in the left and right eye images, including the cornea, sclera and the position of the inner and outer canthus, and get the recognition results; According to the recognition results, multiple eye parameters in different positions and different states in the left and right eye images are calculated. 2. The method for measuring human eye parameters based on deep learning according to claim 1, wherein the extraction of left and right eye images in the human face picture comprises: Use open source engineering to process face images and extract multiple eye key points in the face image, and extract the complete left and right eye images according to the positions of the eye key points. 3. the human eye parameter measurement method based on deep learning according to claim 1, is characterized in that, described deep neural network, adopts multi-task convolutional neural network, comprises corneal block, sclera block and inner and outer canthal block; Wherein: The corneal block adopts pixel-based prediction, and predicts a bounding box and confidence at each pixel position, and the confidence reflects the probability of whether the pixel is located in the cornea. After non-maximum suppression, all bounding boxes are averaged as the final prediction result; The sclera block adopts a fully convolutional network, where the input feature map is upsampled by a factor of m to obtain a classification score map; The inner and outer canthus block adopts the design structure of the last three layers of the YOLO target detection neural network, divides the image into n × n grids, predicts the grid where the inner and outer canthus falls and the coordinate offset relative to the center of the grid, and then Directly predict the coordinates of key points. 4. The method for measuring human eye parameters based on deep learning according to claim 1, wherein the eye parameters are 10, including: palpebral fissure height, corneal longitudinal diameter, corneal transverse diameter, corneal covering, eyelid The amount of upper retraction, the amount of lower eyelid retraction, pupil diameter, the amount of downward rotation limitation, the amount of upward rotation limitation, and the amount of outer rotation limitation. 5 . The method for measuring human eye parameters based on deep learning according to claim 4 , wherein the eye parameters are calculated by using a quadratic function to fit the eyelid, and in combination with a method for calculating the scale of the physical world and image size in conjunction with stickers. 6 . 6. The method for measuring human eye parameters based on deep learning according to claim 5, wherein the method for fitting the eyelid by the quadratic function comprises: After obtaining the identification result of the sclera, use the quadratic function to fit the edge of the sclera to obtain the position of the eyelid, and set the center of the cornea as the vertex of the quadratic curve. The expression of the quadratic curve is: p(y-cy)=±(x-cx) ² (1) where (cx, cy) is the center of the cornea, and (x, y) is the position coordinate of the pixel in the image. When a positive sign is used on the right side of the equation, the equation fits the lower eyelid, and if a negative sign is used, the equation fits the upper eyelid. The value of p is determined by: p=d ₁ +d ₂ (2) Among them, d ₁ and d ₂ respectively represent the pixel distance of the sclera width and the maximum opening of the sclera that cannot be connected due to the position of the cornea or the pixel distance of the corneal diameter; among them, for the sclera recognition result of the upper and lower eyelids unilaterally blocking the cornea, d ₁ and d ₂ are taken according to the pixel distance of the sclera width and the pixel distance of the maximum opening of the sclera that cannot be connected due to the position of the cornea; for the sclera recognition result in which the upper and lower eyelids jointly block the cornea, d ₂ is based on the pixel distance of the upper and lower opening of the sclera. For the sclera recognition result in which the upper and lower eyelids do not block the cornea, d ₂ takes the value according to the pixel distance of the corneal diameter. Methods for calculating the scale of the physical world and image dimensions using stickers, including: Apply two identical standard circular stickers on the patient's forehead and identify this sticker to obtain the scale S _scale between the picture size and the real world physical size, specifically: First, use the Hough transform to identify the two stickers in the picture, and extract the diameters of the two stickers. Assuming that the diameters of the two stickers obtained from the picture are D _A and D _B , respectively, the average diameter of the two stickers is recorded as Make D _image :

The physical size of the sticker in the real world is D _real , then the scale S _scale between the picture size and the physical size in the real world is:

The method to calculate the eye parameters is: Eyelid fissure height: After fitting the edge of the eyelid, calculate the distance between the highest point and the lowest point of the eyelid; Corneal longitudinal diameter and corneal transverse diameter: directly calculated according to the recognition result; Corneal coverage: After fitting the edge of the eyelid, calculate the distance between the highest point of the eyelid and the highest point of the cornea; Upper eyelid retraction: After fitting the eyelid edge, calculate the distance between the highest point of the eyelid and the highest point of the cornea; The amount of retraction under the eyelid: after fitting the edge of the eyelid, calculate the distance between the lowest point of the eyelid and the lowest point of the cornea; Pupil diameter: After the cornea is identified, it is directly calculated after binarizing the cornea; Downward rotation limitation: Calculate the distance between the line connecting the inner and outer canthus and the lowest point of the cornea; Upward rotation limit: Calculate the distance between the line connecting the inner and outer canthus and the lowest point of the cornea; External rotation limit: Calculate the distance between the lateral canthus and the outermost point of the cornea. 7. A human eye parameter measurement system based on deep learning, characterized in that, comprising: The eye image extraction module extracts the left and right eye images in the face image according to the acquired face image; The eye part identification module adopts deep neural network to identify different parts in the left and right eye images, including the positions of the cornea, sclera, and inner and outer canthus, and obtain the identification results; The eye parameter calculation module calculates a plurality of eye parameters in different positions and different states in the left and right eye images according to the recognition results obtained by the eye part recognition module. 8. The human eye parameter measurement system based on deep learning according to claim 7, wherein the eye image extraction module adopts open source engineering to process the human face image and extract a plurality of eye key points in the human face image , and extract the complete left and right eye images according to the positions of the eye key points. 9. The human eye parameter measurement system based on deep learning according to claim 7, characterized in that, in the eye part identification module, the deep neural network adopts a multi-task convolutional neural network, comprising a corneal block, a sclera block and a Inner and outer canthal blocks; of which: The corneal block adopts pixel-based prediction, and for each pixel position, a bounding box and a confidence level are predicted, and the confidence level reflects the probability of whether the pixel is located in the cornea. After non-maximum suppression, the weighted average of all bounding boxes is calculated as the final prediction result; The sclera block adopts a fully convolutional network, where the input feature map is upsampled by a factor of m to obtain a classification score map; The inner and outer canthal block adopts the design structure of the last three layers of the YOLO target detection neural network, divides the image into an n×n grid, predicts the grid where the inner and outer canthus falls and the coordinate offset relative to the center of the grid, and then Directly predict the coordinates of key points. 10. A device comprising a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor can be used to execute claims 1 to 6 when the processor executes the computer program The method of any of the above.

Images