CN115690389A - Cornea center positioning system in cataract operation based on deep learning - Google Patents
Cornea center positioning system in cataract operation based on deep learning Download PDFInfo
- Publication number
- CN115690389A CN115690389A CN202211162357.0A CN202211162357A CN115690389A CN 115690389 A CN115690389 A CN 115690389A CN 202211162357 A CN202211162357 A CN 202211162357A CN 115690389 A CN115690389 A CN 115690389A
- Authority
- CN
- China
- Prior art keywords
- cornea
- picture
- deep learning
- heat map
- gaussian
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Eye Examination Apparatus (AREA)
Abstract
The invention relates to the field of computer-assisted surgery systems, in particular to a cornea center positioning method and system in cataract surgery based on deep learning. The method comprises the following steps: acquiring cataract surgery video data in real time by utilizing a video recording device; converting video data into continuous picture frames, and preprocessing the continuous picture frames into a picture sequence; constructing a cornea Gaussian heat map generation network based on a deep learning model, inputting the preprocessed picture sequence into the cornea Gaussian heat map generation network, and obtaining a cornea heat map and a central position offset map; and decoding the cornea Gaussian heat map to obtain the peak position of the cornea, so as to obtain the central position coordinates of the cornea. The invention can accurately and quickly position the central position of the cornea in cataract operation.
Description
Technical Field
The invention relates to the field of computer-assisted surgery systems, in particular to a cornea center positioning system in cataract surgery based on deep learning.
Background
Cataract is a common eye disease and is one of the leading blinding factors in the world and China. Data statistics shows that the cataract incidence rate of people over 60 years old in China is about 80%. According to the estimation, the number of cataract patients in China is as high as 2.08 hundred million. Cataract refers to degenerative change of lens vision quality caused by reduction of lens transparency or color change, mainly caused by lens tissue change caused by aging or injury, and from the perspective of evidence-based medicine, no medicine is available for effectively treating cataract, and surgical treatment is the main means. At present, the domestic operation mode is mainly a refractive cataract operation scheme combining ultrasonic emulsification cataract aspiration and refractive intraocular lens (Toric IOL) implantation.
The accurate placement of the artificial lens and the prevention of the displacement and the rotation of the artificial lens become the difficulty of refractive cataract surgery, and have important influence on the postoperative vision recovery of cataract patients, the astigmatism axis of the Toric IOL and the corneal meridian are required to be accurately aligned in the cataract surgery, and related researches show that the 1-degree error can cause the corrected astigmatism to be reduced by 3.3 percent, and the accurate positioning of the cornea center is an important preposition task for realizing the accurate alignment of the astigmatism axis of the Toric IOL. The computer surgery aided navigation system can help a surgeon to accurately master the displacement and rotation conditions of the cornea in the surgical process by using a related picture processing method, the real-time accurate positioning of the cornea center is an indispensable part and the most critical part in the surgical navigation system, and the positioning accuracy directly influences the alignment condition of a Toric IOL astigmatism axis and a cornea meridian. In addition, most of the corneal center positioning methods in the conventional computer-assisted surgery navigation system are conventional image processing methods, a complex manually designed feature extractor is required to obtain the corneal limbus boundary profile so as to determine the center position of the corneal limbus boundary profile, and the corneal center positioning method is slow in processing speed, poor in generalization and still has a large promotion space.
Therefore, the cornea center positioning method and system in the cataract surgery are realized by learning of massive medical data by means of a deep learning method, and the cornea center positioning method and system have very important clinical application value.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: how to not only accurately position the cornea center in cataract surgery, but also meet the actual application requirements in real time, and provide the cornea center positioning service for the subsequent tasks of the computer surgery aided navigation system.
The technical scheme adopted by the invention is as follows: a cornea center positioning system in cataract operation based on deep learning comprises
The data acquisition module is used for acquiring cataract surgery video data;
the preprocessing module is used for preprocessing the cataract surgery video data obtained by the data acquisition module to form a preprocessed picture sequence;
a cornea Gaussian heat map generation network module constructed based on a deep learning model inputs the preprocessed image sequence into the cornea Gaussian heat map generation network module to obtain a cornea heat map and a central position offset map;
the decoding module is used for decoding the cornea heat map and the central position deviation map to obtain the peak position of the cornea heat map and the central position deviation map, and the decoding module outputs the central position coordinates of the cornea;
the data storage module is used for storing the central position coordinates of the cornea for being called by an external program;
and the data display module is used for displaying the central position coordinates of the cornea.
The data acquisition module is a video recording device, and in the cataract surgery process, the video recording device is adopted to record the eye surgery area of the patient to generate video data. The preprocessing module samples the video data into a continuous original picture sequence according to the rate of 30 frames/second, then carries out pixel completion on each original picture, namely finds out the value a with the longest length and the value b with the widest width in the original picture sequence, enables c to be equal to the maximum value in a and b, establishes a picture template filled with 0 pixel value of c, then places each original picture in the picture template at the middle position, ensures that each original picture is not positioned outside the picture template, then combines the layers to form a picture with the size of c, then adjusts the picture to be 256 multiplied by 256 resolution, namely the preprocessed picture, and all the preprocessed pictures form the preprocessed picture sequence.
The cornea Gaussian heat map generation network module constructed based on the deep learning model is characterized in that firstly, cornea Gaussian heat map data with real labels are generated to serve as a training set and a testing set, secondly, the deep learning model is constructed, the training set is input into the constructed deep learning model for training, then the testing set is input into the trained model for detecting the accuracy of a cornea center positioning result, training and detection are repeated until the accuracy of the cornea center positioning result reaches a set value, then, training is completed, and the obtained deep learning model is the cornea Gaussian heat map generation network module constructed based on the deep learning model. The corneal Gaussian heat map data with the real label is the corneal Gaussian heat map data obtained by adopting the self-adaptive Gaussian elliptic heat map method and has the real corneal Gaussian heat map data
The method for adaptively generating and adjusting the image according to different input images comprises the following steps
Step 11, preprocessing the past cataract surgery video data obtained by the data acquisition module to form a preprocessed picture sequence, labeling each picture by a professional ophthalmologist to obtain the central position coordinate of the cornea
Step 12, selecting a picture, and selecting a circular sub-area around the central position of the cornea, wherein the circle center of the sub-area is the central point of the cornea, and the radius is the radius of the cornea;
step 13, drawing a gradient direction histogram in the sub-region according to the gradient direction and the amplitude of each pixel in the sub-region, and further obtaining a gradient main direction in the sub-region, namely the direction with the maximum amplitude change;
step 14, taking the main gradient direction as the major axis direction of the Gaussian ellipse, and taking the direction perpendicular to the gradient direction as the minor axis direction of the Gaussian ellipse;
step 15, respectively calculating pixel difference degree information in the major axis direction and the minor axis direction of the Gaussian ellipse;
step 16, calculating to obtain standard deviations in the major axis direction and the minor axis direction of the Gaussian ellipse according to the pixel difference information in the major axis direction and the minor axis direction of the Gaussian ellipse;
step 17, constructing a self-adaptive Gaussian ellipse heat map of the picture according to the standard deviation of the picture in the direction of the long axis of the Gaussian ellipse, the standard deviation of the picture in the direction of the short axis of the Gaussian ellipse and the main gradient direction;
and 18, repeating the steps 12 to 18 until all the pictures in the preprocessed picture sequence are processed, and dividing all the pictures into a training set and a test set according to the proportion of 4:1. The method comprises the steps that a deep learning model is built, an improved high-resolution key point detection network HRNet model is adopted, the deep feature information in a high-resolution feature map can be mined, a heat map with cornea center position information and a center position offset map are generated, and the improved high-resolution key point detection network HRNet model comprises a down-sampling layer, a backbone network, a deconvolution layer and two convolution branches; the downsampling layer uses convolution operation to continuously downsample the input picture to obtain a feature map with small resolution; the backbone network is a standard HRNet model, and is used for extracting and fusing the features of the feature map with small resolution to obtain a depth feature map; the deconvolution layer is obtained by adopting two times of transposition convolution operations to up-sample the scale of the depth feature map into the scale consistent with the input picture; the two convolution branches are two groups of independent convolution operations, and the number of channels is reduced while the size of the characteristic diagram is kept unchanged in the convolution process; the two convolution branches are respectively used for realizing the prediction of the cornea heat map and the prediction of the central position deviation map. The mode for training the improved HRNet model comprises a training method and a loss function, wherein the training method is optimized by using an Adam optimizer, and the loss function is a weighted mixed loss function comprising global MSE loss, local MSE loss, central coordinate loss and coordinate offset loss.
The decoding process of the decoding module is to determine a peak position coordinate in the cornea heat map, obtain coordinate offset of the peak position coordinate in the x-axis and y-axis directions in the center position offset map, and calculate a position coordinate of the cornea center in the original picture, namely a cornea center position coordinate, according to the peak position coordinate, the coordinate offset of the peak position coordinate in the center position offset map in the x-axis and y-axis directions and the original high resolution picture zoom ratio. The data acquisition module is a video recording device.
The pre-processing module is a GPU graphics processor and a computer Chen Xu. The cornea Gaussian heat map generation network module and the decoding module are a CPU and a computer program. The data storage module is a hard disk. The data display module is a display. The invention also includes high speed communication lines.
The beneficial technical effects of the invention are as follows: the cornea center feature information is extracted and the position coordinates are determined end to end through the deep learning model, the center position is determined without depending on a method for extracting and fitting the cornea contour in the traditional method, and the method has higher calculation efficiency and precision. Meanwhile, the adopted self-adaptive Gaussian elliptic thermograph can self-adaptively adjust the real thermograph label according to different training pictures, so that the deep learning model can fully mine characteristic information related to learning and center, and the accuracy of corneal center positioning in cataract surgery is further improved.
The invention aims at the video data in the cataract surgery, quickly and accurately positions the center of the cornea, reduces the influence of the deviation of the center of the cornea on the postoperative residual astigmatism of a patient, further improves the surgery quality, provides stable central position coordinates for the alignment of the corneal meridian in the surgery, and is an indispensable part in an auxiliary navigation system for the cataract surgery.
Drawings
Fig. 1 is a structure diagram of an HRNet network according to an embodiment of the present invention;
FIG. 2 is a detailed block diagram of the present invention.
Detailed Description
The technical solutions of the present invention are specifically and thoroughly described below with reference to the accompanying drawings of the embodiments of the present invention, so that the technical features of the present invention can be more easily understood by those skilled in the art. It should be noted that the specific embodiments listed herein are only exemplary of the present invention, and do not limit the scope of the present invention.
A corneal centering system in cataract surgery based on deep learning, comprising:
the data acquisition module is used for acquiring cataract surgery video data;
the preprocessing module is used for preprocessing the cataract surgery video data obtained by the data acquisition module to form a preprocessed picture sequence;
a cornea Gaussian heat map generation network module constructed based on a deep learning model inputs the preprocessed picture sequence into the cornea Gaussian heat map generation network module to obtain a cornea heat map and a central position offset map;
the decoding module is used for decoding the cornea heat map and the central position deviation map to obtain the peak position of the cornea heat map and the central position deviation map, and the decoding module outputs the central position coordinates of the cornea;
the data storage module is used for storing the central position coordinates of the cornea for being called by an external program;
and the data display module is used for displaying the central position coordinates of the cornea.
The data acquisition module is a video recording device, and in the cataract surgery process, the video recording device is adopted to record the eye surgery area of the patient to generate video data. The preprocessing module samples the video data into a continuous original picture sequence according to the rate of 30 frames/second, then carries out pixel completion on each original picture, namely finds out the value a with the longest length and the value b with the widest width in the original picture sequence, enables c to be equal to the maximum value in a and b, establishes a picture template filled with 0 pixel value of c, then places each original picture in the picture template at the middle position, ensures that each original picture is not positioned outside the picture template, then combines the layers to form a picture with the size of c, then adjusts the picture to be 256 multiplied by 256 resolution, namely the preprocessed picture, and all the preprocessed pictures form the preprocessed picture sequence.
The cornea Gaussian heat map generation network module constructed based on the deep learning model is characterized in that firstly, cornea Gaussian heat map data with real labels are generated to serve as a training set and a testing set, secondly, the deep learning model is constructed, the training set is input into the constructed deep learning model for training, then the testing set is input into the trained model for detecting the accuracy of a cornea center positioning result, the training and the detection are repeated until the accuracy of the cornea center positioning result reaches a set value, the training is completed, and the obtained deep learning model is the cornea Gaussian heat map generation network module constructed based on the deep learning model. The corneal Gaussian heat map data with the real label is the corneal Gaussian heat map data obtained by adopting a self-adaptive Gaussian ellipse heat map method and has real corneal Gaussian heat map data
The method for adaptively generating and adjusting the image according to different input images comprises the following steps
Step 11, preprocessing the video data of the previous cataract surgery obtained by the data acquisition module to form a preprocessed picture sequence, labeling each picture by a professional ophthalmologist to obtain the coordinate (x) of the central position of the cornea in the picture c ,y c ) (ii) a In one embodiment, the left vertex of the picture is taken as the origin of coordinates, the x axis is from the left vertex of the picture to the right vertex of the picture, and the y axis is from the left vertex of the picture to the left bottom point of the picture.
Step 12, selecting a picture in the center of the cornea (x) c ,y c ) Selecting a circular sub-area around the cornea, wherein the circle center of the sub-area is the central point of the cornea, and the radius of the sub-area is the radius R of the cornea;
step 13, calculating the gradient direction and amplitude of each pixel in the circular sub-area:
where I (x, y) represents a pixel value at (x, y) coordinates in the picture, m (x, y) represents a gradient magnitude of the corresponding pixel location ((x, y) coordinates), and θ (x, y) represents a gradient direction of the corresponding pixel location. Then drawing a gradient direction histogram in the sub-region according to the gradient direction and the amplitude value, and further obtaining a gradient main direction theta in the sub-region; x represents the abscissa value, y represents the ordinate value, x + -1 represents the abscissa value plus or minus 1,y + -1 on the x basis, and the ordinate value plus or minus 1 on the y basis.
Step 14, regarding the gradient main direction θ as the major axis direction θ of the Gaussian ellipse a The direction perpendicular to the gradient direction is defined as the minor axis direction θ of the Gaussian ellipse b ；
Step 15, according to the formula (2), calculating the pixel difference degree of the Gaussian ellipse in the major axis direction and the minor axis direction respectively
And
arbitrary theta of gaussian ellipse x Degree of axial pixel difference
Wherein R represents the radius of the selected sub-region,
is expressed at theta x A pixel value f having a distance i from the center position in the axial direction 0 Representing the center pixel value.
Step 16, according to the formula (3), utilizing the pixel difference degree of the Gaussian ellipse in the major axis direction and the minor axis direction
And
calculating to obtain the standard deviation sigma of the Gaussian function in the direction of the long axis of the ellipse a And standard deviation σ in the minor axis direction b .
Wherein the content of the first and second substances,
σ 0 the initial standard deviation of the two-dimensional gaussian distribution is represented, which has a value of one third of the corneal radius in the input picture.
Step 17, according to the standard deviation sigma of the long axis direction of the Gaussian ellipse of the picture a Standard deviation sigma in the direction of the minor axis of the gaussian ellipse of a picture b And (3) constructing an adaptive Gaussian elliptic heat map according to a formula (4) according to the main gradient direction theta:
where G (x, y) ∈ [0,1], represents the Gaussian value at the (x, y) position in the adaptive Gaussian elliptic heatmap.
And 18, repeating the steps 12 to 18 until all the pictures in the preprocessed picture sequence are processed, and dividing all the pictures into a training set and a test set according to the proportion of 4:1.
The method comprises the steps that a deep learning model is built, an improved high-resolution key point detection network HRNet model is adopted, the deep feature information in a high-resolution feature map can be mined, a heat map with cornea center position information and a center position offset map are generated, and the improved high-resolution key point detection network HRNet model comprises a down-sampling layer, a backbone network, a deconvolution layer and two convolution branches; the downsampling layer uses convolution operation to continuously downsample the input picture to obtain a feature map with small resolution; the backbone network is a standard HRNet model, and is used for extracting and fusing the features of the feature map with small resolution to obtain a depth feature map; the deconvolution layer is obtained by sampling the scale of the depth feature map up to the scale consistent with the input picture by two times of transposition convolution operation; the two convolution branches are two groups of independent convolution operations, and the number of channels is reduced while the size of the characteristic diagram is kept unchanged in the convolution process; the two convolution branches are respectively used for realizing the prediction of the cornea heat map and the prediction of the central position deviation map. The mode of training the improved HRNet model comprises a training method and a loss function, wherein the training method is that an Adam optimizer is used for optimization, and the loss function is a weighted mixed loss function which comprises global MSE loss, local MSE loss, central coordinate loss and coordinate offset loss. The decoding process of the decoding module is to determine a peak position coordinate in the cornea heat map, obtain coordinate offset of the peak position coordinate in the x-axis and y-axis directions in the center position offset map, and calculate a position coordinate of the cornea center in the original picture, namely a cornea center position coordinate, according to the peak position coordinate, the coordinate offset of the peak position coordinate in the center position offset map in the x-axis and y-axis directions and the original high resolution picture zoom ratio. The data acquisition module is a video recording device.
The pre-processing module is a GPU graphics processor and a computer Chen Xu. The cornea Gaussian heat map generation network module and the decoding module are a CPU and a computer program. The data storage module is a hard disk. The data display module is a display. The invention also includes high speed communication lines.
In some embodiments, the converting the corneal center coordinate tag into a form of a true adaptive gaussian heat map comprises the steps of:
in some embodiments, the true coordinate offset of the corneal center is:
wherein (O) x ,O y ) Representing the true coordinate offset of the corneal center in the x and y directions, (I) x ,I y ) Showing the corneaThe coordinate value of the center in the original picture, and λ represents the down-sampling magnification, i.e. the ratio of the resolution of the original picture to the input picture.
In some embodiments, the deep learning-based corneal gaussian heatmap generation network model employs a modified HRNet network, and fig. 1 shows a modified HRNet network structure diagram, which includes a downsampling layer, a backbone network, a deconvolution layer and two convolution branches; the downsampling layer uses convolution operation to continuously downsample the input picture to obtain a feature map with small resolution; the backbone network is a standard HRNet model, and the backbone network is used for extracting and fusing the features of the feature maps with small resolution to obtain a depth feature map; the deconvolution layer is obtained by adopting two times of transposition convolution operations to up-sample the scale of the depth feature map into the scale consistent with the input picture; the two convolution branches are two groups of independent convolution operations, and the number of channels is reduced while the size of the characteristic diagram is kept unchanged in the convolution process; the two convolution branches are respectively used for realizing the prediction of the cornea heat map and the prediction of the central position deviation map.
In some embodiments, the hybrid loss function is a weighting of four loss functions, respectively:
global MSE loss function:
where W, H is the width and height of the heat map picture, and P (i, j), T (i, j) e [0,1] are the values of the (i, j) location in the predicted heat map P and the real heat map T, respectively.
Local MSE loss function:
in the formula, D is a set of all pixel points in the gaussian elliptical distribution area in the real heat map T, and the symbol table |, indicates that multiplication is performed by element correspondence.
Center coordinate loss function:
in the formula (C) x ,C y ) And
respectively representing the true center coordinate position and the predicted center coordinate position.
Coordinate offset penalty function:
wherein (O) x ,O y ) And
representing the true corneal center position offset and the predicted corneal center position offset, respectively.
Further, the corneal gaussian heat map generation network generates a predicted corneal heat map sequence and a central position offset map sequence in time sequence, and in step S4, the corneal heat map sequence is sequentially subjected to position coordinate decoding: obtaining the coordinates of the peak position in the cornea heat map by carrying out global search on the predicted cornea heat map and recording the coordinates as
According to the peak position coordinates
Obtaining predicted amounts of shift of coordinates in x and y directions at corresponding positions in the center position shift map
According to the peak position coordinates
Offset amount
And calculating the position coordinates of the cornea center in the original picture by the downsampling multiplying factor lambda
The position coordinates of the cornea center in the original picture
The calculation formula of (c) is:
in some embodiments, the predicted corneal center coordinate position is determined if the original image was filled
The filling values in the respective directions should be subtracted.
As a second embodiment of the present invention, there is provided a cornea centering system in cataract surgery based on deep learning, including a server terminal device, the server terminal device including: the system comprises a GPU (graphics processing unit), a CPU (central processing unit), a high-speed communication line, a data memory and a computer program, wherein the computer program is executed to start a cornea center positioning system, the cornea center positioning system carries out a real-time cornea center positioning task on a video in cataract surgery by using a deep neural network, and stores the cornea center position in the data memory. Fig. 2 shows a detailed block diagram of the system. The system comprises the following modules:
the data acquisition module is used for acquiring video stream data of an eye operation area of a patient in cataract operation;
the data preprocessing module is used for converting the video stream data into continuous frames and adjusting the continuous frames into a picture sequence with a specific resolution;
the cornea Gaussian heat map generation module is used for constructing a cornea Gaussian heat map generation network based on a deep learning model, inputting the image sequence into the cornea Gaussian heat map generation network and obtaining a predicted cornea heat map sequence and a central position deviation map sequence;
the position coordinate decoding module is used for carrying out coordinate decoding on the cornea heat map sequence to obtain the position coordinate of the cornea center in the original picture;
and the data storage module is used for storing the coordinate information of the central position of the cornea in the data storage.
Preferably, the data acquiring module 501 is configured to acquire real-time video stream data of an eye region of a patient during cataract surgery. The data acquisition module acquires video output signals from professional operation video recording equipment in a high-speed wired communication mode at low time delay and high quality.
Preferably, the data preprocessing module 502 is configured to process the video output signal. And converting the operation video data into continuous frames, sampling the continuous frames into a continuous original picture sequence at the rate of 30 frames/second, filling the original picture sequence, and adjusting the original picture sequence to 256 multiplied by 256 resolution to be used as an input picture sequence of a network model.
In some embodiments, the filling manner of the picture is to fill the original picture with 0 pixel value to the outer areas on both sides of the short side of the picture at equal intervals, so that the height and the width of the picture are equal to maintain the consistency of the picture structure.
Preferably, the cornea heat map generation module 503 calculates the input image sequence in real time through a cornea gaussian heat map generation network model constructed based on deep learning and deployed on a high-performance GPU server, and outputs a predicted cornea heat map sequence and a central position offset map sequence in real time.
In some embodiments, the cornea heat map generation network model is an improved HRNet network model, after the network model is constructed, a training set is input into a network for training, then a test set is input into the trained model to verify the accuracy of the positioning result, if the positioning accuracy requirement is met, the network model is deployed at a server terminal, and if the positioning accuracy requirement is not met, the network model continues to be trained.
In some embodiments, the training set is constructed by converting the corneal center coordinate labels into a real adaptive gaussian heat map form as real training set label data;
preferably, the coordinate position decoding module 504 is configured to perform position coordinate decoding on the predicted cornea heat map sequence, and obtain coordinate information of a cornea center position in the original picture by combining with the center position offset map sequence.
In some embodiments, the predicted corneal center coordinate position is determined if the original image was filled
The filling values in the respective directions should be subtracted.
Preferably, the data storage module 505 is configured to store the real-time corneal center coordinate information in a data storage device, which can be called by an external program.
In some embodiments, the modules 501 to 505 are executed in real-time sequentially, that is, the image sequence passes through modules 501, 502, 503, 504 and 505 sequentially in a data flow manner, until the operation is finished (the last image in the image sequence is processed), and the system is stopped.
In view of the above, according to the method and system for positioning the cornea center in the cataract surgery based on the deep learning, firstly, the video data of the eye region of the patient in the cataract surgery is obtained in real time and is preprocessed to obtain the input picture sequence, the input picture sequence is sequentially subjected to the establishment of the cornea gauss heat map generation network based on the deep learning model in a data stream mode to obtain the predicted cornea heat map sequence and the center position offset map sequence, and the cornea heat map sequence and the center position offset map sequence are subjected to coordinate decoding through the position coordinate decoding module to obtain the position coordinate of the cornea center in the real-time original picture and store the position coordinate. The method is beneficial to realizing the rapid and accurate corneal center positioning in the operation, and can provide the precise corneal center position for the alignment of the Toric IOL astigmatism axis and the corneal meridian.
The above embodiments are merely exemplary embodiments of the present invention, and various changes and modifications within the spirit and principle of the present invention, which are within the understanding of those skilled in the art, are within the scope of the present invention.
Claims (7)
1. A cornea center positioning system in cataract operation based on deep learning is characterized in that: the cataract surgery video data acquisition module is used for acquiring cataract surgery video data;
the preprocessing module is used for preprocessing the cataract surgery video data obtained by the data acquisition module to form a preprocessed picture sequence;
a cornea Gaussian heat map generation network module constructed based on a deep learning model inputs the preprocessed picture sequence into the cornea Gaussian heat map generation network module to obtain a cornea heat map and a central position offset map;
the decoding module is used for decoding the cornea heat map and the central position deviation map to obtain the peak position of the cornea heat map and the central position deviation map, and the decoding module outputs the central position coordinates of the cornea;
the data storage module is used for storing the central position coordinates of the cornea for being called by an external program;
and the data display module is used for displaying the central position coordinates of the cornea.
2. The deep learning based corneal centration system for cataract surgery as claimed in claim 1, wherein: the data acquisition module is a video recording device, and in the cataract surgery process, the video recording device is adopted to record the eye surgery area of the patient to generate video data.
3. The system for corneal center positioning in cataract surgery based on deep learning of claim 1, wherein: the preprocessing module samples the video data into a continuous original picture sequence according to the rate of 30 frames/second, then carries out pixel completion on each original picture, namely finds out the value a with the longest length and the value b with the widest width in the original picture sequence, enables c to be equal to the maximum value in a and b, establishes a picture template filled with 0 pixel value of c, then places each original picture in the picture template at the middle position, ensures that each original picture is not positioned outside the picture template, then combines the layers to form a picture with the size of c, then adjusts the picture to be 256 multiplied by 256 resolution, namely the preprocessed picture, and all the preprocessed pictures form the preprocessed picture sequence.
4. The deep learning based corneal centration system for cataract surgery as claimed in claim 1, wherein: the cornea Gaussian heat map generation network module constructed based on the deep learning model is characterized in that firstly, cornea Gaussian heat map data with real labels are generated to serve as a training set and a testing set, secondly, the deep learning model is constructed, the training set is input into the constructed deep learning model for training, then the testing set is input into the trained model for detecting the accuracy of a cornea center positioning result, the training and the detection are repeated until the accuracy of the cornea center positioning result reaches a set value, the training is completed, and the obtained deep learning model is the cornea Gaussian heat map generation network module constructed based on the deep learning model.
5. The deep learning based corneal centration system for cataract surgery according to claim 4, wherein: the method for preparing the cornea Gaussian heat map data with the real label comprises the following steps of
Step 11, preprocessing the past cataract surgery video data obtained by the data acquisition module to form a preprocessed picture sequence, and labeling each picture by a professional ophthalmologist to obtain the central position coordinates of the cornea; step 12, selecting a picture, and selecting a circular sub-area around the central position of the cornea, wherein the circle center of the sub-area is the central point of the cornea, and the radius is the radius of the cornea;
step 13, drawing a gradient direction histogram in the sub-region according to the gradient direction and the amplitude of each pixel in the sub-region, and further obtaining a gradient main direction in the sub-region, namely the direction with the maximum amplitude change;
step 14, taking the main gradient direction as the major axis direction of the Gaussian ellipse, and taking the direction perpendicular to the gradient direction as the minor axis direction of the Gaussian ellipse;
step 15, respectively calculating pixel difference degree information in the major axis direction and the minor axis direction of the Gaussian ellipse;
step 16, calculating to obtain standard deviations in the major axis direction and the minor axis direction of the Gaussian ellipse according to the pixel difference information in the major axis direction and the minor axis direction of the Gaussian ellipse;
step 17, constructing a self-adaptive Gaussian ellipse heat map of the picture according to the standard deviation of the picture in the direction of the long axis of the Gaussian ellipse, the standard deviation of the picture in the direction of the short axis of the Gaussian ellipse and the main gradient direction;
and 18, repeating the steps 12 to 18 until all the pictures in the preprocessed picture sequence are processed, and dividing all the pictures into a training set and a test set according to the proportion of 4:1.
6. The deep learning based corneal centration system for cataract surgery according to claim 4, wherein: the method comprises the steps that a deep learning model is built, an improved high-resolution key point detection network HRNet model is adopted, the deep feature information in a high-resolution feature map can be mined, a heat map with cornea center position information and a center position offset map are generated, and the improved high-resolution key point detection network HRNet model comprises a down-sampling layer, a backbone network, a deconvolution layer and two convolution branches; the downsampling layer uses convolution operation to continuously downsample the input picture to obtain a feature map with small resolution; the backbone network is a standard HRNet model, and the backbone network is used for extracting and fusing the features of the feature maps with small resolution to obtain a depth feature map; the deconvolution layer is obtained by adopting two times of transposition convolution operations to up-sample the scale of the depth feature map into the scale consistent with the input picture; the two convolution branches are two groups of independent convolution operations, and the number of channels is reduced while the size of the characteristic diagram is kept unchanged in the convolution process; the two convolution branches are respectively used for predicting the cornea heat map and predicting the central position offset map, the mode of training the improved HRNet model comprises a training method and a loss function, the training method is that an Adam optimizer is used for optimization, and the loss function is a weighted mixed loss function comprising global MSE loss, local MSE loss, central coordinate loss and coordinate offset loss.
7. The system for corneal center location in cataract surgery based on deep learning of claim 4, wherein: the decoding process of the decoding module is to determine the peak position coordinate in the cornea heat map, obtain the coordinate offset of the peak position coordinate in the axis and axis directions in the central position offset map, and calculate the position coordinate of the cornea center in the original picture, namely the cornea center position coordinate, according to the peak position coordinate, the coordinate offset of the peak position coordinate in the central position offset map in the axis and axis directions and the original high resolution picture scaling factor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211162357.0A CN115690389A (en) | 2022-09-21 | 2022-09-21 | Cornea center positioning system in cataract operation based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211162357.0A CN115690389A (en) | 2022-09-21 | 2022-09-21 | Cornea center positioning system in cataract operation based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115690389A true CN115690389A (en) | 2023-02-03 |
Family
ID=85061859
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211162357.0A Pending CN115690389A (en) | 2022-09-21 | 2022-09-21 | Cornea center positioning system in cataract operation based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115690389A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116942317A (en) * | 2023-09-21 | 2023-10-27 | 中南大学 | Surgical navigation positioning system |
-
2022
- 2022-09-21 CN CN202211162357.0A patent/CN115690389A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116942317A (en) * | 2023-09-21 | 2023-10-27 | 中南大学 | Surgical navigation positioning system |
| CN116942317B (en) * | 2023-09-21 | 2023-12-26 | 中南大学 | Surgical navigation positioning system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109191476A (en) | The automatic segmentation of Biomedical Image based on U-net network structure | |
| CN110599528A (en) | Unsupervised three-dimensional medical image registration method and system based on neural network | |
| Bian et al. | Optic disc and optic cup segmentation based on anatomy guided cascade network | |
| CN111260667B (en) | Neurofibromatosis segmentation method combined with spatial guidance | |
| CN108510493A (en) | Boundary alignment method, storage medium and the terminal of target object in medical image | |
| WO2024001140A1 (en) | Vertebral body sub-region segmentation method and apparatus, and storage medium | |
| CN113706562B (en) | Image segmentation method, device and system and cell segmentation method | |
| CN112288794B (en) | Method and device for measuring blood vessel diameter of fundus image | |
| CN115690389A (en) | Cornea center positioning system in cataract operation based on deep learning | |
| CN113223028A (en) | Multi-modal liver tumor segmentation method based on MR and CT | |
| CN113822845A (en) | Method, apparatus, device and medium for hierarchical segmentation of tissue structure in medical image | |
| CN112215217A (en) | Digital image recognition method and device for simulating doctor to read film | |
| CN116758087B (en) | Lumbar vertebra CT bone window side recess gap detection method and device | |
| CN112966759B (en) | Fundus image identification and classification method based on deep learning | |
| Goutham et al. | Automatic localization of landmarks in cephalometric images via modified U-Net | |
| Kreitner et al. | Synthetic optical coherence tomography angiographs for detailed retinal vessel segmentation without human annotations | |
| CN116188479B (en) | Hip joint image segmentation method and system based on deep learning | |
| Liu et al. | Tracking-based deep learning method for temporomandibular joint segmentation | |
| CN115409857A (en) | Three-dimensional hydrocephalus CT image segmentation method based on deep learning | |
| CN112734769B (en) | Medical image segmentation and quantitative analysis method based on interactive information guided deep learning method, computer device and storage medium | |
| CN112116673B (en) | Virtual human body image generation method and system based on structural similarity under posture guidance and electronic equipment | |
| CN115359257A (en) | Deep learning-based spine image segmentation method and operation navigation positioning system | |
| CN112330660B (en) | Sperm tail detection method and system based on neural network | |
| CN109509262B (en) | Intelligent enhanced modeling method and device based on artificial intelligence | |
| CN112712507A (en) | Method and device for determining calcified area of coronary artery |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |