CN115862122A - Fundus image acquisition method, fundus image acquisition device, computer equipment and readable storage medium - Google Patents
Fundus image acquisition method, fundus image acquisition device, computer equipment and readable storage medium Download PDFInfo
- Publication number
- CN115862122A CN115862122A CN202211683492.XA CN202211683492A CN115862122A CN 115862122 A CN115862122 A CN 115862122A CN 202211683492 A CN202211683492 A CN 202211683492A CN 115862122 A CN115862122 A CN 115862122A
- Authority
- CN
- China
- Prior art keywords
- event
- resolution image
- image
- low
- resolution
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 31
- 210000004220 fundus oculi Anatomy 0.000 claims abstract description 8
- 239000013307 optical fiber Substances 0.000 claims description 14
- 210000003786 sclera Anatomy 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 9
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000013528 artificial neural network Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 claims description 3
- 239000000126 substance Substances 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 6
- 210000001525 retina Anatomy 0.000 description 5
- 208000002177 Cataract Diseases 0.000 description 3
- 201000010099 disease Diseases 0.000 description 3
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 3
- 208000006069 Corneal Opacity Diseases 0.000 description 2
- 231100000269 corneal opacity Toxicity 0.000 description 2
- 238000002406 microsurgery Methods 0.000 description 2
- 230000002207 retinal effect Effects 0.000 description 2
- 238000001356 surgical procedure Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 101150050759 outI gene Proteins 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses a fundus image acquisition method, which comprises the following steps: acquiring video data of the fundus; acquiring a low-resolution image set formed by low-resolution images from video data, storing the low-resolution image set by taking a frame as a unit, acquiring an event sequence corresponding to the low-resolution image set, and dividing the event sequence into a corresponding number of event sequences according to the number of frames; acquiring a high-resolution image set consisting of high-resolution images from video data, and storing the high-resolution image set by taking a frame as a unit; determining image features of the low-resolution image; determining the event characteristics of the low-resolution image according to the image characteristics; determining global event characteristics of the low-resolution image according to the event characteristics; and determining a high-resolution image according to the image characteristic, the event characteristic and the global event characteristic. The definition of each frame of image in the video data of the fundus oculi is improved, so that some positions in the eye can be directly observed through a microscope, and the operation efficiency and the operation effect are improved.
Description
Technical Field
The invention relates to the technical field of image acquisition, in particular to a fundus image acquisition method, a fundus image acquisition device, computer equipment and a readable storage medium.
Background
In ophthalmic microsurgery, the fundus retinal surgery requires a doctor to observe the state of the fundus retina well and perform fine surgical operation, the difficulty of the surgical operation is high, and under normal conditions, the ophthalmologist can directly observe the fundus retina through an ophthalmic microscope, but diseases such as cataract, corneal opacity and the like cannot be directly observed through the ophthalmic microscope, and some positions in eyes cannot be directly observed through the microscope, so that the surgical efficiency is low, and the surgical effect is influenced.
Disclosure of Invention
In view of the above, it is necessary to provide a fundus image acquisition method, apparatus, computer device, and readable storage medium for solving the above problems.
A fundus image acquisition method, the method comprising:
acquiring video data of the fundus; the video data is acquired after entering from a sclera entrance through an ophthalmic optical fiber endoscope;
acquiring a low-resolution image set composed of low-resolution images from the video data, and storing the low-resolution image set in units of frames, wherein the low-resolution image set is marked as X = { X = { (X) 1 ,x 2 ,…,x i ,...,x N In which x i Represents the i frame low resolution image, i =1,2, and N, N is the frame number of the low resolution image; obtaining an event sequence corresponding to the low-resolution image set X, and dividing the event sequence into a corresponding number of event sequences according to the number of frames N, and recording the event sequences as E = { E = { (E) } 1 ,e 2 ,...,e i ,...,e N }，e i To representThe event sequence corresponding to the ith frame of low-resolution image;
acquiring a high-resolution image set composed of high-resolution images from the video data, and storing the high-resolution image set in units of frames, wherein the high-resolution image set is marked as Y = { Y = { Y = } 1 ,y 2 ,...,y j ,...,y N }，y j Representing a clear image of a j frame;
determining a training image data set, and marking as I = { X, E, Y }, wherein I represents the training image data set;
determining image features of the low resolution image;
determining an event characteristic of the low-resolution image according to the image characteristic;
determining global event characteristics of the low-resolution image according to the event characteristics;
determining a high resolution image according to the image feature, the event feature and the global event feature.
In one embodiment, the determining the image characteristic of the low resolution image comprises:
low resolution image x of ith frame i Processing the first convolution layer to obtain the ith frame of low-resolution image x i Key of (2) i Sum value i (ii) a Wherein, the key i Obtaining dictionary keys after the 2 nd convolutional layer processing; meanwhile, the key keyi is processed by a 3 rd convolution layer and a 1 st sigmoid layer to obtain a query key, and the dictionary key and the query key are multiplied to obtain an incidence matrix A i ,key；
Value i After 1 convolutional layer, the processed value is obtained i ', and associated with said correlation matrix A i Multiplying by key to obtain the ith low-resolution image x i The characteristics of the image of (a) are,
c represents the number of channels.
In one embodiment, the determining the event feature of the low resolution image according to the image feature comprises:
the ith event sequence e i After 3D voxelization, averagely dividing the voxelization into H blocks, then carrying out average processing on each block to obtain H sampled event key points, and marking the H-th event key point as an H-th event key point
Key point of h event
Is set as an event feature flat i,h Areas outside the h event keypoint are taken as its neighborhood->
For the h event key point
Corresponding event feature feat i,h Update is performed such that an updated h event keypoint @isobtained>
Event feature feat' i,h 。
In one embodiment, the event feature feat corresponding to the h event key point is defined by i,h The updating is carried out, and the updating is carried out,
feat’ i ,h ＝Up(Conv(feat i,h )) (1)
where Conv denotes a local convolution operation and Up denotes an upsampling operation.
In one embodiment, the determining the global event feature of the low resolution image according to the event feature comprises:
event feature { feat 'for all event key points' i,h } H Performing stereo graphic operation to obtain the ith event sequence e i Global event feature of
Wherein, the first and the second end of the pipe are connected with each other,
represents the ith event sequence e i The event characteristics corresponding to all event key points; up represents an upsampling operation and Conv represents a convolution operation; feat' i,q Event features representing the qth event key point; />
Representing a product function.
In one embodiment, said determining a high resolution image from said image features, said event features, said global event features and said pixel point number comprises:
characterizing the image
And said global event characteristic>
After x downsampling layers and y convolution layers sharing weight, the common characteristic ^ of the g group is obtained by the following formula>
Thereby obtaining the ith event sequence e i And the ith low resolution image x i Is based on a common characteristic>
Wherein (·,. Cndot.) represents an inner product,
represents a global event feature, and>
the G-th group of characteristics, G represents the number of groups;
defining the iteration number as p, and initializing p =1; defining the maximum iteration number as P; the ith event sequence e i And the ith low resolution image x i Are all characterized in
As input data for the p-th iteration;
respectively inputting input data of the p-th iteration into a formula (1) and a formula (2), and obtaining a global event characteristic and an image characteristic of the corresponding p-th iteration;
inputting the global event characteristic and the image characteristic of the p-th iteration into a formula (4) to obtain input data of the (p + 1) -th iteration; after P +1 is assigned to P, whether P is more than P is judged, if so, input data of the (P + 1) th iteration is input into a formula (5), and thus a j frame high-resolution predicted image is obtained
Wherein R is the high-resolution predicted image of the jth frame
Based on the number of pixels in the pixel(s), is greater than or equal to>
For the ith low resolution image x i An r-th pixel point of the high resolution image generated by the neural network>
For the ith high resolution image Y in the high resolution video image set Y i The corresponding r pixel point.
A fundus image acquisition apparatus, the apparatus comprising:
the first acquisition module is used for acquiring video data of the fundus oculi; the video data is acquired after entering from a sclera entrance through an ophthalmic optical fiber endoscope;
a second obtaining module, configured to obtain a low-resolution image set composed of low-resolution images from the video data, and store the low-resolution image set in units of frames, where X = { X = 1 ,x 2 ,…,x i ,...,x N In which x i Represents the i frame low resolution image, i =1,2, and N, N is the frame number of the low resolution image; a sequence of events corresponding to the low resolution image set X is obtained, and dividing the event sequence into a corresponding number of event sequences according to the number of frames N, and recording as E = { E = { (E) } 1 ,e 2 ,...,e i ,...,e N }，e i Representing an event sequence corresponding to the ith frame of low-resolution image; acquiring a high-resolution image set composed of high-resolution images from the video data, and storing the high-resolution image set in units of frames, wherein the high-resolution image set is marked as Y = { Y = { Y = } 1 ,y 2 ,...,y j ,...,y N }，y j Representing a clear image of a j frame; determining a training image data set, and marking as I = { X, E, Y }, wherein I represents the training image data set
The determining module is used for determining the event characteristics of the low-resolution image according to the image characteristics; determining global event characteristics of the low-resolution image according to the event characteristics; determining a high resolution image according to the image feature, the event feature and the global event feature.
A computer device comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of:
acquiring video data of the fundus; the video data is acquired after entering from a sclera entrance through an ophthalmic optical fiber endoscope;
acquiring a low-resolution image set composed of low-resolution images from the video data, and storing the low-resolution image set in units of frames, wherein the low-resolution image set is marked as X = { X = { (X) 1 ,x 2 ,…,x i ,...,x N In which x i Represents the i frame low resolution image, i =1,2, and N, N is the frame number of the low resolution image; obtaining an event sequence corresponding to the low-resolution image set X, and dividing the event sequence into a corresponding number of event sequences according to the number of frames N, and recording the event sequences as E = { E = { (E) } 1 ,e 2 ,...,e i ,...,e N }，e i Representing an event sequence corresponding to the ith frame of low-resolution image;
acquiring a high-resolution image set composed of high-resolution images from the video data, and storing the high-resolution image set in units of frames, wherein the high-resolution image set is marked as Y = { Y = { Y = } 1 ,y 2 ,...,y j ,...,y N }，y j Representing a clear image of a j frame;
determining a training image data set, and marking as I = { X, E, Y }, wherein I represents the training image data set;
determining image features of the low resolution image;
determining an event characteristic of the low-resolution image according to the image characteristic;
determining global event characteristics of the low-resolution image according to the event characteristics;
determining a high resolution image according to the image feature, the event feature and the global event feature.
A computer-readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of:
acquiring video data of the fundus; the video data is acquired after entering from a sclera entrance through an ophthalmic optical fiber endoscope;
acquiring a low-resolution image set composed of low-resolution images from the video data, and storing the low-resolution image set in units of frames, wherein the low-resolution image set is marked as X = { X = { (X) 1 ,x 2 ,…,x i ,...,x N In which x i Represents the i frame low resolution image, i =1,2, and N, N is the frame number of the low resolution image; obtaining an event sequence corresponding to the low-resolution image set X, and dividing the event sequence into a corresponding number of event sequences according to the number of frames N, and recording the event sequences as E = { E = { (E) } 1 ,e 2 ,...,e i ,...,e N }，e i Representing an event sequence corresponding to the ith frame of low-resolution image;
acquiring a high-resolution image set composed of high-resolution images from the video data, and storing the high-resolution image set in units of frames, wherein the high-resolution image set is marked as Y = { Y = { Y = } 1 ,y 2 ,...,y j ,...,y N }，y j Representing a j frame clear image;
determining a training image data set, and marking as I = { X, E, Y }, wherein I represents the training image data set;
determining image features of the low resolution image;
determining an event characteristic of the low-resolution image according to the image characteristic;
determining global event characteristics of the low-resolution image according to the event characteristics;
determining a high resolution image according to the image feature, the event feature and the global event feature.
The beneficial effect of write-exclusive.
The method comprises the steps of acquiring video data after entering from a sclera entrance through an ophthalmic optical fiber endoscope, acquiring a low-resolution image set and a high-resolution image set which are composed of low-resolution images from the video data, determining the image data set by the low-resolution image set and the high-resolution image set to determine the image characteristics of the image data set, determining the event characteristics and the global event characteristics of the low-resolution images according to the image characteristics, and determining the high-resolution images according to the image characteristics, the event characteristics and the global event characteristics. Therefore, the definition of each frame of image in the video data of the fundus oculi is improved, and some positions in the eye can be directly observed through a microscope, so that the operation efficiency and the operation effect are improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Wherein:
FIG. 1 is a diagram showing an environment in which a fundus image acquiring method is applied in one embodiment;
FIG. 2 is a flowchart of a fundus image acquisition method in one embodiment;
fig. 3 is a block diagram showing the configuration of a fundus image acquiring apparatus in one embodiment;
FIG. 4 is a view of an exemplary ophthalmic fiberoptic endoscope in use;
FIG. 5 is a block diagram of a computer device in one embodiment.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In ophthalmic microsurgery, the fundus retinal surgery requires a doctor to observe the state of the fundus retina well and perform fine surgical operation, the difficulty of the surgical operation is high, and under normal conditions, the ophthalmologist can directly observe the fundus retina through an ophthalmic microscope, but diseases such as cataract, corneal opacity and the like cannot be directly observed through the ophthalmic microscope, and some positions in eyes cannot be directly observed through the microscope, so that the surgical efficiency is low, and the surgical effect is influenced.
Fig. 1 is an environment diagram of an application of the fundus image acquiring method in one embodiment. Referring to fig. 1, the fundus image acquisition method is applied to a fundus image acquisition system. The fundus image acquisition system includes a terminal 110 and a server 120. The terminal 110 and the server 120 are connected through a network, the terminal 110 may be specifically a desktop terminal or a mobile terminal, and the mobile terminal may be specifically at least one of a mobile phone, a tablet computer, a notebook computer, and the like. The server 120 may be implemented as a stand-alone server or a server cluster composed of a plurality of servers. The terminal 110 is configured to send video data to the server 120, where the video data is acquired after entering from a sclera entrance through an ophthalmic fiber endoscope; the server 120 is used to receive video data of the fundus.
To solve the above technical problem, as shown in fig. 2, in one embodiment, the present application provides a fundus image acquisition method. The method can be applied to both a terminal and a server, and this embodiment is exemplified by being applied to a terminal. The fundus image acquisition method specifically comprises the following steps:
s1: acquiring video data of the fundus; the video data is acquired after entering from a sclera entrance through an ophthalmic optical fiber endoscope;
s2: acquiring a low-resolution image set composed of low-resolution images from the video data, and storing the low-resolution image set in units of frames, wherein the low-resolution image set is marked as X = { X = { (X) 1 ,x 2 ,…,x i ,...,x N In which x i Represents the i frame low resolution image, i =1,2, and N, N is the frame number of the low resolution image; obtaining an event sequence corresponding to the low-resolution image set X, and dividing the event sequence into a corresponding number of event sequences according to the number of frames N, and recording the event sequences as E = { E = { (E) } 1 ,e 2 ,...,e i ,...,e N }，e i Indicates that the ith frame is lowAn event sequence corresponding to the resolution image;
s3: acquiring a high-resolution image set composed of high-resolution images from the video data, and storing the high-resolution image set in units of frames, wherein the high-resolution image set is marked as Y = { Y = { Y = } 1 ,y 2 ,...,y j ,...,y N }，y j Representing a clear image of a j frame;
s4: determining a training image data set, and marking as I = { X, E, Y }, wherein I represents the training image data set;
s5: determining image features of the low resolution image;
s6: determining an event feature of the low-resolution image according to the image feature;
s7: determining global event characteristics of the low-resolution image according to the event characteristics;
s8: determining a high resolution image according to the image feature, the event feature and the global event feature.
The method comprises the steps of acquiring video data after entering from a sclera entrance through an ophthalmic optical fiber endoscope, acquiring a low-resolution image set and a high-resolution image set which are formed by low-resolution images from the video data, determining the image data set by the low-resolution image set and the high-resolution image set to determine the image characteristics of the image data set, determining the event characteristics and the global event characteristics of the low-resolution images according to the image characteristics, and determining the high-resolution images according to the image characteristics, the event characteristics and the global event characteristics. The definition of each frame of image in the video data of the fundus oculi is improved, so that some positions in the eye can be directly observed through a microscope, and the operation efficiency and the operation effect are improved.
In one embodiment, the determining the image characteristic of the low resolution image comprises:
s51: low resolution image x of ith frame i Processing the first convolution layer to obtain the ith frame of low-resolution image x i Key of i Sum value i (ii) a Wherein, the key i Obtaining dictionary keys after the 2 nd convolutional layer processing; at the same timeThe key keyi is processed by a 3 rd convolution layer and a 1 st sigmoid layer to obtain a query key, and the dictionary key and the query key are multiplied to obtain an incidence matrix A i ,key；
S52: value i After 1 convolutional layer, the processed value is obtained i ', and associated with said correlation matrix A i Multiplying by key to obtain the ith low-resolution image x i The characteristics of the image of (a) are,
c represents the number of channels.
In one embodiment, the determining the event feature of the low resolution image according to the image feature comprises:
s61: the ith event sequence e i After 3D voxelization, averagely dividing the voxelization into H blocks, then carrying out average processing on each block to obtain H sampled event key points, and marking the H-th event key point as an H-th event key point
S62: key point of h event
Is set as an event feature flat i,h Areas outside the h event keypoint are taken as its neighborhood->
S63: event characteristic feat corresponding to h-th event key point i,h Updating to obtain the updated h-th event key point
Event feature feat' i,h 。
In one embodiment, the event features corresponding to the h event key point are identified byfeat i,h The updating is carried out, and the updating is carried out,
feat’ i ,h ＝Up(Conv(feat i,h )) (1)
where Conv denotes a local convolution operation and Up denotes an upsampling operation.
In one embodiment, the determining the global event feature of the low resolution image according to the event feature comprises:
s71: event feature { feat 'for all event key points' i,h } H Performing stereo graphic operation to obtain the ith event sequence e i Global event feature of
Wherein the content of the first and second substances,
represents the ith event sequence e i Event characteristics corresponding to all event key points; up represents an upsampling operation and Conv represents a convolution operation; feat' i,q Event features representing the qth event key point; />
Representing a product function.
In one embodiment, said determining a high resolution image from said image features, said event features, said global event features and said pixel point number comprises:
s81: characterizing the image
And said global event characteristic>
After x downsampling layers and y convolution layers sharing weight, the common characteristic ^ of the g group is obtained by the following formula>
Thereby obtaining the ith event sequence e i And the ith low-resolution image x i Is based on a common characteristic>
Wherein (·,. Cndot.) represents an inner product,
represents a global event feature, and>
the G-th group of characteristics, G represents the number of groups;
s82: defining the iteration number as p, and initializing p =1; defining the maximum iteration number as P; the ith event sequence e i And the ith low resolution image x i Are all characterized in
As input data for the p-th iteration;
s83: respectively inputting input data of the p-th iteration into a formula (1) and a formula (2), and obtaining global event characteristics and image characteristics of the corresponding p-th iteration;
s84: inputting the global event characteristic and the image characteristic of the p-th iteration into a formula (4) to obtain input data of the (p + 1) -th iteration; after P +1 is assigned to P, whether P is more than P is judged, if so, the (P + 1) th iteration is carried outIs inputted into equation (5) to obtain a j-th frame high resolution predicted image
Wherein R is the high-resolution predicted image of the jth frame
Based on the number of pixels in the pixel(s), is greater than or equal to>
For the ith low resolution image x i The r-th pixel point of the high-resolution image generated by the neural network is judged>
And the pixel point is the r-th pixel point corresponding to the ith high-resolution image yi in the high-resolution video image set Y.
Even if the patient has cataract and other diseases, the state of the fundus oculi can be observed through the ophthalmic optical fiber endoscope, and the state of the fundus oculi retina can be also observed through the ophthalmic optical fiber endoscope for some dead angle positions of the fundus oculi.
A fundus image acquiring apparatus, as shown in fig. 3 and 4, the apparatus comprising:
a first acquisition module 10 for acquiring video data of a fundus; the video data is acquired after entering from a sclera entrance through an ophthalmic optical fiber endoscope;
a second obtaining module 20, configured to obtain a low-resolution image set composed of low-resolution images from the video data, and store the low-resolution image set in units of frames, where X = { X = 1 ,x 2 ,…,x i ,...,x N In which x i Represents the i-th frame low resolution image, i =1,2,..., N, N is the frame number of the low-resolution image; obtaining an event sequence corresponding to the low-resolution image set X, and dividing the event sequence into a corresponding number of event sequences according to the number of frames N, and recording the event sequences as E = { E = { (E) } 1 ,e 2 ,...,e i ,...,e N }，e i Representing an event sequence corresponding to the ith frame of low-resolution image; acquiring a high-resolution image set composed of high-resolution images from the video data, and storing the high-resolution image set in units of frames, wherein the high-resolution image set is marked as Y = { Y = { Y = } 1 ,y 2 ,...,y j ,...,y N }，y j Representing a clear image of a j frame; determining a training image data set, and marking as I = { X, E, Y }, wherein I represents the training image data set
A determining module 30, for determining an event feature of the low resolution image according to the image feature; determining global event characteristics of the low-resolution image according to the event characteristics; determining a high resolution image according to the image feature, the event feature and the global event feature. And may output the high resolution image to a display for use by a surgeon in performing surgical observations. FIG. 5 is a diagram illustrating an internal structure of a computer device in one embodiment. The computer device may be specifically a terminal, and may also be a server. As shown in fig. 5, the computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer apparatus stores an operating system, and may also store a computer program that, when executed by the processor, causes the processor to implement the fundus image acquisition method. The internal memory may also have a computer program stored therein, which when executed by the processor, causes the processor to perform the age identification method. It will be appreciated by those skilled in the art that the configuration shown in fig. 5 is a block diagram of only a portion of the configuration associated with the present application, and is not intended to limit the computing device to which the present application may be applied, and that a particular computing device may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
The present application also provides a computer device comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of:
acquiring video data of the fundus; the video data is acquired after entering from a sclera entrance through an ophthalmic optical fiber endoscope;
acquiring a low-resolution image set composed of low-resolution images from the video data, and storing the low-resolution image set in units of frames, wherein the low-resolution image set is marked as X = { X = { (X) 1 ,x 2 ,…,x i ,...,x N In which x i Represents the i frame low resolution image, i =1,2,. The N, N is the frame number of the low resolution image; obtaining an event sequence corresponding to the low-resolution image set X, and dividing the event sequence into a corresponding number of event sequences according to the number of frames N, and recording the event sequences as E = { E = { (E) } 1 ,e 2 ,...,e i ,...,e N }，e i Representing an event sequence corresponding to the ith frame of low-resolution image;
acquiring a high-resolution image set composed of high-resolution images from the video data, and storing the high-resolution image set in units of frames, wherein the high-resolution image set is marked as Y = { Y = 1 ,y 2 ,...,y j ,...,y N }，y j Representing a clear image of a j frame;
determining a training image data set, and marking as I = { X, E, Y }, wherein I represents the training image data set;
determining image features of the low resolution image;
determining an event feature of the low-resolution image according to the image feature;
determining global event characteristics of the low-resolution image according to the event characteristics;
determining a high resolution image according to the image feature, the event feature and the global event feature.
The present application also provides a computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of:
acquiring video data of the fundus; the video data is acquired after entering from a sclera entrance through an ophthalmic optical fiber endoscope;
acquiring a low-resolution image set composed of low-resolution images from the video data, and storing the low-resolution image set in units of frames, wherein the low-resolution image set is marked as X = { X = { (X) 1 ,x 2 ,…,x i ,...,x N In which x i Represents the i frame low resolution image, i =1,2, and N, N is the frame number of the low resolution image; obtaining an event sequence corresponding to the low-resolution image set X, and dividing the event sequence into a corresponding number of event sequences according to the number of frames N, and recording the event sequences as E = { E = { (E) } 1 ,e 2 ,...,e i ,...,e N }，e i Representing an event sequence corresponding to the ith frame of low-resolution image;
acquiring a high-resolution image set composed of high-resolution images from the video data, and storing the high-resolution image set in units of frames, wherein the high-resolution image set is marked as Y = { Y = { Y = } 1 ,y 2 ,...,y j ,...,y N }，y j Representing a j frame clear image;
determining a training image data set, and marking as I = { X, E, Y }, wherein I represents the training image data set;
determining image features of the low resolution image;
determining an event characteristic of the low-resolution image according to the image characteristic;
determining global event characteristics of the low-resolution image according to the event characteristics;
determining a high resolution image according to the image feature, the event feature and the global event feature.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a non-volatile computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the program is executed. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), rambus (Rambus) direct RAM (RDRAM), direct Rambus Dynamic RAM (DRDRAM), and Rambus Dynamic RAM (RDRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (9)
1. A fundus image acquisition method, the method comprising:
acquiring video data of the fundus; the video data is acquired after entering from a sclera entrance through an ophthalmic optical fiber endoscope;
acquiring a low-resolution image set composed of low-resolution images from the video data, and storing the low-resolution image set in units of frames, wherein the low-resolution image set is marked as X = { X = { (X) 1 ,x 2 ,…,x i ,...,x N In which x i Represents the i frame low resolution image, i =1,2,. The N, N is the frame number of the low resolution image; obtaining an event sequence corresponding to the low-resolution image set X, and dividing the event sequence into a corresponding number of event sequences according to the number of frames N, and recording the event sequences as E = { E = { (E) } 1 ,e 2 ,...,e i ,...,e N }，e i Representing an event sequence corresponding to the ith frame of low-resolution image;
acquiring a high-resolution image set composed of high-resolution images from the video data, and storing the high-resolution image set in units of frames, wherein the high-resolution image set is marked as Y = { Y = { Y = } 1 ,y 2 ,...,y j ,...,y N }，y j Representing a clear image of a j frame;
determining a training image data set, and marking as I = { X, E, Y }, wherein I represents the training image data set;
determining image features of the low resolution image;
determining an event characteristic of the low-resolution image according to the image characteristic;
determining global event characteristics of the low-resolution image according to the event characteristics;
determining a high resolution image according to the image feature, the event feature and the global event feature.
2. A fundus image acquiring method according to claim 1,
the determining image features of the low resolution image comprises:
low resolution image x of ith frame i Processing the first convolution layer to obtain the ith frame of low-resolution image x i Key of i Sum value i (ii) a Wherein, the key i Obtaining dictionary keys after the 2 nd convolutional layer processing; meanwhile, the key keyi is processed by a 3 rd convolution layer and a 1 st sigmoid layer to obtain a query key, and the dictionary key and the query key are multiplied to obtain an incidence matrix A i ,key；
Value i After passing through 1 convolution layerGet the processed value i ', and associated with said incidence matrix A i Multiplying by key to obtain the ith low-resolution image x i The characteristics of the image of (a) are,
c represents the number of channels.
3. A fundus image acquiring method according to claim 2,
the determining the event feature of the low resolution image according to the image feature comprises:
the ith event sequence e i After 3D voxelization, averagely dividing the voxelization into H blocks, then carrying out average processing on each block to obtain H sampled event key points, and marking the H-th event key point as an H-th event key point
Key point of h event
Is set as an event feature flat i,h Areas outside the h-th event keypoint are taken as neighbors thereof>
For the h event key point
Corresponding event feature feat i,h Updating to obtain updated h-th event key point>
Event feature feat' i,h 。
4. A fundus image acquiring method according to claim 3,
event feature feat corresponding to the h-th event key point by i,h The updating is carried out, and the updating is carried out,
feat’ i,h ＝Up(Conv(feat i,h )) (1)
where Conv denotes a local convolution operation and Up denotes an upsampling operation.
5. A fundus image acquiring method according to claim 4,
the determining the global event feature of the low-resolution image according to the event feature comprises:
event feature { feat 'for all event key points' i,h } H Performing stereo graphic operation to obtain the ith event sequence e i Global event feature of
Wherein the content of the first and second substances,
represents the ith event sequence e i Event characteristics corresponding to all event key points; up represents an upsampling operation and Conv represents a convolution operation; feat' i,q Event features representing the qth event key point; />
Representing a product function.
6. A fundus image acquiring method according to claim 5,
the determining a high resolution image according to the image feature, the event feature, the global event feature, and the pixel point number comprises:
characterizing the image
And the global event characteristic>
After x downsampling layers and y convolution layers sharing weight, the common characteristic ^ of the g group is obtained by the following formula>
Thereby obtaining the ith event sequence e i And the ith low resolution image x i Is based on a common characteristic>
Wherein (·,. Cndot.) represents an inner product,
represents a global event feature, and>
the G-th group of characteristics, G represents the number of groups;
defining the iteration number as p, and initializing p =1; defining the maximum iteration number as P; the ith event sequence e i And the ith low-resolution image x i Are all characterized in
As input data for the p-th iteration;
respectively inputting input data of the p-th iteration into a formula (1) and a formula (2), and obtaining global event characteristics and image characteristics of the corresponding p-th iteration;
inputting the global event characteristic and the image characteristic of the p-th iteration into a formula (4) to obtain input data of the (p + 1) -th iteration; after P +1 is assigned to P, whether P is more than P is judged, if so, input data of the (P + 1) th iteration is input into a formula (5), and thus a j frame high-resolution predicted image is obtained
Wherein R is the j frame high-resolution predicted image
Based on the number of pixels in the pixel(s), is greater than or equal to>
For the ith low resolution image x i The r-th pixel point of the high-resolution image generated by the neural network is judged>
For the ith high resolution image Y in the high resolution video image set Y i The corresponding r-th pixel point.
7. An fundus image acquisition apparatus, comprising:
the first acquisition module is used for acquiring video data of the fundus oculi; the video data is acquired after entering from a sclera entrance through an ophthalmic optical fiber endoscope;
a second obtaining module, configured to obtain a low-resolution image set composed of low-resolution images from the video data, and store the low-resolution image set in units of frames, where X = { X = 1 ,x 2 ,…,x i ,...,x N In which x i Represents the i frame low resolution image, i =1,2, and N, N is the frame number of the low resolution image; obtaining an event sequence corresponding to the low-resolution image set X, and dividing the event sequence into a corresponding number of event sequences according to the number of frames N, and recording the event sequences as E = { E = { (E) } 1 ,e 2 ,...,e i ,...,e N }，e i Representing an event sequence corresponding to the ith frame of low-resolution image; acquiring a high-resolution image set composed of high-resolution images from the video data, and storing the high-resolution image set in units of frames, wherein the high-resolution image set is marked as Y = { Y = { Y = } 1 ,y 2 ,...,y j ,...,y N }，y j Representing a clear image of a j frame; determining a training image data set, denoted as I = { X, E, Y }, wherein I represents the training image data set
The determining module is used for determining the event characteristics of the low-resolution image according to the image characteristics; determining global event characteristics of the low-resolution image according to the event characteristics; determining a high resolution image according to the image feature, the event feature and the global event feature.
8. A computer device comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium, storing a computer program which, when executed by a processor, causes the processor to carry out the steps of the method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211683492.XA CN115862122A (en) | 2022-12-27 | 2022-12-27 | Fundus image acquisition method, fundus image acquisition device, computer equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211683492.XA CN115862122A (en) | 2022-12-27 | 2022-12-27 | Fundus image acquisition method, fundus image acquisition device, computer equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115862122A true CN115862122A (en) | 2023-03-28 |
Family
ID=85653447
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211683492.XA Pending CN115862122A (en) | 2022-12-27 | 2022-12-27 | Fundus image acquisition method, fundus image acquisition device, computer equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115862122A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111667442A (en) * | 2020-05-21 | 2020-09-15 | 武汉大学 | High-quality high-frame-rate image reconstruction method based on event camera |
| WO2020253103A1 (en) * | 2019-06-17 | 2020-12-24 | 睿魔智能科技（深圳）有限公司 | Video image processing method, device, apparatus, and storage medium |
| CN114463218A (en) * | 2022-02-10 | 2022-05-10 | 中国科学技术大学 | Event data driven video deblurring method |
| CN114612305A (en) * | 2022-03-14 | 2022-06-10 | 中国科学技术大学 | Event-driven video super-resolution method based on stereogram modeling |
| CN115278013A (en) * | 2022-07-22 | 2022-11-01 | 杭州研极微电子有限公司 | Camera and method for controlling camera |
-
2022
- 2022-12-27 CN CN202211683492.XA patent/CN115862122A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020253103A1 (en) * | 2019-06-17 | 2020-12-24 | 睿魔智能科技（深圳）有限公司 | Video image processing method, device, apparatus, and storage medium |
| CN111667442A (en) * | 2020-05-21 | 2020-09-15 | 武汉大学 | High-quality high-frame-rate image reconstruction method based on event camera |
| CN114463218A (en) * | 2022-02-10 | 2022-05-10 | 中国科学技术大学 | Event data driven video deblurring method |
| CN114612305A (en) * | 2022-03-14 | 2022-06-10 | 中国科学技术大学 | Event-driven video super-resolution method based on stereogram modeling |
| CN115278013A (en) * | 2022-07-22 | 2022-11-01 | 杭州研极微电子有限公司 | Camera and method for controlling camera |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11270169B2 (en) | Image recognition method, storage medium and computer device | |
| US20210209762A1 (en) | Processing fundus images using machine learning models | |
| US20210256266A1 (en) | Target detection method, system, and apparatus, storage medium, and computer device | |
| CN109829894A (en) | Parted pattern training method, OCT image dividing method, device, equipment and medium | |
| US20210390706A1 (en) | Detection model training method and apparatus, computer device and storage medium | |
| CN110459324B (en) | Disease prediction method and device based on long-term and short-term memory model and computer equipment | |
| CN111862009B (en) | Classifying method of fundus OCT (optical coherence tomography) images and computer readable storage medium | |
| WO2022178997A1 (en) | Medical image registration method and apparatus, computer device, and storage medium | |
| CN110570407A (en) | image processing method, storage medium and computer device | |
| CN113889267A (en) | Method for constructing diabetes diagnosis model based on eye image recognition and electronic equipment | |
| CN112530550A (en) | Image report generation method and device, computer equipment and storage medium | |
| CN114881943B (en) | Brain age prediction method, device, equipment and storage medium based on artificial intelligence | |
| Thiéry et al. | Medical application of geometric deep learning for the diagnosis of glaucoma | |
| CN110929730A (en) | Image processing method, image processing device, computer equipment and storage medium | |
| Alazzam et al. | [Retracted] Corneal Biomechanics Computational Analysis for Keratoconus Diagnosis | |
| WO2021151355A1 (en) | Disease sorting method and apparatus based on reinforcement learning model, and device and medium | |
| CN115862122A (en) | Fundus image acquisition method, fundus image acquisition device, computer equipment and readable storage medium | |
| CN110334575B (en) | Fundus picture recognition method, device, equipment and storage medium | |
| US20220165421A1 (en) | Machine learning model based method and analysis system for performing covid-19 testing according to eye image captured by smartphone | |
| CN113378794A (en) | Information correlation method for elephant and symptom information | |
| Athilakshmi et al. | Enhancing Diabetic Retinopathy Diagnosis with Inception v4: A Deep Learning Approach | |
| CN111899879A (en) | Automatic eye table disease screening method and system and block chain | |
| Meshram et al. | MCBM: Implementation Of Multiclass And Transfer Learning Algorithm Based On Deep Learning Model For Early Detection Of Diabetic Retinopathy | |
| CN112446418A (en) | Fundus classification model integration system, method, electronic device and medium | |
| Chaskar et al. | Learning to predict diabetes from iris image analysis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20230328 |