CN114587272A - In-vivo fluorescence imaging deblurring method based on deep learning - Google Patents
In-vivo fluorescence imaging deblurring method based on deep learning Download PDFInfo
- Publication number
- CN114587272A CN114587272A CN202210182173.4A CN202210182173A CN114587272A CN 114587272 A CN114587272 A CN 114587272A CN 202210182173 A CN202210182173 A CN 202210182173A CN 114587272 A CN114587272 A CN 114587272A
- Authority
- CN
- China
- Prior art keywords
- infrared
- fluorescence
- fluorescence image
- zone
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000000799 fluorescence microscopy Methods 0.000 title claims abstract description 50
- 238000013135 deep learning Methods 0.000 title claims abstract description 23
- 238000001727 in vivo Methods 0.000 title claims description 19
- 238000002073 fluorescence micrograph Methods 0.000 claims abstract description 145
- 238000012549 training Methods 0.000 claims abstract description 36
- 238000012795 verification Methods 0.000 claims abstract description 18
- 230000035945 sensitivity Effects 0.000 claims abstract description 12
- 238000012360 testing method Methods 0.000 claims abstract description 11
- 230000000007 visual effect Effects 0.000 claims abstract description 11
- 238000003384 imaging method Methods 0.000 claims description 38
- 241000699666 Mus <mouse, genus> Species 0.000 claims description 14
- 230000003287 optical effect Effects 0.000 claims description 11
- 230000006870 function Effects 0.000 claims description 10
- 238000004422 calculation algorithm Methods 0.000 claims description 8
- 238000010276 construction Methods 0.000 claims description 8
- 230000005284 excitation Effects 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 6
- 230000008859 change Effects 0.000 claims description 5
- 206010002091 Anaesthesia Diseases 0.000 claims description 4
- 241000699670 Mus sp. Species 0.000 claims description 4
- 230000037005 anaesthesia Effects 0.000 claims description 4
- QVGXLLKOCUKJST-UHFFFAOYSA-N atomic oxygen Chemical compound [O] QVGXLLKOCUKJST-UHFFFAOYSA-N 0.000 claims description 4
- 239000007789 gas Substances 0.000 claims description 4
- 239000001301 oxygen Substances 0.000 claims description 4
- 229910052760 oxygen Inorganic materials 0.000 claims description 4
- 230000008569 process Effects 0.000 claims description 4
- 238000001949 anaesthesia Methods 0.000 claims description 3
- 230000008447 perception Effects 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 2
- 230000000694 effects Effects 0.000 description 10
- MOFVSTNWEDAEEK-UHFFFAOYSA-M indocyanine green Chemical compound [Na+].[O-]S(=O)(=O)CCCCN1C2=CC=C3C=CC=CC3=C2C(C)(C)C1=CC=CC=CC=CC1=[N+](CCCCS([O-])(=O)=O)C2=CC=C(C=CC=C3)C3=C2C1(C)C MOFVSTNWEDAEEK-UHFFFAOYSA-M 0.000 description 5
- 229960004657 indocyanine green Drugs 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000007547 defect Effects 0.000 description 3
- 230000035515 penetration Effects 0.000 description 3
- 239000000523 sample Substances 0.000 description 3
- 238000000149 argon plasma sintering Methods 0.000 description 2
- 238000002189 fluorescence spectrum Methods 0.000 description 2
- 239000007850 fluorescent dye Substances 0.000 description 2
- 238000010521 absorption reaction Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 210000003462 vein Anatomy 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0071—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence by measuring fluorescence emission
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Physics & Mathematics (AREA)
- Biophysics (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- General Health & Medical Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- Surgery (AREA)
- Molecular Biology (AREA)
- Medical Informatics (AREA)
- Pathology (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Evolutionary Computation (AREA)
- Fuzzy Systems (AREA)
- Signal Processing (AREA)
- Psychiatry (AREA)
- Physiology (AREA)
- Mathematical Physics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention belongs to the technical field of fluorescence imaging, and discloses a living body fluorescence imaging deblurring method based on deep learning, which utilizes the characteristic that a near-infrared two-zone fluorescence image has high resolution, and adopts a near-infrared one-zone fluorescence camera and a near-infrared two-zone fluorescence camera with a common visual field to simultaneously acquire the fluorescence image of a mouse; randomly dividing the collected living body fluorescence image into a training data set, a verification data set and a test data set according to a certain proportion; constructing a generation countermeasure network GAN for converting a near-infrared first-region fluorescence image into a near-infrared second-region fluorescence image by using a full gradient loss method, and training the acquired living body fluorescence image by using the constructed generation countermeasure network; and calculating the near-infrared first-zone fluorescence image by adopting the trained network to obtain the deblurred fluorescence image. The deblurred fluorescence image has the sensitivity of a near-infrared first-region fluorescence image and also has the definition similar to a near-infrared second-region fluorescence image.
Description
Technical Field
The invention belongs to the technical field of fluorescence imaging, and particularly relates to a living body fluorescence imaging deblurring method based on deep learning.
Background
At present, near-infrared fluorescence imaging mainly relies on a fluorescence probe labeling technology to track and observe change information of an interested area in biological tissues and cells, and has the advantages of rapidness, simplicity, convenience, no radiation damage, non-invasiveness, high sensitivity and the like.
The near-infrared fluorescence imaging can be divided into near-infrared first-zone fluorescence imaging with the wavelength of 700 nm-1000 nm and near-infrared second-zone fluorescence imaging with the wavelength of 1000 nm-1700 nm. The near-infrared first-zone imaging has the characteristics of low background noise and high signal intensity; but because of its light scattering effect is large, tissue autofluorescence and background interference are large, tissue penetration depth is small, imaging effect is fuzzy and resolution is low. The near-infrared two-region fluorescence imaging has the advantages of small scattering effect and absorption effect in biological tissues, deep penetration depth, small tissue autofluorescence and high signal-to-back ratio, clear effect and high resolution; however, because the intensity of the near-infrared two-zone fluorescence imaging signal is not high, some details of the imaging effect are not as good as those of the near-infrared one-zone fluorescence imaging. At present, the emission fluorescence spectrum of most fluorescent probes is in a near-infrared first region, and near-infrared second region fluorescence imaging cannot be carried out. Therefore, it is desirable to design a new deblurring method and system for in vivo fluorescence imaging to overcome the shortcomings of the prior art.
Through the above analysis, the problems and defects of the prior art are as follows:
(1) because the light scattering effect of near-infrared first-region imaging is large, the tissue autofluorescence and background interference are large, the tissue penetration depth is small, the imaging effect is fuzzy and the resolution is low.
(2) Because the intensity of the near-infrared two-zone fluorescence imaging signal is not high, some details of the imaging effect are not as good as those of the near-infrared one-zone fluorescence imaging.
(3) At present, the emission fluorescence spectrum of most fluorescent probes is in a near-infrared first region, and near-infrared second region fluorescence imaging cannot be carried out.
The difficulty in solving the above problems and defects is:
1. the first and second fluorescent cameras simultaneously image in the same field of view.
2. And (4) acquisition of a data set.
The significance of solving the problems and the defects is as follows:
1. two-zone fluorescence images can be obtained without the use of two-zone probes.
2. The fluorescent image with the sensitivity of the near-infrared first-region fluorescent image and the definition of the near-infrared second-region fluorescent image can be obtained.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a living body fluorescence imaging deblurring method based on deep learning and based on deep learning.
The invention is realized in such a way that a living body fluorescence imaging deblurring method based on deep learning comprises the following steps: the characteristic that the near-infrared two-zone fluorescence image has high resolution is utilized, and a near-infrared one-zone fluorescence camera and a near-infrared two-zone fluorescence camera with a common visual field are adopted to simultaneously acquire the fluorescence image of the mouse; randomly dividing the collected living body fluorescence image into a training data set, a verification data set and a test data set according to a certain proportion; constructing a generation countermeasure network GAN for converting a near-infrared first-region fluorescence image into a near-infrared second-region fluorescence image by a full-gradient loss method through a deep learning technology, and training the collected living body fluorescence image by adopting the constructed generation countermeasure network; and calculating the near-infrared first-zone fluorescence image by adopting the trained network to obtain a deblurred fluorescence image, wherein the deblurred fluorescence image has the sensitivity of the near-infrared first-zone fluorescence image and the definition of the near-infrared second-zone fluorescence image.
Further, the living body fluorescence imaging deblurring method based on the deep learning comprises the following steps:
step one, building a same-visual field near-infrared first-zone imaging system and a same-visual field near-infrared second-zone imaging system;
collecting a living body fluorescence image;
step three, constructing a data set;
step four, constructing and generating a countermeasure network by using a full gradient loss method;
and step five, obtaining the trained network and the deblurred near-infrared first-zone fluorescence image.
Further, the building of the same-visual field near-infrared first-zone and second-zone imaging system in the first step comprises the following steps:
the near-infrared first-zone fluorescence camera is used for acquiring a near-infrared first-zone fluorescence image; the optical filter is used for filtering optical signals below a 850nm waveband; the near-infrared two-zone fluorescence camera is used for collecting near-infrared two-zone fluorescence images; the optical filter is used for filtering optical signals below a 1300nm wave band; the beam splitting cube is used for simultaneously transmitting the fluorescence image to the near-infrared first-region fluorescence camera and the near-infrared second-region fluorescence camera; a convex lens with a focal length of 100 mm; a convex lens with a focal length of 100 mm; the lens is used for increasing the imaging visual field range; the reflecting mirror is used for reflecting the image of the mouse to the lens through a certain angle; an excitation light source for emitting excitation light; the computer is used for collecting fluorescence images formed by the near-infrared first-zone fluorescence camera and the near-infrared second-zone fluorescence camera; a gas anaesthesia device for anaesthetizing mice and providing oxygen.
Further, the acquiring of the living body fluorescence image in the second step includes:
starting an exciting light emitting device to emit exciting light with a certain wavelength and intensity to irradiate the mouse, wherein the ICG emits fluorescence; imaging by using the built in-vivo fluorescence imaging system with the same visual field of the near-infrared first region and the near-infrared second region; collecting a near-infrared first-region fluorescence image and a near-infrared second-region fluorescence image in a mouse body, and collecting the number of the required fluorescence images according to different parts.
The constructing the data set in the third step comprises the following steps:
randomly dividing the collected near-infrared first-zone living body fluorescence image and near-infrared second-zone living body fluorescence image of the same imaging target into a training data set, a verification data set and a test data set according to a certain proportion.
Further, the step four of constructing the generation countermeasure network by the full gradient loss method includes:
training the data set by a generation countermeasure network improved by a full gradient loss method; the gradient of the near-infrared first-region living body fluorescence image and the gradient of the near-infrared second-region living body fluorescence image are compared by minimizing the gradient of adjacent pixels of each pixel in the image, and then the gradient between the original image and the generated image is compared, so that a loss function is established, and feedback is provided for network training.
In a generation countermeasure network constructed by a full gradient loss method, a pre-trained VGG and ResNet model is used for calculating perception loss, the network is trained through a loss function with fixed hyper-parameters, and then the network is trained through a loss function with set hyper-parameters; the set hyper-parameters are adjusted according to loss change and the performance of a verification data set, and the set hyper-parameters comprise generator loss and peak signal-to-noise ratio obtained when each epoch is finished in the training process; the network carries out down sampling on the near-infrared first-region image through step convolution, meanwhile, the main characteristics are extracted through parameters of learning step convolution kernels, the image is zoomed through a ResNet and sub-pixel convolution layer structure, and finally, the network performance of each epoch is evaluated through a verification data set.
Further, the obtaining of the trained network and the deblurred near-infrared first-zone fluorescence image in the fifth step includes:
and predicting the blurred living body fluorescence image by using the trained generation countermeasure network to generate a high-definition deblurred living body fluorescence image.
And deblurring the near-infrared first-region fluorescence image through a trained algorithm network to obtain a deblurred fluorescence image, wherein the deblurred fluorescence image has the sensitivity of the near-infrared first-region fluorescence image and the definition of the near-infrared second-region fluorescence image.
Another object of the present invention is to provide a living body fluorescence imaging deblurring system applying the living body fluorescence imaging deblurring method based on deep learning, the living body fluorescence imaging deblurring system comprising:
the imaging system building module is used for building a same-visual field near infrared first-zone imaging system and a same-visual field near infrared second-zone imaging system;
the living body fluorescence image acquisition module is used for acquiring a near-infrared first-zone living body fluorescence image and a near-infrared second-zone living body fluorescence image;
the data set construction module is used for randomly dividing the near-infrared first-zone living body fluorescence image and the near-infrared second-zone living body fluorescence image into a training data set, a verification data set and a test data set according to a certain proportion;
the generation countermeasure network construction module is used for constructing a generation countermeasure network by utilizing a full gradient loss method;
the generation confrontation network training module is used for training the constructed data set by adopting a full-gradient loss method to improve a generation confrontation network training method so as to obtain a trained network;
and the network deblurring module is used for deblurring the near-infrared first-zone fluorescent image through the trained algorithm network to obtain the deblurred near-infrared first-zone fluorescent image.
It is a further object of the invention to provide a computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of:
the characteristic that the near-infrared two-zone fluorescence image has high resolution is utilized, and a near-infrared one-zone fluorescence camera and a near-infrared two-zone fluorescence camera with a common visual field are adopted to simultaneously acquire the fluorescence image of the mouse; randomly dividing the collected living body fluorescence image into a training data set, a verification data set and a test data set according to a certain proportion; constructing a generation countermeasure network GAN for converting a near-infrared first-region fluorescence image into a near-infrared second-region fluorescence image by a full-gradient loss method through a deep learning technology, and training the collected living body fluorescence image by adopting the constructed generation countermeasure network; and calculating the near-infrared first-zone fluorescence image by adopting the trained network to obtain a deblurred fluorescence image, wherein the deblurred fluorescence image has the sensitivity of the near-infrared first-zone fluorescence image and the definition of the near-infrared second-zone fluorescence image.
By combining all the technical schemes, the invention has the advantages and positive effects that: the living body fluorescence imaging deblurring method based on deep learning provided by the invention utilizes the characteristic that a near-infrared two-region fluorescence image has high resolution, constructs a generation countermeasure network for converting the near-infrared one-region fluorescence image into the near-infrared two-region fluorescence image through a deep learning technology, predicts the infrared one-region living body fluorescence imaging by utilizing the network to obtain the deblurring effect, and the deblurred fluorescence image has the sensitivity of the near-infrared one-region fluorescence image and also has the definition similar to the near-infrared two-region fluorescence image.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the embodiments of the present invention will be briefly described below, and it is obvious that the drawings described below are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flowchart of a method for deblurring in vivo fluorescence imaging according to an embodiment of the present invention.
FIG. 2 is a schematic diagram of a deblurring method for in-vivo fluorescence imaging according to an embodiment of the present invention.
FIG. 3 is a block diagram of a de-blurring system for fluorescence imaging in vivo according to an embodiment of the present invention;
in the figure: 1. an imaging system building module; 2. a living body fluorescence image acquisition module; 3. a data set construction module; 4. generating a confrontation network construction module; 5. generating a confrontation network training module; 6. and a network deblurring module.
FIG. 4 is a diagram of a co-field of view near infrared first and second zone imaging system provided by an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
In view of the problems in the prior art, the present invention provides a method, a system, a computer device and an intelligent terminal for deblurring in vivo fluorescence imaging, which are described in detail below with reference to the accompanying drawings.
As shown in fig. 1, the method for deblurring in vivo fluorescence imaging provided by the embodiment of the present invention includes the following steps:
s101, building a same-view near-infrared first-zone imaging system and a same-view near-infrared second-zone imaging system;
s102, collecting a living body fluorescence image;
s103, constructing a data set;
s104, constructing and generating a countermeasure network by using a full gradient loss method;
and S105, obtaining the trained network and the deblurred near-infrared first-zone fluorescence image.
The principle diagram of the in-vivo fluorescence imaging deblurring method provided by the embodiment of the invention is shown in FIG. 2.
As shown in fig. 3, the living body fluorescence imaging deblurring system provided by the embodiment of the present invention includes:
the imaging system building module 1 is used for building a same-visual field near infrared first-zone imaging system and a same-visual field near infrared second-zone imaging system;
the living body fluorescence image acquisition module 2 is used for acquiring a near-infrared first-region living body fluorescence image and a near-infrared second-region living body fluorescence image;
the data set construction module 3 is used for randomly dividing the near-infrared first-region living body fluorescence image and the near-infrared second-region living body fluorescence image into a training data set, a verification data set and a test data set according to a certain proportion;
the generation countermeasure network construction module 4 is used for constructing a generation countermeasure network by using a full gradient loss method;
the generation confrontation network training module 5 is used for training the constructed data set by adopting a full-gradient loss method to improve a generation confrontation network training method so as to obtain a trained network;
and the network deblurring module 6 is used for deblurring the near-infrared first-zone fluorescent image through the trained algorithm network to obtain the deblurred near-infrared first-zone fluorescent image.
The technical solution of the present invention is further described with reference to the following specific examples.
Example 1
The invention provides a living body fluorescence imaging deblurring method based on deep learning, and a deblurred fluorescence image has the sensitivity of a near-infrared first-region fluorescence image and also has the definition similar to a near-infrared second-region fluorescence image.
The living body fluorescence imaging deblurring method based on deep learning provided by the invention comprises the following contents:
(1) the self-built in-vivo fluorescence imaging system with the same visual field for the first near-infrared region and the second near-infrared region is adopted for imaging, and the imaging region and the imaging quantity can be adjusted according to requirements; and then randomly dividing the collected near-infrared first-region living body fluorescence image and near-infrared second-region living body fluorescence image into a training data set, a verification data set and a test data set according to a certain proportion.
(2) In the algorithm network, the gradient of a near-infrared first-region living body fluorescence image and the gradient of a near-infrared second-region living body fluorescence image are compared, and then the gradient between an original image and a generated image is compared, so that a loss function is established, and feedback is provided for network training.
(3) The method comprises the steps that a generation countermeasure network constructed by a full gradient loss method uses pre-trained VGG and ResNet models to calculate perception loss, the network is trained through a loss function with fixed super parameters, the parameters are adjusted according to loss change and performance of a verification data set, the loss of a generator and a peak signal-to-noise ratio are obtained when each epoch is finished in the training process, and finally the performance of the network of each epoch is evaluated through the verification data set.
(4) And calculating the near-infrared first-zone fluorescence image by adopting the trained network to obtain a deblurred fluorescence image, wherein the deblurred fluorescence image has the sensitivity of the near-infrared first-zone fluorescence image and also has the definition of the near-infrared second-zone fluorescence image.
Example 2
The invention adopts a self-built co-visual field near-infrared first-zone and second-zone imaging system to acquire and process images of a mouse, and the method comprises the following steps:
step 1: constructing a same-visual-field near-infrared one-zone and two-zone imaging system (see FIG. 4)
1. The near-infrared first-zone fluorescence camera is used for collecting a near-infrared first-zone fluorescence image; 2. an 850nm optical filter for filtering optical signals below 850 nm; 3. the near-infrared two-zone fluorescence camera is used for collecting near-infrared two-zone fluorescence images; 4. a 1300nm filter for filtering optical signals below a 1300nm wave band; 5. the beam splitting cube simultaneously transmits the fluorescence images to the near-infrared first-region fluorescence camera and the near-infrared second-region fluorescence camera; 6. a convex lens with a focal length of 100 mm; 7. a convex lens with a focal length of 100 mm; 8. the lens is used for increasing the imaging visual field range; 9. the reflector reflects the image of the mouse to the lens through a certain angle; 10. an excitation light source that emits excitation light; 11. the computer is used for collecting fluorescence images formed by the near-infrared first-zone fluorescence camera and the near-infrared second-zone fluorescence camera; 12. a gas anaesthesia device, anaesthetising the mice and providing oxygen.
Step 2: collecting a live fluorescence image
Mice were anesthetized and provided with oxygen using a gas anesthesia apparatus. Then injecting a probe ICG (indocyanine green) into the body of the mouse through a tail vein, starting an exciting light emitting device to emit exciting light with certain wavelength and intensity to irradiate the mouse, and then emitting fluorescence by the ICG (indocyanine green). Then, a self-built living body fluorescence imaging system with the same visual field of the first near-infrared region and the second near-infrared region is used for imaging; the method is used for collecting the near-infrared first-zone fluorescence image and the near-infrared second-zone fluorescence image in the mouse body, and the required number of the fluorescence images can be collected according to different parts.
And step 3: building a data set
Randomly dividing the collected near-infrared first-region living body fluorescence image and near-infrared second-region living body fluorescence image into a training data set, a verification data set and a test data set according to a certain proportion.
And 4, step 4: full-gradient loss method for constructing and generating countermeasure network
Training the data set by generating a countermeasure network improved by a full gradient loss method; in the algorithm network, the gradient of a near-infrared first-region living body fluorescence image and the gradient of a near-infrared second-region living body fluorescence image are compared, and then the gradient between an original image and a generated image is compared, so that a loss function is established, and feedback is provided for network training.
In the network, the sensing loss is calculated by using a pre-trained VGG (variable G) and ResNet model, and the network is trained by setting a loss function of a super parameter, wherein the set super parameter can be adjusted according to loss change and the performance of a verification data set. The network performs downsampling on the near-infrared first-region image through step convolution to reduce the size of the image and accelerate calculation, meanwhile, main features are extracted through parameters of a learning step convolution kernel, and the ResNet and a sub-pixel convolution layer structure are adopted to zoom the image to improve the performance of a training network.
And 5: obtaining a trained network
The trained generation countermeasure network can predict the blurred living body fluorescence image to generate a high-definition deblurred living body fluorescence image.
Step 6: obtaining the deblurred near-infrared one-zone fluorescence image
And deblurring the near-infrared first-region fluorescence image through a trained algorithm network to obtain a deblurred fluorescence image, wherein the deblurred fluorescence image has the sensitivity of the near-infrared first-region fluorescence image and the definition of the near-infrared second-region fluorescence image.
The above description is only for the purpose of illustrating the present invention and the appended claims are not to be construed as limiting the scope of the invention, which is intended to cover all modifications, equivalents and improvements that are within the spirit and scope of the invention as defined by the appended claims.
Claims (7)
1. A living body fluorescence imaging deblurring method based on deep learning is characterized in that the living body fluorescence imaging deblurring method based on deep learning utilizes the characteristic that a near-infrared two-zone fluorescence image has high resolution, and adopts a near-infrared one-zone fluorescence camera and a near-infrared two-zone fluorescence camera with a common visual field to simultaneously acquire the fluorescence image of a mouse; randomly dividing the collected living body fluorescence image into a training data set, a verification data set and a test data set according to a proportion; constructing a generation countermeasure network GAN for converting a near-infrared first-region fluorescence image into a near-infrared second-region fluorescence image by a full gradient loss method through deep learning, and training the collected living body fluorescence image by adopting the constructed generation countermeasure network; and calculating the near-infrared first-zone fluorescence image by adopting the trained network to obtain a deblurred fluorescence image, wherein the deblurred fluorescence image has the sensitivity of the near-infrared first-zone fluorescence image and the definition of the near-infrared second-zone fluorescence image.
2. The depth-learning-based in-vivo fluorescence imaging deblurring method of claim 1, wherein the depth-learning-based in-vivo fluorescence imaging deblurring method comprises the steps of:
step one, building a same-visual field near infrared first-zone imaging system and a same-visual field near infrared second-zone imaging system;
collecting a living body fluorescence image;
step three, constructing a data set;
step four, constructing and generating a countermeasure network by using a full gradient loss method;
and step five, obtaining the trained network and the deblurred near-infrared first-zone fluorescence image.
3. The in-vivo fluorescence imaging deblurring method based on deep learning of claim 2, wherein the building of the same-view near-infrared first-zone and second-zone imaging system in the first step comprises: the near-infrared first-zone fluorescence camera is used for acquiring a near-infrared first-zone fluorescence image; the optical filter is used for filtering optical signals below a 850nm waveband; the near-infrared two-zone fluorescence camera is used for collecting near-infrared two-zone fluorescence images; the optical filter is used for filtering optical signals below a 1300nm wave band; the beam splitting cube is used for simultaneously transmitting the fluorescence image to the near-infrared first-region fluorescence camera and the near-infrared second-region fluorescence camera; a convex lens with a focal length of 100 mm; a convex lens with a focal length of 100 mm; the lens is used for increasing the imaging visual field range; the reflecting mirror is used for reflecting the image of the mouse to the lens through a certain angle; an excitation light source for emitting excitation light; the computer is used for collecting fluorescence images formed by the near-infrared first-zone fluorescence camera and the near-infrared second-zone fluorescence camera; a gas anaesthesia device for anaesthetizing mice and providing oxygen.
4. The method for deblurring in vivo fluorescence imaging based on deep learning of claim 2, wherein the acquiring the in vivo fluorescence image in the second step comprises: starting an exciting light emitting device to emit exciting light with a certain wavelength and intensity to irradiate the mouse, wherein the ICG emits fluorescence; imaging by using the built in-vivo fluorescence imaging system with the same visual field of the near-infrared first region and the near-infrared second region; collecting a near-infrared first-region fluorescence image and a near-infrared second-region fluorescence image in a mouse body, and collecting the quantity of the required fluorescence images according to different parts;
the constructing the data set in the third step comprises the following steps: randomly dividing the collected near-infrared first-zone living body fluorescence image and near-infrared second-zone living body fluorescence image of the same imaging target into a training data set, a verification data set and a test data set according to a certain proportion.
5. The in-vivo fluorescence imaging deblurring method based on deep learning of claim 2, wherein the full gradient depletion method in the fourth step of constructing the generation countermeasure network comprises: training the data set by a generation countermeasure network improved by a full gradient loss method; the gradient of the near-infrared first-region living body fluorescence image and the gradient of the near-infrared second-region living body fluorescence image are compared by minimizing the gradient of adjacent pixels of each pixel in the image, and then the gradient between the original image and the generated image is compared, so that a loss function is established, and feedback is provided for network training;
in a generation countermeasure network constructed by a full gradient loss method, a pre-trained VGG and ResNet model is used for calculating perception loss, the network is trained through a loss function with fixed hyper-parameters, and then the network is trained through a loss function with set hyper-parameters; the set hyper-parameters are adjusted according to loss change and the performance of a verification data set, and the set hyper-parameters comprise generator loss and peak signal-to-noise ratio obtained when each epoch is finished in the training process; the network carries out down sampling on the near-infrared first-region image through step convolution, meanwhile, the main characteristics are extracted through parameters of learning step convolution kernels, the image is zoomed through a ResNet and sub-pixel convolution layer structure, and finally, the network performance of each epoch is evaluated through a verification data set.
6. The in-vivo fluorescence imaging deblurring method based on deep learning of claim 2, wherein the obtaining of the trained network and the deblurred near-infrared first-region fluorescence image in the fifth step comprises: predicting the blurred living body fluorescence image by using the trained generation countermeasure network to generate a high-definition deblurred living body fluorescence image;
and deblurring the near-infrared first-region fluorescence image through a trained algorithm network to obtain a deblurred fluorescence image, wherein the deblurred fluorescence image has the sensitivity of the near-infrared first-region fluorescence image and the definition of the near-infrared second-region fluorescence image.
7. A living body fluorescence imaging deblurring system for implementing the living body fluorescence imaging deblurring method based on the deep learning of any one of claims 1 to 6, wherein the living body fluorescence imaging deblurring system comprises:
the imaging system building module is used for building a same-visual field near infrared first-zone imaging system and a same-visual field near infrared second-zone imaging system;
the living body fluorescence image acquisition module is used for acquiring a near-infrared first-zone living body fluorescence image and a near-infrared second-zone living body fluorescence image;
the data set construction module is used for randomly dividing the near-infrared first-zone living body fluorescence image and the near-infrared second-zone living body fluorescence image into a training data set, a verification data set and a test data set according to a certain proportion;
the generation countermeasure network construction module is used for constructing a generation countermeasure network by utilizing a full gradient loss method;
the generation confrontation network training module is used for training the constructed data set by adopting a full-gradient loss method to improve a generation confrontation network training method so as to obtain a trained network;
and the network deblurring module is used for deblurring the near-infrared first-zone fluorescent image through the trained algorithm network to obtain the deblurred near-infrared first-zone fluorescent image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210182173.4A CN114587272B (en) | 2022-02-25 | 2022-02-25 | In-vivo fluorescence imaging deblurring method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210182173.4A CN114587272B (en) | 2022-02-25 | 2022-02-25 | In-vivo fluorescence imaging deblurring method based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114587272A true CN114587272A (en) | 2022-06-07 |
| CN114587272B CN114587272B (en) | 2024-07-02 |
Family
ID=81804317
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210182173.4A Active CN114587272B (en) | 2022-02-25 | 2022-02-25 | In-vivo fluorescence imaging deblurring method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114587272B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003164414A (en) * | 2001-11-29 | 2003-06-10 | Fuji Photo Film Co Ltd | Method and device for displaying fluoroscopic image |
| US20130109963A1 (en) * | 2011-10-31 | 2013-05-02 | The University Of Connecticut | Method and apparatus for medical imaging using combined near-infrared optical tomography, fluorescent tomography and ultrasound |
| CN107266929A (en) * | 2017-06-21 | 2017-10-20 | 四川大学 | One class is using Cyanine Dyes Fluorescence group as near infrared fluorescent dye of precursor skeleton structure and preparation method and application |
| CN109480776A (en) * | 2018-10-30 | 2019-03-19 | 中国科学院自动化研究所 | Near-infrared fluorescent surgical imaging systems and its application method |
| CN109776380A (en) * | 2019-03-12 | 2019-05-21 | 遵义医科大学 | It is applied in the bis- targeting near infrared fluorescent probe preparations of IR780 and tumour diagnosis and treatment |
| US20190328207A1 (en) * | 2016-08-08 | 2019-10-31 | Sony Corporation | Endoscope apparatus and control method of endoscope apparatus |
| CN110804434A (en) * | 2019-10-17 | 2020-02-18 | 西安电子科技大学 | Rare earth probe capable of identifying squamous cell lung carcinoma in targeted manner and preparation method and application thereof |
| US20210000437A1 (en) * | 2019-07-04 | 2021-01-07 | Institute Of Automation, Chinese Academy Of Sciences | Second near-infrared window / first near-infrared window dual-mode fluorescence tomography system and method |
| CN112274108A (en) * | 2020-08-25 | 2021-01-29 | 中国人民解放军军事科学院军事医学研究院 | CKIP-1-targeted fluorescent probe for detecting osteoporosis and application of probe in-vivo detection |
| WO2021246968A1 (en) * | 2020-06-05 | 2021-12-09 | National University Of Singapore | Deep fluorescence imaging by laser-scanning excitation and artificial neural network processing |
-
2022
- 2022-02-25 CN CN202210182173.4A patent/CN114587272B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003164414A (en) * | 2001-11-29 | 2003-06-10 | Fuji Photo Film Co Ltd | Method and device for displaying fluoroscopic image |
| US20130109963A1 (en) * | 2011-10-31 | 2013-05-02 | The University Of Connecticut | Method and apparatus for medical imaging using combined near-infrared optical tomography, fluorescent tomography and ultrasound |
| US20190328207A1 (en) * | 2016-08-08 | 2019-10-31 | Sony Corporation | Endoscope apparatus and control method of endoscope apparatus |
| CN107266929A (en) * | 2017-06-21 | 2017-10-20 | 四川大学 | One class is using Cyanine Dyes Fluorescence group as near infrared fluorescent dye of precursor skeleton structure and preparation method and application |
| CN109480776A (en) * | 2018-10-30 | 2019-03-19 | 中国科学院自动化研究所 | Near-infrared fluorescent surgical imaging systems and its application method |
| CN109776380A (en) * | 2019-03-12 | 2019-05-21 | 遵义医科大学 | It is applied in the bis- targeting near infrared fluorescent probe preparations of IR780 and tumour diagnosis and treatment |
| US20210000437A1 (en) * | 2019-07-04 | 2021-01-07 | Institute Of Automation, Chinese Academy Of Sciences | Second near-infrared window / first near-infrared window dual-mode fluorescence tomography system and method |
| CN110804434A (en) * | 2019-10-17 | 2020-02-18 | 西安电子科技大学 | Rare earth probe capable of identifying squamous cell lung carcinoma in targeted manner and preparation method and application thereof |
| WO2021246968A1 (en) * | 2020-06-05 | 2021-12-09 | National University Of Singapore | Deep fluorescence imaging by laser-scanning excitation and artificial neural network processing |
| CN112274108A (en) * | 2020-08-25 | 2021-01-29 | 中国人民解放军军事科学院军事医学研究院 | CKIP-1-targeted fluorescent probe for detecting osteoporosis and application of probe in-vivo detection |
Non-Patent Citations (2)
| Title |
|---|
| 乔要宾等: "基于荧光技术的时域近红外光学成像", 《工程热物理学报》, vol. 37, no. 8, 31 August 2016 (2016-08-31), pages 1701 - 1704 * |
| 马喜波等: "多模态分子影像对肝癌进展和 血管生成的研究", 《生物物理学报》, vol. 27, no. 4, 30 April 2011 (2011-04-30), pages 355 - 364 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114587272B (en) | 2024-07-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yang et al. | Review of deep learning for photoacoustic imaging | |
| Vu et al. | A generative adversarial network for artifact removal in photoacoustic computed tomography with a linear-array transducer | |
| Deng et al. | Deep learning in photoacoustic imaging: a review | |
| US10064584B2 (en) | Combined x-ray and optical tomographic imaging system | |
| US20080103390A1 (en) | Apparatus and methods for fluorescence guided surgery | |
| US20100155599A1 (en) | Hand-Held Optical Probe Based Imaging System with 3D Tracking Facilities | |
| Rajendran et al. | Photoacoustic imaging aided with deep learning: a review | |
| CA2809798A1 (en) | Systems and methods for tomographic imaging in diffuse media using a hybrid inversion technique | |
| CN104055489B (en) | A kind of blood vessel imaging device | |
| Zhou et al. | Photoacoustic microscopy with sparse data by convolutional neural networks | |
| CN104305957B (en) | Wear-type molecular image navigation system | |
| Cheng et al. | High-resolution photoacoustic microscopy with deep penetration through learning | |
| Lu et al. | LV‐GAN: A deep learning approach for limited‐view optoacoustic imaging based on hybrid datasets | |
| CN102028477A (en) | Device and method for measuring blood oxygen saturation of eye fundus retina | |
| CN114209278B (en) | Deep learning skin disease diagnosis system based on optical coherence tomography | |
| Yang et al. | Deep learning for photoacoustic imaging: a survey | |
| Moustakidis et al. | Fully automated identification of skin morphology in raster‐scan optoacoustic mesoscopy using artificial intelligence | |
| CN109924949A (en) | A kind of near infrared spectrum tomography rebuilding method based on convolutional neural networks | |
| Wu et al. | System-level optimization in spectroscopic photoacoustic imaging of prostate cancer | |
| Wu et al. | Learned end-to-end high-resolution lensless fiber imaging towards real-time cancer diagnosis | |
| CN103815929B (en) | Subject information acquisition device | |
| CN116503258B (en) | Super-resolution computing imaging method, device, electronic equipment and storage medium | |
| Hu et al. | Full-angle optical imaging of near-infrared fluorescent probes implanted in small animals | |
| CN114587272A (en) | In-vivo fluorescence imaging deblurring method based on deep learning | |
| Sun et al. | 3D photoacoustic tomography system based on full-view illumination and ultrasound detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |