CN111445428A - Method and device for enhancing biological living body blood vessel imaging data based on unsupervised learning - Google Patents

Method and device for enhancing biological living body blood vessel imaging data based on unsupervised learning Download PDF

Info

Publication number
CN111445428A
CN111445428A CN202010145022.2A CN202010145022A CN111445428A CN 111445428 A CN111445428 A CN 111445428A CN 202010145022 A CN202010145022 A CN 202010145022A CN 111445428 A CN111445428 A CN 111445428A
Authority
CN
China
Prior art keywords
blood vessel
imaging
blood vessels
living body
unsupervised learning
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010145022.2A
Other languages
Chinese (zh)
Other versions
CN111445428B (en
Inventor
方璐
马少华
王勇
季梦奇
蒋胜威
王好谦
戴琼海
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen International Graduate School of Tsinghua University
Original Assignee
Shenzhen International Graduate School of Tsinghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen International Graduate School of Tsinghua University filed Critical Shenzhen International Graduate School of Tsinghua University
Priority to CN202010145022.2A priority Critical patent/CN111445428B/en
Publication of CN111445428A publication Critical patent/CN111445428A/en
Application granted granted Critical
Publication of CN111445428B publication Critical patent/CN111445428B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/50Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/088Non-supervised learning, e.g. competitive learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10116X-ray image
    • G06T2207/10121Fluoroscopy
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30101Blood vessel; Artery; Vein; Vascular

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Eye Examination Apparatus (AREA)
  • Image Analysis (AREA)

Abstract

A method and a device for enhancing biological living body blood vessel imaging data based on unsupervised learning are disclosed, the method comprises the following steps: respectively carrying out living body dynamic imaging on the blood vessels of the living body of the organism by utilizing a non-invasive fluorescence imaging or X-ray angiography imaging technology; by means of domain migration, a blood vessel binary image of an open-source fundus retina is used as a target domain, and a part with a blood vessel representation is extracted from an obtained image in an unsupervised learning mode; and performing visual enhancement on the extracted part with the vessel characterization. The invention can realize the extraction and the enhancement of the blood vessel characteristics from the fuzzy and disordered biological blood vessel images without the need of calibrated true value data and large-scale data sets, thereby being beneficial to the diagnosis of blood vessels and blood vessel related diseases in subsequent application.

Description

Method and device for enhancing biological living body blood vessel imaging data based on unsupervised learning
Technical Field
The invention relates to the field of living organism imaging, in particular to a method for enhancing living organism blood vessel imaging data based on unsupervised learning.
Background
In vivo biological imaging is of great importance for the analysis of the health status of an organism. The topological morphology of blood vessels, the blood flow rate and the like are closely related to Alzheimer's disease, cardiovascular and cerebrovascular diseases, intestinal diseases and the like. The in-vivo imaging of the blood vessel can be used for real-time monitoring and early prevention of blood vessel diseases, and has very important significance for pathological research and accurate diagnosis.
Biological living body imaging can be performed based on multiple modalities such as single photon fluorescence imaging, multiphoton fluorescence imaging, CT, MRI, photoacoustic imaging and the like. Biological tissues such as skin, internal organs and the like scatter very strongly in the visible light band, so that the depth of single photon fluorescence imaging is severely limited. CT uses X-rays to penetrate an animal body, three information in an interested area is reconstructed through calculation based on different tissues with different X-ray absorption capacities, but the radiation effect of the X-rays is harmful to the animal and can induce cancers. The methods such as multiphoton fluorescence imaging, MRI and photoacoustic imaging have strong anti-scattering capability, but the equipment cost is high and complicated. In addition, the living organism imaging data has the characteristics of large individual difference, low signal to noise ratio and the like. Meanwhile, due to respiration and movement of the living body, useful signals are doped in background signals and noise which move irregularly, so that the quality of data acquired by biological living body imaging is low in general, and the subsequent analysis or disease diagnosis is not facilitated.
In order to solve the problem of poor quality of biological imaging data, two main research methods exist, the first method is to consider the extraction process of useful signals as a deconvolution operation. The method considers that the fuzzy image shot by the imaging system is the result obtained by performing space convolution on a clear object to be imaged and a fuzzy kernel function and then overlapping the clear object to be imaged and noise, so that the clear image can be calculated by a deconvolution algorithm as long as the fuzzy kernel function is modeled. The method needs to calibrate an imaging system or combine a priori knowledge to obtain a rough model of a fuzzy kernel function and then perform parameter adjustment, which is difficult to implement for wide-field in-vivo imaging because the individual difference of organisms is large and the scattering of light by biological tissues has anisotropic characteristics, while the modeling and calibration based method needs the optical characteristics of the object or tissue to be imaged to be kept approximately constant.
The second method is a supervised learning method based on a deep neural network, and a large amount of paired calibration data is used for learning the relation between a fuzzy imaging result and clear raw data. Because the amount of biological data is usually not rich enough, researchers train a model capable of extracting various features in an image, such as edges, forms and the like, on a natural image data set in a transfer learning mode, and then continue training and testing a deep neural network on a small-scale biological data set. However, a small-scale data set also requires hundreds to thousands of pieces of calibrated and high-quality data, which are difficult to obtain for many tasks of biological in-vivo imaging experiments in view of the individual differences of organisms and the principle that in-vivo imaging causes the least possible damage to organisms. Learning and training of deep neural networks on biological imagery data with insufficiently large or even no known data sets remains a challenging problem.
Disclosure of Invention
The main purpose of the present invention is to overcome the above technical defects, and to provide a method for enhancing blood vessel imaging data of a living organism based on unsupervised learning.
In order to achieve the purpose, the invention adopts the following technical scheme:
a method for enhancing biological living body blood vessel imaging data based on unsupervised learning comprises the following steps:
a1, respectively carrying out living body dynamic imaging on living body blood vessels of a living body by utilizing a non-invasive fluorescence imaging or X-ray angiography imaging technology;
a2, extracting a part with a blood vessel representation from the image obtained in the step A1 in an unsupervised learning mode by using the blood vessel binary image of the eye fundus retina of an open source as a target domain in a domain transfer mode;
and A3, performing visual enhancement on the part with the vessel characteristics extracted in the step A2.
Further:
in step a1, a non-invasive fluorescence imaging technique is used to perform living body dynamic imaging on the brain blood vessel and/or intestine blood vessel of the mouse, or an X-ray angiography imaging technique is used to perform living body dynamic imaging on the blood vessel in the human body.
In step a1, the performing in vivo dynamic imaging on the cerebral vessels of the mouse by using non-invasive fluorescence imaging includes:
in the near infrared band, the living body dynamic imaging is carried out on the cerebral blood vessels of the mouse by penetrating the skull and the cerebral skin of the mouse.
In step a1, the performing in vivo dynamic imaging on the intestinal blood vessel of the mouse by using non-invasive fluorescence imaging includes:
in the near infrared band, the living body dynamic imaging is carried out on the intestinal blood vessels of the mice through the abdominal skin and peritoneum of the mice.
In step a1, the in-vivo blood vessel data of the original modality of the X-ray angiography imaging is also preprocessed to eliminate the influence of shaking of the internal organs and blood vessels of the human body.
And training a neural network model by using the blood vessel characteristic information obtained in the step A2, and performing multi-scale test fusion on the to-be-reconstructed picture by using the trained model.
An enhancement device for blood vessel imaging data of a living organism based on unsupervised learning, comprising:
an imaging device configured for: respectively carrying out living body dynamic imaging on the blood vessels of the living body of the organism by utilizing a non-invasive fluorescence imaging or X-ray angiography imaging technology;
a processor configured for: by means of domain migration, a blood vessel binary image of an open-source fundus retina is used as a target domain, and a part with a blood vessel representation is extracted from an image obtained by an imaging device in an unsupervised learning mode; and performing visual enhancement on the extracted part with the vessel characterization.
Further:
and the processor trains a neural network model by using the obtained blood vessel characteristic information, and performs multi-scale test fusion on the to-be-reconstructed picture by using the trained model.
The invention has the following beneficial effects:
the invention provides a method and a device for enhancing biological living body blood vessel imaging data based on unsupervised learning, which can get rid of the dependence of training of biological images containing blood vessel information on a large-scale data set, accurately and stably extract the blood vessel information, and can be used for subsequent quantitative analysis and enhanced diagnosis.
The invention is based on the idea of domain migration, and uses an open-source fundus retina blood vessel data set as a target domain, extracts a vascular structure from living body imaging data of a living body blood vessel and analyzes the vascular structure. The vascular structure, as large as arteriovenous and as small as capillary, has morphological commonality, and the implicit characterization of vascular sharing can be obtained by mining and learning through a deep neural network without limitation of complexity of the size of an organ part or a blood vessel. The invention can realize the extraction and the enhancement of the blood vessel characteristics from the fuzzy and disordered biological blood vessel images without the need of calibrated true value data and large-scale data sets, thereby being beneficial to the diagnosis of blood vessels and blood vessel related diseases in subsequent application.
Drawings
Fig. 1 is a flowchart of an application example of the method for enhancing blood vessel imaging data of a living organism based on unsupervised learning according to the present invention;
FIG. 2 is a schematic diagram of a non-invasive mouse in-vivo fluorescence imaging device and a thrombus model mouse in a preparation manner according to an embodiment of the present invention;
FIG. 3 is a block diagram of a deep neural network for unsupervised learning according to an embodiment of the present invention;
FIG. 4 is a graph comparing the original data of the human X-ray angiography and the result outputted by the deep neural network to which the embodiment of the present invention is applied.
Fig. 5 is a schematic diagram of automatic detection of bleeding points in X-ray angiography imaging using the neural network of the present invention.
Detailed Description
The embodiments of the present invention will be described in detail below. It should be emphasized that the following description is merely exemplary in nature and is not intended to limit the scope of the invention or its application.
The embodiment of the invention provides a method for enhancing biological living body blood vessel imaging data based on unsupervised learning, which comprises the following steps:
a1, respectively carrying out living body dynamic imaging on living body blood vessels of a living body by utilizing a non-invasive fluorescence imaging or X-ray angiography imaging technology;
a2, extracting a part with a blood vessel representation from the image obtained in the step A1 in an unsupervised learning mode by using the blood vessel binary image of the eye fundus retina of an open source as a target domain in a domain transfer mode;
and A3, performing visual enhancement on the part with the vessel characteristics extracted in the step A2.
In some embodiments, step a1, in vivo dynamic imaging of mouse cerebral blood vessels and/or mouse intestinal blood vessels is performed using a non-invasive fluorescence imaging technique. For example, in the near infrared band, the blood vessels in the brain of a mouse are imaged dynamically in vivo by penetrating the skull and the brain skin of the mouse. In the near infrared band, the living body dynamic imaging is carried out on the intestinal blood vessels of the mice through the abdominal skin and peritoneum of the mice.
In some embodiments, step a1, a live-action imaging technique is used to perform live-action imaging of a blood vessel in the human body. Preferably, when the living body dynamic imaging is performed on the blood vessel in the human body, the blood vessel data in the original mode of the X-ray angiography imaging in the human body is also preprocessed, so as to eliminate the influence caused by shaking of internal organs and blood vessels in the human body.
In a preferred embodiment, the method further comprises the steps of training a neural network model by using the blood vessel characteristic information obtained in the step a2, and performing multi-scale test fusion on the to-be-reconstructed picture by using the trained model.
Embodiments of the present invention also provide an enhancement device for blood vessel imaging data of a living organism based on unsupervised learning, including an imaging device and a processor, wherein the imaging device is configured to: respectively carrying out living body dynamic imaging on the blood vessels of the living body of the organism by utilizing a non-invasive fluorescence imaging or X-ray angiography imaging technology; the processor is configured for: by means of domain migration, a blood vessel binary image of an open-source fundus retina is used as a target domain, and a part with a blood vessel representation is extracted from an image obtained by an imaging device in an unsupervised learning mode; and performing visual enhancement on the extracted part with the vessel characterization. Preferably, the processor trains a neural network model by using the obtained blood vessel characteristic information, and performs multi-scale test fusion on the image to be reconstructed by using the trained model.
The method and the device for enhancing the blood vessel imaging data of the living organism based on the unsupervised learning can get rid of the dependence of the training of the biological image containing the blood vessel information on a large-scale data set, and accurately and stably extract the blood vessel information, so that the subsequent quantitative analysis and the enhanced diagnosis can be carried out. The invention is based on the idea of domain migration, and uses an open-source fundus retina blood vessel data set as a target domain, extracts a vascular structure from living body imaging data of a living body blood vessel and analyzes the vascular structure. The vascular structure, as large as arteriovenous and as small as capillary, has morphological commonality, and the implicit characterization of vascular sharing can be obtained by mining and learning through a deep neural network without limitation of complexity of the size of an organ part or a blood vessel. The invention can realize the extraction and the enhancement of the blood vessel characteristics from the fuzzy and disordered biological blood vessel images without the need of calibrated true value data and large-scale data sets, thereby being beneficial to the subsequent diagnosis of blood vessels and blood vessel related diseases.
Application example
With reference to fig. 1-5, specific applications in disease diagnosis by means of various embodiments of the invention are further described below.
A method for applying an embodiment of the invention comprises the following steps:
respectively carrying out living body dynamic imaging on a cerebral blood vessel of a mouse, an intestinal blood vessel of the mouse and a blood vessel in a human body by utilizing a non-invasive fluorescence imaging and X-ray angiography imaging technology;
by means of domain migration, a blood vessel binary image of an open-source fundus retina is used as a target domain, and a part with blood vessel representation is extracted from the image in an unsupervised learning mode; and
and performing visual enhancement on the blood vessel characteristic information extracted from the blurred image and performing quantitative analysis, such as blood flow rate, blood vessel morphological information, bleeding point judgment and the like.
The method can realize the extraction and the enhancement of the blood vessel characteristics from the fuzzy and disordered biological blood vessel images without calibrated truth value data and large-scale data sets, thereby assisting in the diagnosis of blood vessels and blood vessel related diseases.
Furthermore, living body dynamic imaging can be carried out on blood vessels of the brain of the mouse through the skull and the brain skin of the mouse in a near infrared band; and a mouse brain thrombus model is constructed, and the function of enhancing disease diagnosis by an algorithm is reflected by comparing the brain blood vessel imaging results of a healthy mouse and a thrombus mouse.
Furthermore, living body dynamic imaging can be carried out on the intestinal blood vessels of the mice through the abdominal skin and peritoneum of the mice in the near infrared band; and a mouse colitis model is constructed, and the intestinal vascular imaging results of a healthy mouse and an enteritis mouse are compared to reflect the function of enhancing disease diagnosis by an algorithm.
Furthermore, the trained model can be used for carrying out multi-scale test fusion on the picture to be reconstructed, and meanwhile, quantitative pathological analysis and diagnosis, such as blood flow velocity extraction, vascular morphology information extraction and the like, can be carried out by combining the originally acquired data.
Further, automatic detection and identification of bleeding points in an angiographic image can be realized. In which a data set of angiographic bleeding points is constructed using an angiographic image sequence of 15 patients, and for each image, a real bleeding region (true label) and 2-3 regions (false labels) that are normal but have a morphology similar to the bleeding points, such as vessel corners, are marked using rectangular boxes.
By training the classification network, whether a certain area in the image is a real bleeding point or not can be judged, and a probability value is given for reference judgment of a doctor or an observer.
Fig. 1 is a flowchart of a specific application of the method for enhancing blood vessel imaging data of a living organism based on unsupervised learning according to an embodiment of the present invention, which comprises the following steps:
in step S101, a living body dynamic imaging is performed on the cerebral blood vessels of the mouse, the intestinal blood vessels of the mouse, and the blood vessels of the human body respectively by using the non-invasive fluorescence imaging and the X-ray angiography imaging techniques.
It will be appreciated that non-invasive fluorescence imaging and X-ray angiography imaging techniques are performed to acquire biological tissue scattered, noisy live biological vessel data for subsequent algorithmic processing and disease analysis.
Specifically, the data acquisition phase comprises non-invasive mouse brain blood vessel fluorescence imaging, non-invasive mouse intestine blood vessel fluorescence imaging and human body X-ray angiography imaging, and comprises the following steps:
non-invasive mouse brain vessel fluorescence imaging: as shown on the left of fig. 2, the experimental mice were anesthetized with isoflurane gas, fixed head up on the experimental platform, and the hair on the mouse head was removed using depilatory cream. 785nm near infrared laser light is collimated and then covers the head area of the mouse, the power is about 29mW/cm2, and the power is lower than the safe light intensity threshold value which can be borne by the skin of the animal under the wavelength. After preparation, an aqueous solution of the fluorescent agent indocyanine green was injected intravenously from the mouse tail. Indocyanine green is a fluorescent agent approved by the U.S. food and drug administration, has no toxicity to organisms within a safe dose range, and can be used for clinical surgery. Due to the fluorescence effect, indocyanine green absorbs photons in the 780-810nm band and emits photons in the 800-900nm band. The photosensitive array in the industrial black-and-white camera can work in the wave band range smaller than 900nm, so that the fluorescent data acquisition can be carried out by using the industrial black-and-white camera. The camera was first connected to a lens with a focal length of 25mm, and then focused on the mouse brain skin through a band pass filter of 810 and 890nm, and the whole process of the fluorescent agent from the tail vein through the blood circulation to the cerebral vessels was recorded.
The size of the captured image was 2000 x 2000 pixels, and the frame rate was 25 frames per second. The signal-to-noise ratio of the image sequence obtained by shooting is low, because the originally thin blood vessel signals are blurred after being transmitted outside the body due to the scattering effect of the scalp and the skull on light. In addition, more noise exists, such as texture on the mouse brain skin, background signals with uneven brightness, noise introduced by the camera, and the like.
In particular, by comparing the difference of fluorescence dynamic images of a healthy mouse and a thrombus model mouse, the generalizability and the medical value of the method are verified. The manufacturing process of the thrombus model mouse is as follows: as shown in the right part of the figure 2, the hair on the head of the mouse is removed, the rose bengal solution is injected into the body of the mouse from the tail vein, and meanwhile, the small-area part of the head of the mouse is irradiated by green laser with the wavelength of 532nm for 3-5 minutes to induce the rose bengal molecules to generate photochemical reaction, so that the cerebral vascular embolism model can be constructed. 24 hours after the thrombus model mouse is successfully prepared, the mouse is subjected to the non-invasive brain blood vessel fluorescence imaging operation.
Non-invasive mouse intestinal vessel fluorescence imaging: the experimental conditions were almost identical to non-invasive mouse brain vessel fluorescence imaging, except for the following details: the mice for the experiment are anesthetized by isoflurane gas, the abdomen of the mice is upward and fixed on an experiment platform, and the hairs on the abdomen of the mice are removed by using depilatory cream; the size of the captured image was 2600 x 2600 pixels, which is larger than the number of pixels needed for the brain, since the area of the mouse abdomen is larger than the brain area, requiring a larger field of view to cover the entire abdominal area. The signal-to-noise ratio of the fluorescence image obtained by shooting is low, because the scattering effect of the abdominal skin and the peritoneum on light makes the originally thin blood vessels become blurred after being transmitted to the outside of the body, and meanwhile, more noises exist, such as the peristalsis of the intestine of a mouse, background signals with uneven brightness, noise introduced by a camera and the like.
In particular, the proposed vascular extraction algorithm can enhance the diagnosis by constructing a mouse certificate for enteritis models. The enteritis model mouse is prepared by the following steps: healthy mice were divided into 4 groups of 5-6 mice each, of which three groups were fed with 5% strength aqueous dextran sulfate (DSS) solution for 2 days, 4 days and 6 days, respectively, and one group was fed with double distilled water as a control. The dextran sodium sulfate aqueous solution can stably induce colitis, and each mouse carries out the non-invasive in-vivo imaging operation of the intestinal blood vessel after drinking the DSS solution for the corresponding days to obtain a fuzzy and noisy image of the intestinal blood vessel.
Human X-ray angiography imaging: the original data is obtained from Beijing 301 hospital, iodixanol is used as developer, and X-ray can penetrate human body but cannot penetrate the developer, so that the developer is injected into the region to be examined in the X-ray imaging process, and the image result of blood vessels can be obtained by a digital subtraction method. Due to the movement of organs and blood vessels caused by heartbeat and human body movement, the blood vessel image obtained by using the digital subtraction method contains more noise and error information, thereby influencing the accuracy of diagnosis. In order to enhance the blood vessel signal and remove noise, the data of the original modality (before digital subtraction) of angiography is preprocessed, so that the influence caused by shaking of internal organs and blood vessels of a human body is eliminated, and the accuracy and the effect of subsequent processing based on a deep learning algorithm are improved.
In step S102, a portion having a blood vessel representation is extracted from the blurred blood vessel image in an unsupervised learning manner by way of domain migration using the blood vessel binary image of the fundus retina of the open source as a target domain.
It can be understood that to extract a clear structure from the blurred biological blood vessel image is equivalent to moving the image from the initial domain where the blurred blood vessel image is located to the target domain where the clear blood vessel image is located. Clear vascular structures, whether different in size or in different organ sites, share common features of vessel morphology, such as vessel bifurcation, continuous variation in vessel width, etc. The vascular structure of the tissues of the brain, the intestine, etc. and the retinal vascular structure may be considered to be in the same domain, i.e., the target domain where the clear blood vessel image is located. Therefore, the idea of domain migration can be utilized, so that the deep neural network learns the common representation, and the fuzzy biological blood vessel image is subjected to a blood vessel extraction task.
Specifically, the training of the deep network comprises the loading of data and the structure of the deep neural network, and specifically comprises the following steps: loading of data: the fuzzy biological blood vessel image is from fluorescent images of the brain and the intestine of 3-4 mice, the clear blood vessel image is from a black-white binary image of fundus retina blood vessels which are calibrated by hand and are from 10 open sources, the used training data is small in quantity and is easy to obtain. To improve the generalization performance of the model, adding data enhancement comprises: random cropping, rotation and inversion.
The data obtained in the above steps are input into a deep neural network, and the structure diagram of the network is shown in fig. 3. Firstly, a domain adaptive neural network is utilized to mine features similar to a clear retinal blood vessel image as much as possible from a fuzzy living organism blood vessel image, so that the fuzzy blood vessel image is divided into two parts, namely fuzzy background signal, noise and other non-blood vessel information and clear blood vessel information. In order to enable the deep neural network to accurately distinguish the blood vessel information from the non-blood vessel information and more accurately complete the blood vessel extraction task, the deep neural network adopts a circulation architecture, namely, the algorithm secondarily verifies the previously extracted result, thereby improving the robustness and the accuracy of the algorithm.
Specifically, non-blood vessel information extracted from a blurred biological blood vessel image is superposed with a clear retinal blood vessel image to generate a false blurred retinal blood vessel image, and then a neural network extracts clear blood vessel information and non-blood vessel information from the artificially synthesized blurred image. Adding a cycle consistency loss function, calculating the difference between clear blood vessel information extracted from the artificially synthesized blurred image and the original clear blood vessel image used for generating the blurred image, and enabling the difference to be as small as possible, so that the neural network can accurately distinguish the blood vessel information from the non-blood vessel information. Similarly, the difference between the original blurred image and the artificially generated false blurred image is calculated and made as small as possible, ensuring that the non-vessel information is not extracted as vessel information by the neural network.
In step S103, the blood vessel feature information extracted from the blurred image is visually enhanced and quantitatively analyzed, such as blood flow rate, blood vessel morphology information, bleeding point determination, and the like.
Specifically, the data processing stage comprises non-invasive mouse brain blood vessel fluorescence imaging, non-invasive mouse intestine blood vessel fluorescence imaging and blood vessel extraction and quantitative analysis of human body X-ray angiography imaging data, and comprises the following steps:
non-invasive mouse brain vessel fluorescence imaging: and extracting blood vessels from the blurred brain blood vessel image, analyzing the morphological structure of the blood vessels, and inspecting the relationship between the morphological structure of the blood vessels and the blood flow speed and the position before the brain thrombus.
Further, in an application example, the signal-to-noise ratio of the fluorescence image obtained by shooting is low, because the thin blood vessels originally look blurred from the outside due to the scattering effect of the brain skin and the skull on light, and meanwhile, more noise exists, such as the texture of the mouse brain skin, the background signal with uneven brightness, the noise introduced by the camera, and the like. The image acquired by the experiment is used as input, and the neural network trained in the above way is used for testing, so that the blood vessel information contained in the image is accurately extracted.
Further, by calculating the change speed of the fluorescence intensity of the position in the fluorescence map corresponding to the extracted position of the blood vessel, the change of the blood flow velocity at different blood vessel positions can be quantitatively analyzed. By comparing the healthy mouse with the mouse of the brain thrombus model, the conclusion that the blood vessels at the brain thrombus model are obviously reduced or even completely disappear and the blood flow speed at each part of the brain is obviously reduced can be obtained, and the conclusion is identical with the medical performance of thrombus.
Non-invasive mouse intestinal vessel fluorescence imaging: blood vessels are extracted from the blurred intestinal blood vessel image, the morphological structure of the blood vessels is analyzed, and the relationship between the morphological structure of the blood vessels and the number of days for taking medicine and the severity of enteritis is examined.
Further, in the application example, the signal-to-noise ratio of the fluorescence image obtained by shooting is low, because the scattering effect of the abdominal skin and peritoneum on light makes the originally thin blood vessel look blurred from the outside, and meanwhile, more noise exists, such as the peristalsis of the intestine of the mouse, background signals with uneven brightness, noise introduced by the camera, and the like. The image acquired by the experiment is used as input, and the neural network trained in the above way is used for testing, so that the blood vessel information contained in the image is accurately extracted.
Further, by calculating the connectivity and average length of all blood vessels that each mouse shines in the fluorescence image using morphological operations in digital image processing, it can be concluded that the connectivity of all blood vessels becomes lower and the average length decreases as the number of days for drinking DSS solution increases, and that the blood vessels in the graph become fine and messy. This conclusion was not found in previous reports and the mechanism of DSS-induced colitis is not clear. The proposed algorithm not only extracts the intestinal blood vessels from the fuzzy biological image, but also has great revelation effect on the pathogenic mechanism of DSS and the early symptom development of enteritis.
Human X-ray angiography imaging: in an application example, preprocessed original modality X-ray angiography imaging data is input into a neural network to perform characterization extraction of blood vessel features. In order to enhance the small and thin blood vessels which are not easy to observe in the image, the original image is subjected to multi-scale processing and fusion to obtain a result image which has high contrast, clear blood vessel and bleeding area and is obviously enhanced compared with the original image, so that the image can be analyzed and diagnosed more clearly, as shown in fig. 4. Experiments prove the feasibility of extracting the blood vessel information in the biological living body imaging data by using the blood vessel data of the retina.
The multi-scale processing and fusion of the original image refers to scaling the original image, for example, amplifying the original image by 2 times or 4 times, and then using the scaled original image as input, and performing fusion after obtaining an output result through a trained neural network model. In this way, the thickness degree of the blood vessel width in the acquired blurred blood vessel image is changed greatly, and the thickness of the blood vessel width in the black and white image of the fundus retinal blood vessel used for training is consistent, so that the trained model directly processes the original image, and the output result lacks information of a part of small blood vessels. The multi-scale processing fusion is carried out, so that the blood vessels with different thicknesses can be reconstructed, the accuracy and the visual quality of the algorithm are improved, and the observation and the subsequent analysis are facilitated.
In particular, automated detection of bleeding points was performed on X-ray angiography imaging data using a classification neural network, as shown in fig. 5. For each patient image sequence, 4-6 frames are selected, and each frame is marked with a real bleeding point area and 2-3 areas similar to the bleeding point in shape but healthy and normal. The automated detection algorithm was trained on data from 15 patients and tested on data from 20 patients, which finally achieved 97% accuracy of automated detection of bleeding points and gave probability values of blood leakage (i.e., bleeding) everywhere in the image.
According to the enhancement technology of the biological living body blood vessel imaging data based on the unsupervised learning, which is provided by the embodiment of the invention, clear blood vessel information is extracted from the fuzzy non-invasive living body fluorescence imaging data and the X-ray angiography data by utilizing the domain migration thought, and the method can be used for processing and enhancing the real-time fuzzy biological blood vessel image; meanwhile, quantitative analysis is carried out on the extracted blood vessel information, such as blood flow rate, morphological structure information of blood vessels, bleeding point judgment and the like, the visualization quality of biological images can be improved, diagnosis of blood vessels and diseases related to the blood vessels can be assisted, and the method is simple and easy to implement.
It should be noted that the present invention is not only applicable to the illustrated embodiments, but also to other blood vessel images and tubular images such as lymphatic vessels obtained by other imaging means, and has the potential to function only by retraining the neural network.
The background of the present invention may contain background information related to the problem or environment of the present invention and does not necessarily describe the prior art. Accordingly, the inclusion in the background section is not an admission of prior art by the applicant.
The foregoing is a more detailed description of the invention in connection with specific/preferred embodiments and is not intended to limit the practice of the invention to those descriptions. It will be apparent to those skilled in the art that various substitutions and modifications can be made to the described embodiments without departing from the spirit of the invention, and these substitutions and modifications should be considered to fall within the scope of the invention. In the description herein, references to the description of the term "one embodiment," "some embodiments," "preferred embodiments," "an example," "a specific example," or "some examples" or the like are intended to mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Various embodiments or examples and features of various embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction. Although embodiments of the present invention and their advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the scope of the claims.

Claims (8)

1. A method for enhancing blood vessel imaging data of a living organism based on unsupervised learning is characterized by comprising the following steps:
a1, respectively carrying out living body dynamic imaging on living body blood vessels of a living body by utilizing a non-invasive fluorescence imaging or X-ray angiography imaging technology;
a2, extracting a part with a blood vessel representation from the image obtained in the step A1 in an unsupervised learning mode by using the blood vessel binary image of the eye fundus retina of an open source as a target domain in a domain transfer mode;
and A3, performing visual enhancement on the part with the vessel characteristics extracted in the step A2.
2. The method for enhancing imaging data of blood vessels of living organism based on unsupervised learning of claim 1, wherein step a1 is implemented by using non-invasive fluorescence imaging technique to perform dynamic imaging of blood vessels of brain and/or intestine of mouse, or X-ray angiography imaging technique to perform dynamic imaging of blood vessels of living organism.
3. The method for enhancing imaging data of blood vessels of living organisms based on unsupervised learning as claimed in claim 2, wherein in the step A1, the performing living organism dynamic imaging on the blood vessels of the brain of the mouse by using non-invasive fluorescence imaging comprises:
in the near infrared band, the living body dynamic imaging is carried out on the cerebral blood vessels of the mouse by penetrating the skull and the cerebral skin of the mouse.
4. The method for enhancing imaging data of blood vessels of living organisms based on unsupervised learning as claimed in claim 2, wherein in the step A1, the performing living body dynamic imaging of blood vessels of intestines of mice by using non-invasive fluorescence imaging comprises:
in the near infrared band, the living body dynamic imaging is carried out on the intestinal blood vessels of the mice through the abdominal skin and peritoneum of the mice.
5. The method for enhancing blood vessel imaging data of a living organism based on unsupervised learning of claim 2, wherein in step a1, the blood vessel data in the human body in the original modality of the X-ray angiography imaging is also preprocessed to eliminate the influence of shaking of internal organs and blood vessels in the human body.
6. The method for enhancing blood vessel imaging data of a living organism based on unsupervised learning of any one of claims 1 to 5, further comprising training a neural network model by using the blood vessel characteristic information obtained in the step A2, and performing multi-scale test fusion on the to-be-reconstructed picture by using the trained model.
7. An apparatus for enhancing blood vessel imaging data of a living organism based on unsupervised learning, comprising:
an imaging device configured for: respectively carrying out living body dynamic imaging on the blood vessels of the living body of the organism by utilizing a non-invasive fluorescence imaging or X-ray angiography imaging technology;
a processor configured for: by means of domain migration, a blood vessel binary image of an open-source fundus retina is used as a target domain, and a part with a blood vessel representation is extracted from an image obtained by an imaging device in an unsupervised learning mode; and performing visual enhancement on the extracted part with the vessel characterization.
8. The apparatus of claim 7, wherein the processor trains a neural network model using the obtained vessel feature information, and performs multi-scale test fusion on the trained model for the picture to be reconstructed.
CN202010145022.2A 2020-03-04 2020-03-04 Method and device for enhancing biological living body blood vessel imaging data based on unsupervised learning Active CN111445428B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010145022.2A CN111445428B (en) 2020-03-04 2020-03-04 Method and device for enhancing biological living body blood vessel imaging data based on unsupervised learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010145022.2A CN111445428B (en) 2020-03-04 2020-03-04 Method and device for enhancing biological living body blood vessel imaging data based on unsupervised learning

Publications (2)

Publication Number Publication Date
CN111445428A true CN111445428A (en) 2020-07-24
CN111445428B CN111445428B (en) 2023-05-16

Family

ID=71627332

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010145022.2A Active CN111445428B (en) 2020-03-04 2020-03-04 Method and device for enhancing biological living body blood vessel imaging data based on unsupervised learning

Country Status (1)

Country Link
CN (1) CN111445428B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190110753A1 (en) * 2017-10-13 2019-04-18 Ai Technologies Inc. Deep learning-based diagnosis and referral of ophthalmic diseases and disorders
CN109919831A (en) * 2019-02-13 2019-06-21 广州视源电子科技股份有限公司 A kind of method for migrating retinal fundus images in different images domain, electronic equipment and computer readable storage medium
CN110070535A (en) * 2019-04-23 2019-07-30 东北大学 A kind of retinal vascular images dividing method of Case-based Reasoning transfer learning
CN110443813A (en) * 2019-07-29 2019-11-12 腾讯医疗健康(深圳)有限公司 Blood vessel, the dividing method of eye fundus image, device, equipment and readable storage medium storing program for executing

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190110753A1 (en) * 2017-10-13 2019-04-18 Ai Technologies Inc. Deep learning-based diagnosis and referral of ophthalmic diseases and disorders
CN109919831A (en) * 2019-02-13 2019-06-21 广州视源电子科技股份有限公司 A kind of method for migrating retinal fundus images in different images domain, electronic equipment and computer readable storage medium
CN110070535A (en) * 2019-04-23 2019-07-30 东北大学 A kind of retinal vascular images dividing method of Case-based Reasoning transfer learning
CN110443813A (en) * 2019-07-29 2019-11-12 腾讯医疗健康(深圳)有限公司 Blood vessel, the dividing method of eye fundus image, device, equipment and readable storage medium storing program for executing

Also Published As

Publication number Publication date
CN111445428B (en) 2023-05-16

Similar Documents

Publication Publication Date Title
KR101977645B1 (en) Eye image analysis method
Gu et al. CycleGAN denoising of extreme low-dose cardiac CT using wavelet-assisted noise disentanglement
US7471975B2 (en) Optical tensor imaging
JP2018538076A (en) Perfusion digital subtraction angiography
Chlis et al. A sparse deep learning approach for automatic segmentation of human vasculature in multispectral optoacoustic tomography
Longo et al. Assessment of hessian-based Frangi vesselness filter in optoacoustic imaging
US20220151496A1 (en) Device and method for analyzing optoacoustic data, optoacoustic system and computer program
KR20190021749A (en) Method and application for determining cardiovascular disease using ct image
Liu et al. Disentangled representation learning for OCTA vessel segmentation with limited training data
Madhumathy et al. Deep learning based photo acoustic imaging for non-invasive imaging
KR20190125592A (en) 3D Blood Vessel Construction Method using medical images
Eswaran et al. Applying Machine Learning for Medical Image Processing
CN111242850B (en) Wide-area fundus optical coherence blood flow imaging resolution improving method
da Silva et al. An analysis of the influence of transfer learning when measuring the tortuosity of blood vessels
Alidoost et al. Model utility of a deep learning-based segmentation is not Dice coefficient dependent: A case study in volumetric brain blood vessel segmentation
Kugelman et al. A review of generative adversarial network applications in optical coherence tomography image analysis
CN111445428B (en) Method and device for enhancing biological living body blood vessel imaging data based on unsupervised learning
An et al. Application of machine learning method in optical molecular imaging: a review
Morano et al. Deep multimodal fusion of data with heterogeneous dimensionality via projective networks
Chen et al. Ultrasound lmaging-vulnerable plaque diagnostics: Automatic carotid plaque segmentation based on deep learning
Mahesh et al. Intelligent Systems for Medical Diagnostics with the Detection of Diabetic Retinopathy at Reduced Entropy
Lalitha et al. Medical imaging modalities and different image processing techniques: State of the art review
KR20190125793A (en) 3D Blood Vessel Construction Method using medical images
Di Ruberto et al. Special Issue on Image Processing Techniques for Biomedical Applications
Kandaya et al. Analysis of Early Stroke Diagnosis Based on Brain Magnetic Resonance Imaging using Machine Learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant