WO2016090572A1 - 视野可切换的双光路分子影像导航系统及成像方法 - Google Patents

视野可切换的双光路分子影像导航系统及成像方法 Download PDF

Info

Publication number
WO2016090572A1
WO2016090572A1 PCT/CN2014/093474 CN2014093474W WO2016090572A1 WO 2016090572 A1 WO2016090572 A1 WO 2016090572A1 CN 2014093474 W CN2014093474 W CN 2014093474W WO 2016090572 A1 WO2016090572 A1 WO 2016090572A1
Authority
WO
WIPO (PCT)
Prior art keywords
module
imaging
image
color
camera
Prior art date
Application number
PCT/CN2014/093474
Other languages
English (en)
French (fr)
Inventor
田捷
毛亚敏
迟崇巍
杨鑫
Original Assignee
中国科学院自动化研究所
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院自动化研究所 filed Critical 中国科学院自动化研究所
Priority to PCT/CN2014/093474 priority Critical patent/WO2016090572A1/zh
Priority to US15/534,882 priority patent/US10564103B2/en
Publication of WO2016090572A1 publication Critical patent/WO2016090572A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64—Fluorescence; Phosphorescence
    • G01N21/645—Specially adapted constructive features of fluorimeters
    • G01N21/6456—Spatial resolved fluorescence measurements; Imaging
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
    • G01N21/25—Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
    • G01N21/251—Colorimeters; Construction thereof
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
    • G01N21/25—Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
    • G01N21/31—Investigating relative effect of material at wavelengths characteristic of specific elements or molecules, e.g. atomic absorption spectrometry
    • G01N21/35—Investigating relative effect of material at wavelengths characteristic of specific elements or molecules, e.g. atomic absorption spectrometry using infrared light
    • G01N21/359—Investigating relative effect of material at wavelengths characteristic of specific elements or molecules, e.g. atomic absorption spectrometry using infrared light using near infrared light
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B23/00—Telescopes, e.g. binoculars; Periscopes; Instruments for viewing the inside of hollow bodies; Viewfinders; Optical aiming or sighting devices
    • G02B23/24—Instruments or systems for viewing the inside of hollow bodies, e.g. fibrescopes
    • G02B23/2476—Non-optical details, e.g. housings, mountings, supports
    • G02B23/2484—Arrangements in relation to a camera or imaging device
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • H04N23/11—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths for generating image signals from visible and infrared light wavelengths
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/69—Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/71—Charge-coupled device [CCD] sensors; Charge-transfer registers specially adapted for CCD sensors
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N2201/00—Features of devices classified in G01N21/00
    • G01N2201/06—Illumination; Optics
    • G01N2201/068—Optics, miscellaneous
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B23/00—Telescopes, e.g. binoculars; Periscopes; Instruments for viewing the inside of hollow bodies; Viewfinders; Optical aiming or sighting devices
    • G02B23/24—Instruments or systems for viewing the inside of hollow bodies, e.g. fibrescopes
    • G02B23/2407—Optical details
    • G02B23/2461—Illumination
    • G02B23/2469—Illumination using optical fibres
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B23/00—Telescopes, e.g. binoculars; Periscopes; Instruments for viewing the inside of hollow bodies; Viewfinders; Optical aiming or sighting devices
    • G02B23/24—Instruments or systems for viewing the inside of hollow bodies, e.g. fibrescopes
    • G02B23/26—Instruments or systems for viewing the inside of hollow bodies, e.g. fibrescopes using light guides
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H04N23/555—Constructional details for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes

Definitions

  • the invention belongs to the technical field of optical molecular imaging, and relates to an excitation fluorescence imaging method, an image processing method, an endoscopic optical molecular image navigation method, and an open optical molecular image navigation method.
  • Molecular imaging refers to the non-destructive detection and imaging of organisms at the cellular and molecular levels, such as nuclear magnetic resonance, PET, ultrasound, and optical molecular imaging.
  • optical molecular imaging has become a research hotspot with its advantages of low cost, high throughput, non-invasive, non-contact, non-ionizing radiation, high sensitivity and high specificity.
  • Excitation fluorescence molecular imaging technology is an important branch of optical molecular imaging. It uses an external light source to excite fluorescent markers in the living body, and the excited fluorescent label emits fluorescence in the near-infrared band, and the fluorescence is received by a highly sensitive detector. Information and form a fluorescent image.
  • the optical molecular imaging navigation device is used to help us know the fluorescent information.
  • Traditional navigation equipment combined with optical molecular imaging technology is limited by imaging depth, and the endoscopic optical molecular imaging navigation system can penetrate into the imaging area to solve the problem of imaging depth.
  • the endoscopic optical molecular imaging navigation system has a small imaging field and a limited application range.
  • the present invention provides a dual-path molecular imaging navigation system and an imaging method capable of switching between fields of view, which can detect deep imaging regions and observe Imaging area of large field of view.
  • the field of view switchable dual optical path molecular image navigation system comprises a camera module 10, an imaging system conversion module 20, an open imaging module 30, an endoscopic imaging module 40, a data processing module 50, and a system support module 60. ;
  • the camera module 10 is configured to simultaneously perform color imaging and fluorescence imaging on the optical signal sent to the imaging system conversion module 20, and input corresponding color images and fluorescent images to the data processing module 50;
  • the imaging system conversion module 20 is configured to selectively switch between the open imaging module 30 and the endoscopic imaging module 40, and establish a connection with the selected module to establish an optical signal;
  • the data processing module 50 is configured to control the camera module 10, and process, store, and display the image captured by the camera module 10;
  • the system support module 60 is used to support and connect the components.
  • the camera module 10 includes a color CCD camera 11 for color imaging and a fluorescent CCD camera 12 for fluorescence imaging.
  • the imaging system conversion module 20 includes a beam splitting prism 21, a filter one 22, a filter two 23, a camera interface one 24, a camera interface two 25, a lens converter 26;
  • a dichroic prism 21 for dividing the light transmitted by the lens converter 26 into two, and respectively feeding them into the color CCD camera 11 and the fluorescent CCD camera 12;
  • the filter 22 is used for filtering the light transmitted by the dichroic prism 21, and the light having a wavelength between 400 nm and 650 nm is sent to the color CCD camera 11 through the camera interface 24.
  • the filter 23 is used to filter the light transmitted by the dichroic prism 21, and the light having a wavelength between 810 nm and 870 nm is sent to the fluorescent CCD camera 12 through the camera interface 2 25;
  • a lens converter 26 is provided for selectively connecting the open imaging module 30 and the endoscopic imaging module 40.
  • the open imaging module 30 includes a wide-angle lens 31, a white light source 32, a near-infrared source-33, an optical fiber 34, and an imaging region 35;
  • the white light source 32, the near-infrared source-33 illuminate the imaging region 35 through the optical fiber 34, and the wide-angle lens 31 captures the optical signal of the imaging region 35 and sends it to the imaging system conversion module 20.
  • the endoscopic imaging module 40 includes an endoscope lens 41, a white light source 42, a near-infrared light source 43, an endoscope fiber 44, and a detection area 45;
  • the white light source 42 and the near-infrared source 2 are directly coupled into the endoscope fiber 44.
  • the endoscope fiber 44 is sent to the detection area 45, and the collected optical signal is passed through the endoscope fiber 44. It is fed into the endoscope lens 41 and further fed into the imaging system conversion module 20.
  • the imaging system conversion module 20 further includes a rotating shaft 71, which divides the imaging system conversion module 20 into a fixed portion and a rotating portion.
  • the fixed portion is composed of a beam splitting prism 21, a filter 22, a filter 23, and a camera.
  • the interface 24, the camera interface 2, and the rotating shaft 71 are formed.
  • the rotating portion is formed by the lens converter 26.
  • the rotating portion is rotatably connected to the fixed portion through the rotating shaft 71.
  • the lens converter 26 is provided with two through holes for respectively mounting.
  • the open imaging module 30 and the endoscopic imaging module 40 rotate the lens converter 26 to align the optical axis of the wide-angle lens 31 or the endoscope lens 41 with the optical axis of the fixed portion of the imaging system conversion module 20 during use.
  • the data processing module 50 includes a camera control module 51, an image processing module 52, an image storage module 53, and an image display module 54;
  • a camera control module 51 for adjusting parameters of the color CCD camera 11 and the fluorescent CCD camera 12;
  • the image processing module 52 is configured to perform denoising and pseudo color processing on the fluorescence images collected by the color CCD camera 11 and the fluorescent CCD camera 12, and fuse the color image and the fluorescence image by using an image fusion algorithm;
  • the image storage module 53 is configured to store the color image collected by the color CCD camera 11 and the fluorescent CCD camera 12, the fluorescent image, and the image fused by the image processing module 52;
  • the image display module 54 is configured to display the color image, the fluorescent image, and the fused image in real time on the screen.
  • the system support module 60 includes a camera bracket 61, a light source bracket 62, an imaging system conversion module bracket 63, a computer bracket 64, a display bracket 65, and a system branch.
  • Rack 66 ;
  • a camera holder 61 for supporting the color CCD camera 11 and the fluorescent CCD camera 12;
  • a light source bracket 62 for supporting the light source
  • An imaging system conversion module bracket 63 for supporting the imaging system conversion module 20;
  • System brackets 66 are used to connect and support the brackets in the system support module 60.
  • the invention also provides an imaging method for a dual-path molecular imaging navigation system with a view switchable, the method comprising the following steps:
  • Step S1 selecting an open imaging module 30 or an endoscopic imaging module 40 to connect with the imaging system conversion module 20 by determining the detection area;
  • Step S2 When the open imaging module 30 is selected to be connected to the imaging system conversion module 20, the white light source 32 and the near-infrared light source 33 are used to illuminate the imaging region 35, the aperture of the wide-angle lens 31 is adjusted and the lens is focused, and then the fluorescent CCD is simultaneously passed.
  • the camera 12 and the color CCD camera 11 respectively collect a fluorescent image and a color image; when the endoscopic imaging module 40 is selected to be connected to the imaging system conversion module 20, the endoscope lens 41 is adjusted to focus, and the fluorescence is increased by the camera control module 51.
  • the exposure time and the gain multiplier of the CCD camera 12 are then turned on by the white light source 42 and the near-infrared source 234, and the endoscope fiber 44 is penetrated into the detection area 45, and the endoscope fiber 44 is moved to find the fluorescently labeled portion. And simultaneously acquiring a fluorescent image and a color image by the fluorescence CCD camera 12 and the color CCD camera 11;
  • Step S3 If there is a large change in the detection area, step S1 and step S2 are performed again;
  • Step S4 The image processing module 52 is used to fuse the fluorescence image with the color image to obtain a fused image, which is displayed on the computer display screen by the image display module 54.
  • the fusion processing of the fluorescent image and the color image comprises the following steps:
  • Step S41 detecting SIFT feature points in the color image and the fluorescence image
  • Step S42 establishing a k-d tree on the color image and the fluorescence image
  • Step S43 each feature point in the fluorescent image matches the color image
  • Step S44 randomly selecting 4 pairs of matching points to form 8 linear equations to calculate a homography matrix H for color image transformation to a fluorescent image;
  • Step S45 calculating a consistent set of H
  • Step S46 repeatedly performing step S44 and step S45 not less than 500 times to obtain a maximum consistent set
  • Step S47 all matching points in the maximum uniform set constitute an overdetermined linear equation group, and the linear least squares method is used to solve H;
  • Step S48 Convert the color image into the coordinate system of the fluorescent image by H, and perform coordinate transformation
  • Step S49 adding pseudo color to the fluorescent image, and merging the color image in the same coordinate system and the fluorescent image added in the pseudo color to obtain a fused image.
  • the invention realizes the selection of an open imaging module or an endoscopic imaging module through a lens converter, and realizes simultaneous collection of a fluorescent image and a color image through a beam splitter in the imaging system conversion module, and can effectively observe the observation according to one device.
  • the region selects the appropriate observation mode to achieve effective compatibility between imaging depth and breadth, broadens the effective working range of the molecular imaging navigation system, and has a wide range of application scenarios.
  • FIG. 1 is a block diagram of a field of view switchable dual optical path molecular image navigation system in accordance with an embodiment of the present invention
  • FIG. 2 is a schematic diagram of an open imaging module of a field of view switchable dual optical path molecular image navigation system in accordance with an embodiment of the present invention
  • FIG. 3 is a schematic diagram of an endoscopic imaging module of a field of view switchable dual optical path molecular image navigation system according to an embodiment of the present invention
  • FIG. 4 is a schematic diagram showing the conversion of an imaging system of an optical molecular image navigation system based on a beam splitting prism according to an embodiment of the present invention.
  • the field of view switchable dual optical path molecular image navigation system comprises: a camera module 10 , an imaging system conversion module 20 , an open imaging module 30 , an endoscopic imaging module 40 , a data processing module 50 , and a system support module 60 . .
  • the camera module 10 includes a color CCD camera 11 for color imaging and a fluorescent CCD camera 12 for fluorescence imaging for simultaneously performing color imaging and fluorescence imaging on the optical signal sent to the imaging system conversion module 20, and to the data.
  • Processing module 50 inputs the corresponding color image and fluorescent image.
  • the imaging system conversion module 20 includes a beam splitting prism 21, a filter one 22, a filter two 23, a camera interface one 24, a camera interface two 25, and a lens converter 26 for imaging the open imaging module 30 and endoscopic imaging.
  • the module 40 performs selective switching and establishes a connection with the selected module to establish an optical signal.
  • the dichroic prism 21 is configured to divide the light transmitted by the lens converter 26 into two, and respectively send them into the color CCD camera 11 and the fluorescent CCD camera 12; the filter one 22 is used to filter the light transmitted by the dichroic prism 21.
  • the light having a wavelength between 400 nm and 650 nm is sent to the color CCD camera 11 through the camera interface 24; the filter 23 is used to filter the light transmitted by the beam splitting prism 21, and the wavelength is between 810 nm and 870 nm.
  • Light is fed into the fluorescent CCD camera 12 through the camera interface 225; a lens converter 26 is provided for selectively connecting the open imaging module 30 and the endoscopic imaging module 40.
  • the imaging system conversion module 20 is divided into a fixed portion and a rotating portion.
  • the fixed portion is composed of a beam splitting prism 21, a filter 22, a filter 23, a camera interface 24, a camera interface 25, and a rotating shaft 71.
  • the lens converter 26 is configured, the rotating portion is rotatably connected to the fixed portion through the rotating shaft 71, and the lens converter 26 is provided with two through holes for respectively installing the open imaging module 30 and the endoscopic imaging module 40.
  • the middle rotary lens converter 26 aligns the optical axis of the wide-angle lens 31 or the endoscope lens 41 with the optical axis of the fixed portion of the imaging system conversion module 20.
  • the open imaging module 30 includes a wide-angle lens 31, a white light source 32, a near-infrared source-33, an optical fiber 34, and an imaging region 35 for large-field observation imaging.
  • the white light source 32, the near-infrared source-33 illuminate the imaging region 35 through the optical fiber 34, and the wide-angle lens 31 captures the optical signal of the imaging region 35 and sends it to the imaging system conversion module 20.
  • the endoscopic imaging module 40 includes an endoscope lens 41, a white light source 42 , and a near red
  • the external light source 43, the endoscope fiber 44, and the detection area 45 are used for deep field detection imaging.
  • the white light source 42 and the near-infrared source 2 are directly coupled into the endoscope fiber 44.
  • the endoscope fiber 44 is sent to the detection area 45, and the collected optical signal is passed through the endoscope fiber 44. It is fed into the endoscope lens 41 and further fed into the imaging system conversion module 20.
  • the data processing module 50 includes a camera control module 51, an image processing module 52, an image storage module 53, and an image display module 54 for controlling the camera module 10, and processing, storing, and displaying images captured by the camera module 10.
  • the camera control module 51 is configured to adjust parameters of the color CCD camera 11 and the fluorescent CCD camera 12;
  • the image processing module 52 is configured to perform denoising and pseudo color processing on the fluorescence images collected by the color CCD camera 11 and the fluorescent CCD camera 12.
  • the image storage module 53 is configured to store the color image collected by the color CCD camera 11 and the fluorescence CCD camera 12, the fluorescence image, and the image fused by the image processing module 52;
  • the module 54 is configured to display the color image, the fluorescent image and the fused image in real time on the screen.
  • the system support module 60 includes a camera holder 61, a light source holder 62, an imaging system conversion module holder 63, a computer holder 64, a display holder 65, and a system holder 66 for supporting and connecting the components.
  • a camera holder 61 for supporting the color CCD camera 11 and the fluorescent CCD camera 12;
  • a light source holder 62 for supporting the light source;
  • an imaging system conversion module holder 63 for supporting the imaging system conversion module 20;
  • a computer holder 64 for supporting the computer a display stand 65 for supporting the display; a system stand 66 for connecting and supporting each of the brackets in the system support module 60.
  • the color CCD camera 11 and the fluorescent CCD camera 12 in the camera module 10 are connected to the imaging system conversion module 20 through the camera interface 24 and the camera interface 2, respectively; the wide-angle lens 31 of the open imaging module 30 or the endoscopic imaging module 40
  • the endoscope lens 41 is connected to the imaging system conversion module 20 through the lens converter 26; the camera control module 51 in the data processing module 50 performs data with the color CCD camera 11 and the fluorescent CCD camera 12 in the camera module 10 through the camera data line.
  • the camera holder 61 in the system support module 60 is used to support the color CCD camera 11 and the fluorescent CCD camera 12, and the light source holder 62 is used to support the white light source 32 and the near-infrared light source of the open imaging module 30.
  • the imaging system conversion module bracket 63 is used to support the entire imaging system conversion module 20, and the computer bracket 64 is used to support the data processing module 50.
  • the computer, display stand 65 is used to support the display used in the data processing module 50, and the system stand 66 is used for connection and support between the various modules.
  • the camera module 10 performs color imaging and fluorescence imaging; the imaging system conversion module 20 connects the open imaging module 30, and collects the open imaging module 30.
  • the optical signal is divided into two into the camera module 10; the open imaging module 30 provides an open imaging method; and the data processing module 50 provides camera control software and image acquisition, processing, and display methods.
  • the imaging module of the open imaging module is as follows: the white light source 32 and the near-infrared light source 33 are used to illuminate the imaging region 35; the light is collected by the imaging system conversion module 20 to split the optical signal collected by the wide-angle lens 31 into two filters 22 and The filter 23 is filtered and sent to the color CCD camera 11 and the fluorescent CCD camera 12 respectively; the camera control module 51 in the data processing module 50 is turned on, the image capturing mode is turned on, and the imaging of the color CCD camera 11 and the fluorescent CCD camera 12 is adjusted.
  • the parameter, and the image storage module 53 determines the storage location of the image; adjusts the aperture of the wide-angle lens 31 by the obtained captured image, adjusts the lens focus knob to focus the lens; and uses the image processing module 52 to collect the fluorescence of the fluorescent CCD camera 12.
  • the image is processed by denoising, brightness adjustment, adding pseudo color, and the like, and is matched and merged with the color image, and is dynamically displayed on the computer display screen by the image display module 54 in real time.
  • the camera module 10 performs color imaging and fluorescence imaging; the imaging system conversion module 20 connects the endoscopic imaging module 40, and the endoscopic imaging module
  • the collected optical signal is divided into two into the camera module 10; the endoscopic imaging module 40 provides an endoscopic imaging method; and the data processing module 50 provides camera control software and image acquisition, processing, and display methods.
  • the endoscopic imaging module is imaged in the following manner: the lens converter 26 of the imaging system conversion module 20 is connected to the endoscope lens 41 of the endoscopic imaging module 40; the white light source 42 and the near-infrared light source 43 and endoscope The mirror fiber 44 is connected, and the power switch of the white light source 42 and the near-infrared source 2 is turned on; the endoscope fiber 44 is probed into the probe.
  • the optical signal collected by the endoscope lens 41 is split into two by the imaging system conversion module 20, filtered by the filter 22 and the filter 23, and then sent to the color CCD camera 11 and the fluorescent CCD, respectively.
  • the camera control module 51 in the data processing module 50 is turned on, the image capturing mode is turned on, the imaging parameters of the color CCD camera 11 and the fluorescent CCD camera 12 are adjusted, and the image storage module 53 determines the storage location of the image;
  • the image is used to adjust the aperture of the endoscope lens 41 to change the amount of light entering, adjust the lens focus knob to focus the lens, and use the image processing module 52 to perform denoising, brightness adjustment, and pseudo color processing on the fluorescence image collected by the fluorescent CCD camera 12, And matching with the color image, the fusion processing, and the image display module 54 is dynamically displayed on the computer display screen in real time.
  • the lens converter 26 is connected to the fixed portion of the system conversion module 20 via the rotating shaft 71, and realizes the rotational positioning of the lens converter 26 through the spring and the rotating positioning structure specially provided on the rotating shaft 71, which is conveniently realized. Switching of the open imaging module 30 or the endoscopic imaging module in the working mode.
  • two through holes are formed in the lens converter for respectively installing the open imaging module 30 and the endoscopic imaging module 40, and the lens converter 26 is rotated to make the wide-angle lens 31 or the endoscope lens during use.
  • the optical axis of 41 is in line with the optical axis of the fixed portion of imaging system conversion module 20.
  • Step S1 selecting an open imaging module 30 or an endoscopic imaging module 40 to connect with the imaging system conversion module 20 by determining the detection area;
  • Step S2 When the open imaging module 30 is selected to be connected to the imaging system conversion module 20, the white light source 32 and the near-infrared light source 33 are used to illuminate the imaging region 35, the aperture of the wide-angle lens 31 is adjusted and the lens is focused, and then the fluorescent CCD is simultaneously passed.
  • the camera 12 and the color CCD camera 11 respectively collect a fluorescent image and a color image; when the endoscopic imaging module 40 is selected to be connected to the imaging system conversion module 20, the endoscope lens 41 is adjusted to focus, and the fluorescence is increased by the camera control module 51.
  • the exposure time and the gain multiplier of the CCD camera 12 are then turned on by the white light source 42 and the near-infrared source 234, and the endoscope fiber 44 is penetrated into the detection area 45, and the endoscope fiber 44 is moved to find the fluorescently labeled portion. And simultaneously acquiring a fluorescent image and a color image by the fluorescence CCD camera 12 and the color CCD camera 11;
  • Step S3 If there is a large change in the observed area, step S1 and step S2 are performed again;
  • Step S4 The image processing module 52 is used to fuse the fluorescence image with the color image to obtain a fused image, which is displayed on the computer display screen by the image display module 54.
  • step S4 The steps of the image fusion processing method adopted in step S4 in this embodiment are as follows:
  • Step S41 detecting SIFT feature points in the color image and the fluorescence image
  • Step S42 establishing a k-d tree on the color image and the fluorescence image
  • Step S43 each feature point in the fluorescent image matches the color image
  • Step S44 randomly selecting 4 pairs of matching points to form 8 linear equations to calculate a homography matrix H for color image transformation to a fluorescent image;
  • Step S45 calculating a consistent set of H
  • Step S46 repeatedly performing step S44 and step S45 not less than 500 times to obtain a maximum consistent set
  • Step S47 all matching points in the maximum uniform set constitute an overdetermined linear equation group, and the linear least squares method is used to solve H;
  • Step S48 Convert the color image into the coordinate system of the fluorescent image by H, and perform coordinate transformation
  • Step S49 adding pseudo color to the fluorescent image, and merging the color image in the same coordinate system and the fluorescent image added in the pseudo color to obtain a fused image.
  • the embodiment combines the advantages of the open imaging method with the wide imaging field and the endoscopic imaging method to detect the depth of field.
  • the imaging system conversion module can freely convert the imaging field of view according to the visual field requirement.
  • the imaging system conversion module is a dual optical path structure, which can split the light collected by the lens into two. Simultaneous acquisition of fluorescent images and color images by two CCD cameras can effectively select according to the observation area.
  • the observation mode realizes the effective compatibility between imaging depth and breadth, broadens the effective working range of the molecular image navigation system, and has a wide range of application scenarios.

Landscapes

  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Signal Processing (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Chemical & Material Sciences (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Analytical Chemistry (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Biochemistry (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Astronomy & Astrophysics (AREA)
  • Optics & Photonics (AREA)
  • Endoscopes (AREA)

Abstract

本发明公布了一种视野可切换的双光路分子影像导航系统及成像方法，其中包括：相机模块，进行彩色成像及荧光成像；成像系统转换模块，根据成像需求来转换开放式成像方式或者内窥式成像方式；开放式成像模块，进行大视野观测成像；内窥式成像模块，进行深视野探测成像；数据处理模块，提供相机控制软件及图像采集、处理、显示方法；系统支撑模块，为导航设备提供支撑和连接。

Description

视野可切换的双光路分子影像导航系统及成像方法 技术领域
本发明属于光学分子影像技术领域，涉及的内容包括激发荧光成像方法，图像处理方法，内窥式光学分子影像导航方法，开放式光学分子影像导航方法。
背景技术
分子影像是指在细胞和分子水平上对生物体进行无损的探测并进行成像，如核磁共振、PET、超声以及光学分子成像。作为其中重要的一种成像模态，光学分子影像凭借低成本、高通量、非侵入、非接触、非电离辐射、高灵敏度、高特异性等优势已经成为了研究热点。激发荧光分子成像技术是光学分子影像的一个重要的分支，它使用外部光源激发生物体内的荧光标记物，得到激发后的荧光标记物发射出近红外波段的荧光，使用高灵敏度的探测器接受荧光信息并形成荧光图像。
但是近红外波段的光人肉眼是不可见的，需要借助一定的设备来观测，光学分子影像导航设备就是用来辅助我们获知荧光信息。传统与光学分子影像技术相结合的导航设备受到成像深度的限制，而内窥式光学分子影像导航系统可以深入成像区域内部，解决了成像深度的问题。但是内窥式光学分子影像导航系统成像视野小，适用范围有限。
发明内容
为了解决现有光学分子影像导航系统中成像深度与成像广度的问题，本发明提供了一种视野可切换的双光路分子影像导航系统及成像方法，既可以探测较深的成像区域，也可观测大视野的成像区域。
本发明提出的一种视野可切换的双光路分子影像导航系统，包括相机模块10、成像系统转换模块20、开放式成像模块30、内窥式成像模块40、数据处理模块50、系统支撑模块60；
相机模块10，用于对送入成像系统转换模块20的光信号同时进行彩色成像及荧光成像，并向数据处理模块50输入相应的彩色图像和荧光图像；
成像系统转换模块20，用于对开放式成像模块30和内窥式成像模块40进行选择性切换，并与选择到的模块建立光信号的连接；
开放式成像模块30，用于大视野观测成像；
内窥式成像模块40，用于深视野探测成像；
数据处理模块50，用于控制相机模块10，并对控制相机模块10采集的图像进行处理、存储并显示；
系统支撑模块60，用于支撑和连接各部件。
优选的，所述的相机模块10包括用于彩色成像的彩色CCD相机11和用于荧光成像的荧光CCD相机12。
优选的，所述的成像系统转换模块20包括分光棱镜21、滤光片一22、滤光片二23、相机接口一24、相机接口二25、镜头转换器26；
分光棱镜21，用于将由镜头转换器26传输过来的光线一分为二，分别送入彩色CCD相机11和荧光CCD相机12中；
滤光片一22，用于过滤分光棱镜21传输来的光线，将波长在400nm～650nm之间的光线通过相机接口一24送入彩色CCD相机11中；
滤光片二23，用于过滤分光棱镜21传输来的光线，将波长在810nm～870nm之间的光线通过相机接口二25送入荧光CCD相机12中；
镜头转换器26，用于选择性连接开放式成像模块30和内窥式成像模块40。
优选的，所述的开放式成像模块30包括广角镜头31、白光光源一32、近红外光源一33、光纤34、成像区域35；
在使用过程中，白光光源一32、近红外光源一33通过光纤34对成像区域35进行照射，广角镜头31采集成像区域35的光信号并送入成像系统转换模块20中。
优选的，所述的内窥式成像模块40包括内窥镜镜头41、白光光源二42、近红外光源二43、内窥镜光纤44、探测区域45；
白光光源二42、近红外光源二43直接耦合进内窥镜光纤44中，在使用过程中，将内窥镜光纤44送入到探测区域45，将采集到的光信号通过内窥镜光纤44送入到内窥镜镜头41中，并进一步送入成像系统转换模块20中。
优选的，所述的成像系统转换模块20还包括转轴71，将成像系统转换模块20分为固定部分和转动部分，固定部分由分光棱镜21、滤光片一22、滤光片二23、相机接口一24、相机接口二25、转轴71构成，转动部分由镜头转换器26构成，转动部分通过转轴71与固定部分转动连接，镜头转换器26上开设有两个通孔，分别用于装设开放式成像模块30和内窥式成像模块40，在使用过程中旋转镜头转换器26使广角镜头31或内窥镜镜头41的光轴与成像系统转换模块20的固定部分的光轴成一条直线。
优选的，所述的数据处理模块50包括相机控制模块51、图像处理模块52、图像存储模块53、图像显示模块54；
相机控制模块51，用于调节彩色CCD相机11和荧光CCD相机12的参数；
图像处理模块52，用于将彩色CCD相机11和荧光CCD相机12采集到的荧光图像进行去噪、加伪彩处理，并利用图像融合算法将彩色图像和荧光图像融合；
图像存储模块53，用于存储彩色CCD相机11和荧光CCD相机12采集到的彩色图像、荧光图像以及图像处理模块52融合的图像；
图像显示模块54，用于将彩色图像、荧光图像及融合图像实时的显示在屏幕中。
优选的，所述的系统支撑模块60包括相机支架61、光源支架62、成像系统转换模块支架63、计算机支架64、显示器支架65、系统支 架66；
相机支架61，用于支撑彩色CCD相机11和荧光CCD相机12；
光源支架62，用于支撑光源；
成像系统转换模块支架63，用于支撑成像系统转换模块20；
计算机支架64，用于支撑计算机；
显示器支架65，用于支撑显示器；
系统支架66，用于连接和支撑系统支撑模块60中各支架。
本发明还提出了一种视野可切换的双光路分子影像导航系统的成像方法，该方法包含以下步骤：
步骤S1：通过对探测区域的判断选择开放式成像模块30或内窥式成像模块40与成像系统转换模块20相连接；
步骤S2：当选择开放式成像模块30与成像系统转换模块20相连接时，使用白光光源一32及近红外光源一33照射成像区域35，调节广角镜头31光圈并使镜头对焦，然后同时通过荧光CCD相机12和彩色CCD相机11分别采集荧光图像和彩色图像；当选择内窥式成像模块40与成像系统转换模块20相连接时，调节内窥镜镜头41使其对焦，通过相机控制模块51增加荧光CCD相机12的曝光时间以及增益倍数，接着打开白光光源二42、近红外光源二43，将内窥镜光纤44深入到探测区域45内，通过移动内窥镜光纤44来寻找带荧光标记的部位，并同时通过荧光CCD相机12和彩色CCD相机11分别采集荧光图像和彩色图像；
步骤S3：若探测区域有较大的变化，则重新执行步骤S1、步骤S2；
步骤S4：利用图像处理模块52将荧光图像与彩色图像进行融合处理，得到融合图像，并通过图像显示模块54显示在计算机显示屏上。
优选的，所述的荧光图像与彩色图像的融合处理包括以下步骤：
步骤S41：检测彩色图像和荧光图像中的SIFT特征点；
步骤S42：在彩色图像和荧光图像上建立k-d树；
步骤S43：荧光图像中每个特征点与彩色图像相匹配；
步骤S44：随机选4对匹配点构成8个线性方程组计算彩色图像向荧光图像变换的单应矩阵H；
步骤S45：计算H的一致集；
步骤S46：重复执行步骤S44、步骤S45不少于500次，得到最大一致集；
步骤S47：最大一致集中的所有匹配点构成超定线性方程组，利用线性最小二乘法求解H；
步骤S48：通过H，将彩色图像转换到荧光图像的坐标系中，进行坐标变换；
步骤S49：给荧光图像添加伪彩，将处于同一坐标系下的彩色图像和添加伪彩后的荧光图像进行融合，得到融合图像。
本发明通过镜头转换器实现开放式成像模块或内窥式成像模块的选择，并通过成像系统转换模块中的分光器实现荧光图像和彩色图像的同时采集，采用一台设备既可有效的根据观测区域选择合适的观测模式，实现了成像深度与广度的有效兼容，拓宽了了分子影像导航系统的有效工作范围，具有广泛的应用场景。
附图说明
图1是依照本发明实施的视野可切换的双光路分子影像导航系统的架构图；
图2是依照本发明实施的视野可切换的双光路分子影像导航系统的开放式成像模块示意图；
图3是依照本发明实施的视野可切换的双光路分子影像导航系统的内窥式成像模块示意图；
图4是依照本发明实施的基于分光棱镜的光学分子影像导航系统的成像系统转换示意图。
具体实施方式
为使本发明的目的、技术方案和优点更加清楚明白，下面结合附图和实施例子进一步说明。
根据图1所示，视野可切换的双光路分子影像导航系统包括：相机模块10、成像系统转换模块20、开放式成像模块30、内窥式成像模块40、数据处理模块50、系统支撑模块60。
其中相机模块10，包括用于彩色成像的彩色CCD相机11和用于荧光成像的荧光CCD相机12，用于对送入成像系统转换模块20的光信号同时进行彩色成像及荧光成像，并向数据处理模块50输入相应的彩色图像和荧光图像。
成像系统转换模块20包括分光棱镜21、滤光片一22、滤光片二23、相机接口一24、相机接口二25、镜头转换器26，用于对开放式成像模块30和内窥式成像模块40进行选择性切换，并与选择到的模块建立光信号的连接。分光棱镜21，用于将由镜头转换器26传输过来的光线一分为二，分别送入彩色CCD相机11和荧光CCD相机12中；滤光片一22，用于过滤分光棱镜21传输来的光线，将波长在400nm～650nm之间的光线通过相机接口一24送入彩色CCD相机11中；滤光片二23，用于过滤分光棱镜21传输来的光线，将波长在810nm～870nm之间的光线通过相机接口二25送入荧光CCD相机12中；镜头转换器26，用于选择性连接开放式成像模块30和内窥式成像模块40。
将成像系统转换模块20分为固定部分和转动部分，固定部分由分光棱镜21、滤光片一22、滤光片二23、相机接口一24、相机接口二25、转轴71构成，转动部分由镜头转换器26构成，转动部分通过转轴71与固定部分转动连接，镜头转换器26上开设有两个通孔，分别用于装设开放式成像模块30和内窥式成像模块40，在使用过程中旋转镜头转换器26使广角镜头31或内窥镜镜头41的光轴与成像系统转换模块20的固定部分的光轴成一条直线。
开放式成像模块30包括广角镜头31、白光光源一32、近红外光源一33、光纤34、成像区域35，用于大视野观测成像。在使用过程中，白光光源一32、近红外光源一33通过光纤34对成像区域35进行照射，广角镜头31采集成像区域35的光信号并送入成像系统转换模块20中。
内窥式成像模块40包括内窥镜镜头41、白光光源二42、近红 外光源二43、内窥镜光纤44、探测区域45，用于深视野探测成像。白光光源二42、近红外光源二43直接耦合进内窥镜光纤44中，在使用过程中，将内窥镜光纤44送入到探测区域45，将采集到的光信号通过内窥镜光纤44送入到内窥镜镜头41中，并进一步送入成像系统转换模块20中。
数据处理模块50包括相机控制模块51、图像处理模块52、图像存储模块53、图像显示模块54，用于控制相机模块10，并对控制相机模块10采集的图像进行处理、存储并显示。相机控制模块51，用于调节彩色CCD相机11和荧光CCD相机12的参数；图像处理模块52，用于将彩色CCD相机11和荧光CCD相机12采集到的荧光图像进行去噪、加伪彩处理，并利用图像融合算法将彩色图像和荧光图像融合；图像存储模块53，用于存储彩色CCD相机11和荧光CCD相机12采集到的彩色图像、荧光图像以及图像处理模块52融合的图像；图像显示模块54，用于将彩色图像、荧光图像及融合图像实时的显示在屏幕中。
系统支撑模块60包括相机支架61、光源支架62、成像系统转换模块支架63、计算机支架64、显示器支架65、系统支架66，用于支撑和连接各部件。相机支架61，用于支撑彩色CCD相机11和荧光CCD相机12；光源支架62，用于支撑光源；成像系统转换模块支架63，用于支撑成像系统转换模块20；计算机支架64，用于支撑计算机；显示器支架65，用于支撑显示器；系统支架66，用于连接和支撑系统支撑模块60中各支架。
相机模块10中的彩色CCD相机11和荧光CCD相机12分别通过相机接口一24、相机接口二25与成像系统转换模块20相连接；开放式成像模块30的广角镜头31或内窥式成像模块40的内窥镜镜头41通过镜头转换器26与成像系统转换模块20相连接；数据处理模块50中的相机控制模块51通过相机数据线与相机模块10中的彩色CCD相机11和荧光CCD相机12进行数据传输；系统支撑模块60中的相机支架61用于支撑彩色CCD相机11和荧光CCD相机12，光源支架62用于支撑开放式成像模块30的白光光源一32、近红外光源 一33以及内窥式成像模块40的白光光源二42、近红外光源二43，成像系统转换模块支架63用于支撑整个成像系统转换模块20，计算机支架64用于支撑数据处理模块50中用到的计算机，显示器支架65用于支撑数据处理模块50中用到的显示器，系统支架66用于各个模块之间的连接和支撑。
根据图2所示，本实施例开放式成像模块工作状态下：相机模块10，进行彩色成像及荧光成像；成像系统转换模块20，连接开放式成像模块30，并将开放式成像模块30采集到的光信号一分为二送入相机模块10中；开放式成像模块30，提供开放式成像方法；数据处理模块50，提供相机控制软件及图像采集、处理、显示方法。
开放式成像模块成像方式如下：使用白光光源一32及近红外光源一33照射成像区域35；光线由成像系统转换模块20将广角镜头31采集到的光信号一分为二经滤光片一22和滤光片二23过滤后分别送入彩色CCD相机11和荧光CCD相机12中；打开数据处理模块50中的相机控制模块51，开启图像采集模式，调整彩色CCD相机11和荧光CCD相机12的成像参数，以及图像存储模块53确定图像的存储位置；通过得到的采集图像，来调节广角镜头31光圈改变进光量，调节镜头对焦旋钮使镜头对焦；使用图像处理模块52将荧光CCD相机12采集到的荧光图像进行去噪、亮度调节、添加伪彩等处理，并与彩色图像进行匹配、融合处理，用图像显示模块54实时动态的显示在计算机显示屏上。
根据图3所示，本实施例内窥式成像模块工作状态下：相机模块10，进行彩色成像及荧光成像；成像系统转换模块20，连接内窥式成像模块40，并将内窥式成像模块40采集到的光信号一分为二送入相机模块10中；内窥式成像模块40，提供内窥式成像方法；数据处理模块50，提供相机控制软件及图像采集、处理、显示方法。
内窥式成像模块成像方式如下：将成像系统转换模块20的镜头转换器26与内窥式成像模块40的内窥镜镜头41相连接；将白光光源二42及近红外光源二43与内窥镜光纤44相连接，并打开白光光源二42及近红外光源二43的电源开关；将内窥镜光纤44探入探测 区域45中，光线由成像系统转换模块20将内窥镜镜头41采集到的光信号一分为二经滤光片一22和滤光片二23过滤后分别送入彩色CCD相机11和荧光CCD相机12中；打开数据处理模块50中的相机控制模块51，开启图像采集模式，调整彩色CCD相机11和荧光CCD相机12的成像参数，以及图像存储模块53确定图像的存储位置；通过得到的采集图像，来调节内窥镜镜头41光圈改变进光量，调节镜头对焦旋钮使镜头对焦；使用图像处理模块52将荧光CCD相机12采集到的荧光图像进行去噪、亮度调节、添加伪彩等处理，并与彩色图像进行匹配、融合处理，用图像显示模块54实时动态的显示在计算机显示屏上。
根据图4所示，镜头转换器26通过转轴71与系统转换模块20的固定部分连接，并通过特设在转轴71上的弹簧及转动定位结构实现镜头转换器26的转动定位，方便的实现了工作模式下开放式成像模块30或内窥式成像模块的切换。本实施例中镜头转换器上开设有两个通孔，分别用于装设开放式成像模块30和内窥式成像模块40，在使用过程中旋转镜头转换器26使广角镜头31或内窥镜镜头41的光轴与成像系统转换模块20的固定部分的光轴成一条直线。
本实施例的成像方法包含以下步骤：
步骤S1：通过对探测区域的判断选择开放式成像模块30或内窥式成像模块40与成像系统转换模块20相连接；
步骤S2：当选择开放式成像模块30与成像系统转换模块20相连接时，使用白光光源一32及近红外光源一33照射成像区域35，调节广角镜头31光圈并使镜头对焦，然后同时通过荧光CCD相机12和彩色CCD相机11分别采集荧光图像和彩色图像；当选择内窥式成像模块40与成像系统转换模块20相连接时，调节内窥镜镜头41使其对焦，通过相机控制模块51增加荧光CCD相机12的曝光时间以及增益倍数，接着打开白光光源二42、近红外光源二43，将内窥镜光纤44深入到探测区域45内，通过移动内窥镜光纤44来寻找带荧光标记的部位，并同时通过荧光CCD相机12和彩色CCD相机11分别采集荧光图像和彩色图像；
步骤S3：若观测区域有较大的变化，则重新执行步骤S1、步骤S2；
步骤S4：利用图像处理模块52将荧光图像与彩色图像进行融合处理，得到融合图像，并通过图像显示模块54显示在计算机显示屏上。
本实施例在步骤S4中采用的图像融合处理方法的步骤如下：
步骤S41：检测彩色图像和荧光图像中的SIFT特征点；
步骤S42：在彩色图像和荧光图像上建立k-d树；
步骤S43：荧光图像中每个特征点与彩色图像相匹配；
步骤S44：随机选4对匹配点构成8个线性方程组计算彩色图像向荧光图像变换的单应矩阵H；
步骤S45：计算H的一致集；
步骤S46：重复执行步骤S44、步骤S45不少于500次，得到最大一致集；
步骤S47：最大一致集中的所有匹配点构成超定线性方程组，利用线性最小二乘法求解H；
步骤S48：通过H，将彩色图像转换到荧光图像的坐标系中，进行坐标变换；
步骤S49：给荧光图像添加伪彩，将处于同一坐标系下的彩色图像和添加伪彩后的荧光图像进行融合，得到融合图像。
本实施例结合了开放式成像方法成像视野广和内窥式成像方法探测视野深的优点，通过成像系统转换模块可以根据视野需求自由转换成像视野。同时该成像系统转换模块是双光路结构，可将镜头采集到的光线一分为二，用两台CCD相机实现荧光图像和彩色图像的同时采集采用一台设备既可有效的根据观测区域选择合适的观测模式，实现了成像深度与广度的有效兼容，拓宽了了分子影像导航系统的有效工作范围，具有广泛的应用场景。
在操作人员实际使用过程中，可以根据成像需求切换到合适的成像模式中。
以上所述的具体实施方式，对本发明的目的、技术方案和有益效 果进行了进一步详细说明，但是以上所述仅为本发明的具体实施方式本发明的保护范围并不仅限于此，凡在本发明的思想和规则之内，所做的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。

Claims (10)

  1. 一种视野可切换的双光路分子影像导航系统，其特征在于，该系统包括相机模块(10)、成像系统转换模块(20)、开放式成像模块(30)、内窥式成像模块(40)、数据处理模块(50)、系统支撑模块(60)；
    相机模块(10)，用于对送入成像系统转换模块(20)的光信号同时进行彩色成像及荧光成像，并向数据处理模块(50)输入相应的彩色图像和荧光图像；
    成像系统转换模块(20)，用于对开放式成像模块(30)和内窥式成像模块(40)进行选择性切换，并与选择到的模块建立光信号的连接；
    开放式成像模块(30)，用于大视野观测成像；
    内窥式成像模块(40)，用于深视野探测成像；
    数据处理模块(50)，用于控制相机模块(10)，并对控制相机模块(10)采集的图像进行处理、存储并显示；
    系统支撑模块(60)，用于支撑和连接各部件。
  2. 根据权利要求1所述的视野可切换的双光路分子影像导航系统，其特征在于，所述的相机模块(10)包括用于彩色成像的彩色CCD相机(11)和用于荧光成像的荧光CCD相机(12)。
  3. 根据权利要求2所述的视野可切换的双光路分子影像导航系统，其特征在于，所述的成像系统转换模块(20)包括分光棱镜(21)、滤光片一(22)、滤光片二(23)、相机接口一(24)、相机接口二(25)、镜头转换器(26)；
    分光棱镜(21)，用于将由镜头转换器(26)传输过来的光线一分为二，分别送入彩色CCD相机(11)和荧光CCD相机(12)中；
    滤光片一(22)，用于过滤分光棱镜(21)传输来的光线，将波长在400nm～650nm之间的光线通过相机接口一(24)送入彩色CCD相机(11)中；
    滤光片二(23)，用于过滤分光棱镜(21)传输来的光线，将波 长在810nm～870nm之间的光线通过相机接口二(25)送入荧光CCD相机(12)中；
    镜头转换器(26)，用于选择性连接开放式成像模块(30)和内窥式成像模块(40)。
  4. 根据权利要求2或3所述的视野可切换的双光路分子影像导航系统，其特征在于，所述的开放式成像模块(30)包括广角镜头(31)、白光光源一(32)、近红外光源一(33)、光纤(34)、成像区域(35)；
    在使用过程中，白光光源一(32)、近红外光源一(33)通过光纤(34)对成像区域(35)进行照射，广角镜头(31)采集成像区域(35)的光信号并送入成像系统转换模块(20)中。
  5. 根据权利要求4所述的视野可切换的双光路分子影像导航系统，其特征在于，所述的内窥式成像模块(40)包括内窥镜镜头(41)、白光光源二(42)、近红外光源二(43)、内窥镜光纤(44)、探测区域(45)；
    白光光源二(42)、近红外光源二(43)直接耦合进内窥镜光纤(44)中，在使用过程中，将内窥镜光纤(44)送入到探测区域(45)，将采集到的光信号通过内窥镜光纤(44)送入到内窥镜镜头(41)中，并进一步送入成像系统转换模块(20)中。
  6. 根据权利要求5所述的视野可切换的双光路分子影像导航系统，其特征在于，所述的所述的成像系统转换模块(20)还包括转轴(71)，将成像系统转换模块(20)分为固定部分和转动部分，固定部分由分光棱镜(21)、滤光片一(22)、滤光片二(23)、相机接口一(24)、相机接口二(25)、转轴(71)构成，转动部分由镜头转换器(26)构成，转动部分通过转轴(71)与固定部分转动连接，镜头转换器(26)上开设有两个通孔，分别用于装设开放式成像模块(30)和内窥式成像模块(40)，在使用过程中旋转镜头转换器(26)使广角镜头(31)或内窥镜镜头(41)的光轴与成像系统转换模块(20)的固定部分的光轴成一条直线。
  7. 根据权利要求6所述的视野可切换的双光路分子影像导航系统，其特征在于，所述的数据处理模块(50)包括相机控制模块(51)、 图像处理模块(52)、图像存储模块(53)、图像显示模块(54)；
    相机控制模块(51)，用于调节彩色CCD相机(11)和荧光CCD相机(12)的参数；
    图像处理模块(52)，用于将彩色CCD相机(11)和荧光CCD相机(12)采集到的荧光图像进行去噪、加伪彩处理，并利用图像融合算法将彩色图像和荧光图像融合；
    图像存储模块(53)，用于存储彩色CCD相机(11)和荧光CCD相机(12)采集到的彩色图像、荧光图像以及图像处理模块(52)融合的图像；
    图像显示模块(54)，用于将彩色图像、荧光图像及融合图像实时的显示在屏幕中。
  8. 根据权利要求7所述的视野可切换的双光路分子影像导航系统，其特征在于，所述的系统支撑模块(60)包括相机支架(61)、光源支架(62)、成像系统转换模块支架(63)、计算机支架(64)、显示器支架(65)、系统支架(66)；
    相机支架(61)，用于支撑彩色CCD相机(11)和荧光CCD相机(12)；
    光源支架(62)，用于支撑光源；
    成像系统转换模块支架(63)，用于支撑成像系统转换模块(20)；
    计算机支架(64)，用于支撑计算机；
    显示器支架(65)，用于支撑显示器；
    系统支架(66)，用于连接和支撑系统支撑模块(60)中各支架。
  9. 一种视野可切换的双光路分子影像导航系统的成像方法，其特征在于，该方法包含以下步骤：
    步骤S1：通过对探测区域的判断选择开放式成像模块(30)或内窥式成像模块(40)与成像系统转换模块(20)相连接；
    步骤S2：当选择开放式成像模块(30)与成像系统转换模块(20)相连接时，使用白光光源一(32)及近红外光源一(33)照射成像区域(35)，调节广角镜头(31)光圈并使镜头对焦，然后同时通过荧光CCD相机(12)和彩色CCD相机(11)分别采集荧光图像和彩色图 像；当选择内窥式成像模块(40)与成像系统转换模块(20)相连接时，调节内窥镜镜头(41)使其对焦，通过相机控制模块(51)增加荧光CCD相机(12)的曝光时间以及增益倍数，接着打开白光光源二(42)、近红外光源二(43)，将内窥镜光纤(44)深入到探测区域(45)内，通过移动内窥镜光纤(44)来寻找带荧光标记的部位，并同时通过荧光CCD相机(12)和彩色CCD相机(11)分别采集荧光图像和彩色图像；
    步骤S3：若观测区域有较大的变化，则重新执行步骤S1、步骤S2；
    步骤S4：利用图像处理模块(52)将荧光图像与彩色图像进行融合处理，得到融合图像，并通过图像显示模块(54)显示在计算机显示屏上。
  10. 根据权利要求9所述的成像方法，其特征在于，所述的荧光图像与彩色图像的融合处理包括以下步骤：
    步骤S41：检测彩色图像和荧光图像中的SIFT特征点；
    步骤S42：在彩色图像和荧光图像上建立k-d树；
    步骤S43：荧光图像中每个特征点与彩色图像相匹配；
    步骤S44：随机选4对匹配点构成8个线性方程组计算彩色图像向荧光图像变换的单应矩阵H；
    步骤S45：计算H的一致集；
    步骤S46：重复执行步骤S44、步骤S45不少于500次，得到最大一致集；
    步骤S47：最大一致集中的所有匹配点构成超定线性方程组，利用线性最小二乘法求解H；
    步骤S48：通过H，将彩色图像转换到荧光图像的坐标系中，进行坐标变换；
    步骤S49：给荧光图像添加伪彩，将处于同一坐标系下的彩色图像和添加伪彩后的荧光图像进行融合，得到融合图像。
PCT/CN2014/093474 2014-12-10 2014-12-10 视野可切换的双光路分子影像导航系统及成像方法 WO2016090572A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2014/093474 WO2016090572A1 (zh) 2014-12-10 2014-12-10 视野可切换的双光路分子影像导航系统及成像方法
US15/534,882 US10564103B2 (en) 2014-12-10 2014-12-10 Dual-mode optical molecular imaging navigation apparatus with a switchable field of view and imaging method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2014/093474 WO2016090572A1 (zh) 2014-12-10 2014-12-10 视野可切换的双光路分子影像导航系统及成像方法

Publications (1)

Publication Number Publication Date
WO2016090572A1 true WO2016090572A1 (zh) 2016-06-16

Family

ID=56106441

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2014/093474 WO2016090572A1 (zh) 2014-12-10 2014-12-10 视野可切换的双光路分子影像导航系统及成像方法

Country Status (2)

Country Link
US (1) US10564103B2 (zh)
WO (1) WO2016090572A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109191415B (zh) * 2018-08-22 2020-12-15 成都纵横自动化技术股份有限公司 图像融合方法、装置及电子设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060030844A1 (en) * 2004-08-04 2006-02-09 Knight Bradley P Transparent electrode for the radiofrequency ablation of tissue
CN104116497A (zh) * 2014-07-22 2014-10-29 中国科学院自动化研究所 内窥式光学分子影像导航系统和多光谱成像方法
CN104367380A (zh) * 2014-12-10 2015-02-25 中国科学院自动化研究所 视野可切换的双光路分子影像导航系统及成像方法
CN204318916U (zh) * 2014-12-10 2015-05-13 中国科学院自动化研究所 视野可切换的双光路分子影像导航系统

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8620410B2 (en) * 2002-03-12 2013-12-31 Beth Israel Deaconess Medical Center Multi-channel medical imaging system
TR201901658T4 (tr) * 2008-05-20 2019-02-21 Univ Health Network Floresan bazli görüntüleme ve i̇zleme i̇çi̇n ci̇haz ve metot
US8169696B2 (en) * 2009-06-04 2012-05-01 General Electric Company Systems for intraoperative nerve imaging
CN103687529B (zh) * 2011-07-22 2016-05-18 奥林巴斯株式会社 荧光内窥镜装置
US8977331B2 (en) * 2012-12-13 2015-03-10 General Electric Company Systems and methods for nerve imaging
US10580130B2 (en) * 2017-03-24 2020-03-03 Curadel, LLC Tissue identification by an imaging system using color information

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060030844A1 (en) * 2004-08-04 2006-02-09 Knight Bradley P Transparent electrode for the radiofrequency ablation of tissue
CN104116497A (zh) * 2014-07-22 2014-10-29 中国科学院自动化研究所 内窥式光学分子影像导航系统和多光谱成像方法
CN104367380A (zh) * 2014-12-10 2015-02-25 中国科学院自动化研究所 视野可切换的双光路分子影像导航系统及成像方法
CN204318916U (zh) * 2014-12-10 2015-05-13 中国科学院自动化研究所 视野可切换的双光路分子影像导航系统

Also Published As

Publication number Publication date
US20180038795A1 (en) 2018-02-08
US10564103B2 (en) 2020-02-18

Similar Documents

Publication Publication Date Title
US20190046041A1 (en) Image processing apparatus, image processing method, information processing program, fluorescence observation system, and fluorescence navigation surgery system
CN104367380B (zh) 视野可切换的双光路分子影像导航系统及成像方法
JP6023883B2 (ja) 灌流評価マルチモダリティ光学医用デバイス
JP6637653B2 (ja) 顕微鏡およびｓｐｉｍ顕微鏡検査方法
WO2016011611A1 (zh) 内窥式光学分子影像导航系统和多光谱成像方法
CN103300812A (zh) 基于内窥镜的多光谱视频导航系统和方法
JP2009225933A (ja) カプセル内視鏡システム及びカプセル内視鏡の動作制御方法
JP7150867B2 (ja) 顕微鏡システム
JP2011008245A (ja) 生体観察装置
WO2014205738A1 (zh) 基于内窥镜的多光谱视频导航系统和方法
CN110168609A (zh) 用于在数字显微镜中生成试样的三维模型的方法和数字显微镜
CN204318916U (zh) 视野可切换的双光路分子影像导航系统
JP2017176811A (ja) 撮像装置、撮像方法及び医療用観察機器
US8994806B2 (en) Microscope apparatus chronologically storing different types of image information
EP2962622A1 (en) Image processing device, endoscope device, image processing method, and image processing program
JP2013114233A (ja) 画像処理装置、顕微鏡システム、画像処理方法、及び画像処理プログラム
JP7277560B2 (ja) 被験体を非侵襲的に検査するためのマルチモード撮像システムおよび方法
JP6859554B2 (ja) 観察補助装置、情報処理方法、およびプログラム
JP2016099370A (ja) 顕微鏡システム
WO2016090572A1 (zh) 视野可切换的双光路分子影像导航系统及成像方法
TWI554740B (zh) 高速三維成像之光學系統
US20160055639A1 (en) Image processing apparatus, image processing method and image processing program
CN110089992A (zh) 一种成像光谱内窥镜系统
JP2012050617A (ja) 画像取得方法および装置
CN103591892B (zh) 一种便携式多波段光源三维现场勘查取证仪及取证方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14907957

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 15534882

Country of ref document: US

122 Ep: pct application non-entry in european phase

Ref document number: 14907957

Country of ref document: EP

Kind code of ref document: A1