CN114468960A - Endoscope device integrating optics, optoacoustic, OCT and fluorescence multi-mode forward imaging and imaging method thereof - Google Patents

Abstract

The invention discloses an endoscope device integrating optics, optoacoustic, OCT and fluorescence multi-mode forward imaging and an imaging method thereof, wherein the device comprises a control system, a light source system, a light path system, an endoscope probe and a data acquisition and reconstruction system; the control system synchronously triggers the light source system and the data acquisition system, exciting light enters the endoscope probe through the light path system, a dichroic mirror inside the probe realizes the excitation of light beams by opto-acoustic/fluorescence and OCT, the combined light irradiates the surface of a tissue to be imaged through a rod lens, echo signals are processed by the data acquisition system and optical image reconstruction is completed by opto-acoustic images, OCT images and fluorescence images, and optical signals are acquired by a CCD (charge coupled device) at the top of the endoscope for optical image reconstruction. The photoacoustic, fluorescence and OCT exciting lights share a light path, chromatic aberration is eliminated by the adjustable-focus collimator, and the rod-shaped lens structure is adopted to go deep into a cavity organ for in-vivo detection, so that real-time high-resolution confocal endoscopic imaging of photoacoustic imaging, fluorescence imaging and OCT imaging is realized.

Description

Endoscope device integrating optics, optoacoustic, OCT and fluorescence multi-mode forward imaging and imaging method thereof

Technical Field

The invention belongs to the technical field of endoscope nondestructive detection, and particularly relates to an endoscope device integrating optics, optoacoustic, OCT and fluorescence multi-mode forward imaging and an imaging method thereof.

Background

Most cases are diagnosed at middle and late stages and have low survival rate, so early diagnosis and early treatment of cancer become problems to be solved urgently. Taking nasopharyngeal carcinoma as an example, nasopharyngeal carcinoma is a common malignant tumor of the head and neck, which is better to be found on the top wall and the side wall of nasopharynx, has hidden positions, infiltrative growth in early stage, is not obvious in clinical manifestation, and is mainly diagnosed by means of imaging manifestation. Radiotherapy is the first choice but is susceptible to recurrence. Therefore, early diagnosis, early treatment and follow-up after radiotherapy of nasopharyngeal carcinoma are particularly important. CT and MRI are the conventional means for nasopharyngeal carcinoma diagnosis, staging and post-radiotherapy review. CT cannot distinguish local inflammation from tumor, and the detection rate of nasopharyngeal carcinoma by CT is only 57.1%. Disadvantages of MRI imaging are longer imaging time, larger artifacts, and insensitivity to calcific foci and cortical bone imaging.

The white light endoscope is always a standard tool for diagnosing nasopharyngeal carcinoma, irradiates white light on a tissue area to be imaged, receives a white light image through a CCD (charge coupled device) or a CMOS (complementary metal oxide semiconductor), and can obtain morphological information of the imaged area; however, the fundamental principle is that the reflected signal is used to obtain the detection image, which is greatly influenced by the scattering of the medium, and only the shallow surface of the imaging region can be imaged.

The photoacoustic imaging technology is used as a novel noninvasive biomedical imaging method, breaks through the limitation of the traditional optical imaging and ultrasonic imaging in the respective biological tissue imaging field, and has great potential in the aspects of biomedical imaging and clinical application. The photoacoustic imaging technology is based on photoacoustic effect, under the irradiation of short pulse laser beams, biological tissues to be detected absorb photon energy and then are heated and expanded to generate ultrasonic signals, and the ultrasonic signals are received by a transducer and are reconstructed by an algorithm to obtain a distribution image reflecting the light absorption difference in the tissues. Therefore, the photoacoustic imaging technology has the characteristics of high optical imaging resolution and deep ultrasonic imaging depth, and can perform functional imaging and structural imaging on tissues. The photoacoustic imaging can obtain a blood vessel distribution image near the tumor tissue and is used for judging the infiltration depth of the tumor tissue.

The OCT imaging technique is a non-invasive high-resolution medical imaging technique, and can obtain a resolution of micron order at a depth of millimeter order, compared to the conventional imaging technique. The OCT imaging technology is based on the principle of an interferometer, when near-infrared weak coherent light irradiates a tissue to be imaged, backward scattering light of a reference arm interferes with reflected light of a sample arm, and a spectrometer receives interference signals to obtain tissue superficial structure information.

The fluorescence imaging technology is characterized in that exogenous fluorescent substances are injected into a blood vessel, the fluorescent substances are excited by laser, fluorescence is received by a receiving device, tissues are imaged through received fluorescence information, distribution information of different tissues in the region can be obtained, and important reference can be provided for delineating a biological target region due to high sensitivity and specificity of fluorescent molecule targeted imaging.

The current common endoscope is a white light endoscope, can only image the surface of a tissue, lacks tissue depth information, cannot judge the tumor infiltration depth, cannot divide the tumor boundary, and has obvious defects in preoperative diagnosis and postoperative reexamination. An integrated endoscope device integrating optics, optoacoustic, OCT and fluorescence multi-mode forward imaging integrates four imaging modes of optics, optoacoustic, OCT and fluorescence in an integrated endoscope probe. The photoacoustic imaging technology combines the characteristics of high optical imaging resolution and deep acoustic imaging depth, can image a tumor microvascular network, and judges the tumor infiltration depth; the OCT endoscopic imaging technology combined with the endoscopic technology can carry out in-vivo high-resolution imaging on a biological tissue cavity to obtain tissue structure information; the fluorescent molecular targeted imaging has high sensitivity and specificity, can mark tumor boundaries, and provides important reference for delineating a biological target region in the radiotherapy process.

The application number is 201410412368.9, the patent name is 'a multi-mode microscopic imaging system', and the multi-mode microscopic imaging system which utilizes optical means to detect is disclosed, and the system can be used for imaging the internal structure and the function of the tissue by combining three imaging modes of ultrasound, OCT and optoacoustic. Due to fluorescence-free imaging in the imaging mode, morphological information of the tissue surface cannot be obtained; meanwhile, as no achromatic device is arranged in the device, confocal imaging cannot be carried out, and the scanning structure is in a lateral imaging mode, so that forward scanning imaging cannot be carried out on the tissue to be detected.

The application number is 201710112585.X, the patent name is intravascular fluorescence-photoacoustic-ultrasonic multimode imaging device, and the intravascular fluorescence-photoacoustic-ultrasonic multimode imaging device and method are disclosed, and three modes of fluorescence imaging, photoacoustic imaging and ultrasonic imaging are integrated with an intravascular endoscopic imaging system. However, in the imaging mode, no OCT is available, and high-resolution structural imaging cannot be performed on a tissue part to be detected, and meanwhile, the scanning structure is a lateral imaging mode, and forward scanning imaging cannot be performed on the tissue to be detected.

The application number is 202010263211.X, the patent name is "multimode microscopic endoscopic imaging device and method", which discloses a multimode microscopic endoscopic imaging device, which combines three imaging modes of confocal imaging, photoacoustic imaging and ultrasonic imaging and has two modes of forward scanning and lateral scanning. However, no OCT is used in the imaging mode, which does not allow high-resolution structural imaging of the tissue region to be detected, and at the same time, because there is no achromatic device in the device, confocal imaging cannot be performed when the excitation wavelengths are far apart.

Disclosure of Invention

Disclosure of Invention

The invention mainly aims to overcome the defects of the prior art and provide an endoscope device integrating optics, optoacoustic, OCT and fluorescence multi-mode forward imaging and an imaging method thereof. .

In order to achieve the purpose, the invention adopts the following technical scheme:

the invention provides an optical, photoacoustic, OCT and fluorescence multi-mode forward imaging integrated endoscope device which comprises a control system, a light source system, a light path system, an endoscope probe and a data acquisition and reconstruction system, wherein the control system, the light source system, the light path system, the endoscope probe and the data acquisition and reconstruction system are respectively connected;

the control system takes an FPGA development board as a control module and generates time sequence pulses to control each imaging submodule;

the light source system comprises a pulse laser, a continuous laser and a white light source; the continuous laser and the white light source control end are connected with the FPGA development board and used for controlling the output of the light source, and the light source output end is connected to the light path system through an optical fiber;

the optical path system comprises an optical fiber coupler, a single-mode optical fiber, a second dichroic mirror, a second focusing lens, a coupler, a third collimator, a fourth collimator, a second reflecting mirror, a spectrometer and a photomultiplier; the output light of the pulse laser is reflected by the second dichroic mirror to enter the optical fiber coupler, the optical fiber coupler is connected with a second collimator in the endoscope probe through a single-mode optical fiber, the fluorescence excitation light is emitted out through the optical fiber coupler and enters the second dichroic mirror, and the emergent light of the second dichroic mirror is focused by the second focusing lens and enters a photoelectric cathode of the photomultiplier; the continuous laser output light is connected with the coupler through a single-mode optical fiber, the coupler divides the input light into two parts, one part enters the endoscope probe through the first collimator, the other part enters the second reflector through the third collimator in a collimating way, the sample arm return light emitted by the first collimator interferes with the reflected light from the third collimator in the coupler, and the sample arm return light is emitted out of the spectrometer through the fourth collimator. A white light source in the light source system is connected to the side wall optical fiber of the endoscope probe through a single mode optical fiber for tissue illumination;

the endoscope probe is composed of an integrated endoscope and comprises a Charge Coupled Device (CCD), an ocular, a spectroscope, a first collimator, a second collimator, a first dichroic mirror, a first focusing lens, a first reflector, an MEMS mirror, a rod-shaped lens, an optical fiber, an ultrasonic transducer, a light-transmitting and sound-reflecting mirror and an endoscope shell; the output light of the pulse laser enters the integrated endoscope through the second collimator, horizontally enters the integrated endoscope through the first dichroic mirror, the output light of the continuous laser enters the integrated endoscope through the first collimator, is reflected by the first dichroic mirror and then is combined with the pulse light, the combined light is focused by the first focusing lens and then enters the first reflecting mirror, is reflected and then enters the surface of the MEMS mirror, the scanning light is reflected by the beam splitter and then enters the rod-shaped lens, and the scanning light is transmitted inside the rod-shaped lens and passes through the light-transmitting anti-sound mirror and enters the surface of the tissue;

the data acquisition and reconstruction system comprises a first signal amplifier, a first acquisition card, a second signal amplifier, a second acquisition card, a third acquisition card and a computer, and the multi-mode imaging system simultaneously images white light, opto-acoustic, OCT and fluorescence images. In photoacoustic imaging, pulse laser is incident to the surface of a tissue, ultrasonic waves generated by the surface after being excited are received by a side wall transducer of an endoscope probe, signals enter a first acquisition card after being amplified by a first signal amplifier, and original data are sent to a computer by the first acquisition card; in OCT imaging, a spectrometer receives interference signals and sends the interference signals to a computer by a third acquisition card; in fluorescence imaging, a photomultiplier converts a fluorescence signal into a current signal, the current signal is converted into a voltage signal through a second signal amplifier, and a second acquisition card transmits the voltage signal to a computer.

Preferably, the first dichroic mirror is used for enabling fluorescence imaging, photoacoustic imaging and OCT imaging to share a light path, the wavelength difference between fluorescence and photoacoustic signal excitation light and OCT excitation light is large, and the first dichroic mirror with the cutoff frequency of 850nm to 1180nm is selected to reflect the OCT signal excitation light and transmit the fluorescence and photoacoustic signal excitation light, so that beam combination of space light is realized.

Preferably, the first dichroic mirror is a short-wave-pass dichroic mirror, and the cutoff wavelength is 850nm to 1180 nm.

Preferentially, the second dichroic mirror is used for enabling the fluorescence imaging and the photoacoustic imaging to share a light path, the fluorescence signal and the photoacoustic signal excitation light are reflected by the second dichroic mirror to enter the optical fiber coupler, the wavelength of the fluorescence signal is higher than that of the fluorescence excitation light, the fluorescence signal is emitted by the optical fiber coupler to pass through the second dichroic mirror, light lower than the cut-off wavelength of the second dichroic mirror is reflected, and the fluorescence signal is focused by the focusing lens to enter the photomultiplier, so that the photoacoustic imaging and the fluorescence imaging are simultaneously imaged.

Preferably, the second dichroic mirror is a long-wave-pass dichroic mirror, and the cut-off wavelength is 550nm to 780 nm.

Preferentially, pulse laser and continuous laser inside the endoscope probe are combined through the first dichroic mirror, focused through the laser focusing light path and then reflected to the surface of the MEMS mirror by the reflecting mirror, and the control system controls the MEMS mirror to deflect and adjust the scanning range.

Preferentially, the first collimator is a focusing collimator, because the wavelength difference between the photoacoustic and fluorescence signal excitation light and the OCT signal excitation light is large, dispersion can occur when the photoacoustic and fluorescence signal excitation light passes through the same lens, and by adjusting the focusing collimator, a multimode imaging confocal point is realized, and clear images under different imaging modes can be obtained at the same position.

Preferably, the pulse laser is one of a semiconductor laser, a solid laser, a dye laser or a gas laser, the wavelength range of the output pulse laser is 400 nm-2500 nm, and the pulse width is 5 ns-50 ns.

Preferably, the amplification gain of the first signal amplifier is 50dB to 60dB, and the gain of the second signal amplifier is 46 dB.

The invention also provides an imaging method of the endoscope device integrating optical, photoacoustic, OCT and fluorescence multi-mode forward imaging, which comprises the following steps:

(1) and (3) triggering the system: the control system generates a trigger signal to control the pulse laser to generate pulse light, the continuous laser generates continuous light and controls the illumination light source at the same time, the pulse laser enters the optical fiber coupler through the second dichroic mirror and enters the endoscope through the second collimator, the continuous laser enters the endoscope through the first collimator through the coupler and is reflected by the first dichroic mirror to form a beam with the pulse laser; the combined beam light is reflected by a reflecting mirror and focused by a focusing lens to finally reach the surface of the tissue to be imaged. The illumination light irradiates the surface of the tissue to be imaged through the optical fiber;

(2) data acquisition: photoacoustic data acquisition: the photoacoustic signal excited by the pulse laser is reflected by a light-transmitting sound reflector, is received by an ultrasonic transducer on the side wall of the endoscope and is converted into an electric signal, the electric signal enters a first amplifier through a coaxial cable, and the amplified electric signal is collected by a first acquisition card and is sent to a computer; OCT data acquisition: the OCT signal excited by the continuous laser is backward scattered light from the surface of the tissue to be imaged, is received by the endoscope, is emitted along an incident light path, enters the coupler through the first collimator, interferes with reflected light from the reference arm, is detected by the spectrometer, is converted into an electric signal at the same time, is collected by the third collecting card and is sent to the computer; fluorescence data acquisition: the fluorescence signal excited by the pulse laser is received by the endoscope, is emitted along an incident light path, enters the optical fiber coupler through the second collimator, is focused by the focusing lens through the second dichroic mirror, then enters the photomultiplier, converts the fluorescence signal into an electric signal, passes through the second amplifier, is collected by the second collection card and is sent to the computer; optical data acquisition: the optical signal is received by the endoscope, enters the CCD through the spectroscope via the rod lens group, can convert the incident light signal into electric charge to be output by the CCD, and enters the computer through the data line;

(3) image processing: the computer uses the collected data for the reconstruction of optical images, photoacoustic images, OCT images and fluorescence images.

Compared with the prior art, the invention has the following advantages and beneficial effects:

1. the invention realizes the integration of four imaging methods of optical imaging, photoacoustic imaging, OCT imaging and fluorescence imaging, and can carry out real-time high-resolution confocal imaging. Compared with other technologies, the OCT technology has the characteristics of non-invasiveness, high resolution and capability of detecting the biological tissue structure in vivo, and the OCT endoscopic imaging technology combined with the endoscopic technology can perform in-vivo high-resolution imaging on a biological tissue cavity to obtain tissue structure information. The OCT technology can be used for structural detection of malignant tumor tissue lesions, when the tissue structure changes, the tissue scattering coefficient can change, and meanwhile, the change of the tissue structure in the treatment process can be monitored, so that the purpose of accurate treatment is achieved. Fluorescence imaging is to detect fluorescence signals after fluorescent substances are excited to reconstruct fluorescence images, and fluorescent molecular targeted imaging has high sensitivity and specificity, can mark tumor boundaries, provides important reference for delineating biological target regions in the treatment process, and can be applied to real-time positioning of focus and real-time detection of tumor margin residues. The photoacoustic imaging technology combines the characteristics of high optical imaging resolution and deep acoustic imaging depth, can image a tumor microvascular network, judges the tumor infiltration depth, provides important reference for judging the tumor stage, and can be applied to postoperative screening, imaging edge area microstructure morphology and judging whether the incisal edges have residues. Compared with the traditional white light endoscope, the multi-mode endoscope device has the characteristics of information integration, tomography, high resolution and deep imaging depth, is suitable for in-vivo diagnosis of various malignant tumors, can obtain medical images of tissues and organs by combining the advantages of various imaging devices, particularly provides a multi-dimensional data detection device and method for the field of medical images, and can be used for early detection of nasopharyngeal carcinoma and preoperative and postoperative image evaluation.

2. The combination of the light-transmitting and sound-reflecting mirror and the rod lens in the multi-mode endoscope probe can realize optical beam combination and optical path multiplexing, realize real-time high-resolution confocal imaging of a target tissue area, and simultaneously acquire a photoacoustic image, an OCT (optical coherence tomography) image, a fluorescence image and a white light image. The photoacoustic, OCT and fluorescence excitation light is focused and enters the tissue to be imaged through the rod lens, and the working distance of the endoscope can be prolonged due to the existence of the rod lens. The photoacoustic excitation light is incident to biological tissues, echo signals are reflected by a light-transmitting anti-sound mirror and received by a lateral transducer, and OCT backscattered light and fluorescence signals are returned by an incident light path.

3. The multimode endoscope probe part realizes multimode confocal by using a focusable collimator. Due to the difference of the wavelengths of the photoacoustic excitation light, the OCT excitation light and the fluorescence excitation light, dispersion can occur when the light passes through the same lens, and the image resolution is reduced. By using the focusing collimator, the position of the light spot focus incident to the tissue is changed, the confocal point of three imaging modes of optoacoustic, OCT and fluorescence is realized, and the image resolution is improved.

4. The multimode endoscope probe internal scanning structure adopts an MEMS mirror. Compared with galvanometer scanning, the MEMS mirror can reduce the volume of the endoscope, lighten the mass of the endoscope and simultaneously has high reflectivity to incident light.

5. The lens structure in the multi-mode endoscope probe can change the focus positions of the photoacoustic system and the OCT system, the working distance of the multi-mode imaging system is prolonged, the endoscope probe can go deep into narrow cavity organs to carry out in-vivo detection, and the working distance of the system can be changed by adjusting the number of the lens groups, so that high-resolution confocal endoscopic imaging of cavity organ tissues at different depths is realized.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic structural diagram of an endoscope device integrating optics, optoacoustic, OCT and fluorescence multi-mode forward imaging.

Fig. 2 is a schematic diagram of an imaging device of an endoscope device integrating optical, photoacoustic, OCT and fluorescence multi-mode forward imaging.

Fig. 3 is a structural schematic diagram of an endoscope device which integrates optical, photoacoustic, OCT and fluorescence multi-mode forward imaging.

The reference numbers illustrate: 1. a computer; 2. a second acquisition card; 3. a second signal amplifier; 4. a first acquisition card; 5. an FPGA development board; 6. a third acquisition card; 7. a photomultiplier tube; 8. a first signal amplifier; 9. a white light source; 10. a superluminescent light emitting diode; 11. a fourth collimator; 12. a spectrometer; 13. a second focusing lens; 14. a coupler; 15. an integrated endoscope; 16. a third collimator; 17. a second reflector; 18. a pulsed laser; 19. a second dichroic mirror; 20. A fiber coupler; 21. a charge-coupled element; 22. an eyepiece; 23. a beam splitter; 24. a MEMS mirror; 25. a first collimator; 26. a second collimator; 27. an optical fiber; 28. a first reflector; 29. a first focusing lens; 30. a first dichroic mirror; 31. a rod lens; 32. an ultrasonic transducer; 33. light-transmitting and sound-reflecting mirror.

Detailed Description

In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

As shown in fig. 2, the embodiment discloses an optical, photoacoustic, OCT, and fluorescence multi-modal forward imaging integrated endoscope apparatus according to the present invention, which includes a control system, a light source system, an optical path system, an endoscope probe, and a data acquisition and reconstruction system.

The control system takes the FPGA development board 5 as a control module and generates time sequence pulses to control each imaging submodule; the light source system comprises a pulse laser 18, a continuous laser and a white light source 9; the optical path system comprises an optical fiber coupler 20, a single-mode optical fiber, a second dichroic mirror 19, a second focusing lens 13, a coupler 14, a third collimator 16, a fourth collimator 11, a second reflecting mirror 17, a spectrometer 12 and a photomultiplier 7; the endoscope probe is composed of an integrated endoscope 15 and comprises a charge coupled device 21(CCD), an eyepiece 22, a spectroscope 23, a first collimator 25, a second collimator 26, a first dichroic mirror 30, a first focusing lens 29, a first reflector 28, an MEMS mirror 24, a rod lens 31, a light guide fiber 27, an ultrasonic transducer 32, a light-transmitting and sound-reflecting mirror 33 and an endoscope shell; the data acquisition and reconstruction system comprises a first signal amplifier 8, a first acquisition card 4, a second signal amplifier 3, a second acquisition card 2, a third acquisition card 6 and a computer 1.

Furthermore, the control system takes an FPGA5 development board as a control module, and generates time sequence pulses to control each imaging submodule; each imaging sub-module comprises an optical imaging module, a photoacoustic imaging module, an OCT imaging module and a fluorescence imaging module.

Further, in the light source system, the continuous laser and the white light source control end are connected with the FPGA development board 5 for controlling the output of the light source, and the light source output end is connected to the light path system through an optical fiber.

Further, in the optical path system, the output light of the pulse laser 18 is reflected by the second dichroic mirror 19 and enters the fiber coupler 20, the fiber coupler 20 is connected with a second collimator 26 in the endoscope probe through a single-mode fiber, the fluorescence excitation light is emitted and incident on the second dichroic mirror 19 through the fiber coupler 20, and the output light of the second dichroic mirror 19 is focused by the second focusing lens 13 and incident on the photocathode of the photomultiplier tube 7; the continuous laser output light is connected with a coupler 14 through a single mode fiber, the coupler 14 divides the input light into two parts, one part enters the endoscope probe through a first collimator 25, the other part enters a second reflector 17 through a third collimator 16 in a collimation mode, sample arm return light emitted by the first collimator 25 interferes with reflected light from the third collimator 16 in the coupler 14, and the sample arm return light is emitted out of an incident spectrometer through a fourth collimator 11. The white light source in the light source system is connected to the side wall optical fiber of the endoscope probe through the single mode optical fiber for tissue illumination.

Further, in the endoscope probe; the output light of the pulse laser 18 enters the integrated endoscope 15 through the second collimator 26, horizontally enters the integrated endoscope through the first dichroic mirror 30, the output light of the continuous laser enters the integrated endoscope through the first collimator 25, is reflected by the first dichroic mirror 30 and then is combined with the pulse light, the combined light is focused through the first focusing lens 29 and then enters the first reflecting mirror 28, and then enters the surface of the MEMS mirror 24 after being reflected, and the scanning light enters the rod lens 31 after being reflected by the beam splitter 23 and then is transmitted inside the rod lens 31 and passes through the light-transmitting anti-sound mirror and enters the surface of the tissue.

Further, the data acquisition and reconstruction system comprises a first signal amplifier 8, a first acquisition card 4, a second signal amplifier 3, a second acquisition card 2, a third acquisition card 6 and a computer 1, and the multi-mode imaging system simultaneously images white light, photoacoustic, OCT and fluorescence images. In photoacoustic imaging, pulse laser is incident to the surface of a tissue, ultrasonic waves generated by the surface stimulation are received by a side wall transducer of an endoscope probe, signals are amplified by a first signal amplifier and then enter a first acquisition card 4, and the first acquisition card sends original data to a computer; in OCT imaging, a spectrometer receives interference signals and sends the interference signals to a computer 1 by a third acquisition card 6; in fluorescence imaging, the photomultiplier 7 converts a fluorescence signal into a current signal, and the current signal is converted into a voltage signal by the second signal amplifier 3, and the voltage signal is sent to the computer 1 by the second acquisition card 2.

In the embodiment, the multi-modality imaging system can simultaneously image white light, photoacoustic, OCT and fluorescence images, and the imaging process is completed by the optical imaging module, the photoacoustic imaging module, the OCT imaging module and the fluorescence imaging module.

The optical imaging module comprises a Charge Coupled Device (CCD) 21, an ocular lens 22, a spectroscope 23, a rod lens 31, a light-transmitting anti-acoustic mirror 33, an optical fiber 27, an FPGA development board 5 in a control system, a white light source 9 in a light source system and a computer 1 in a data acquisition and reconstruction system in the endoscope probe.

The photoacoustic imaging module comprises a second collimator 26, a first dichroic mirror 30, a first focusing lens 29, a first reflecting mirror 28, an MEMS mirror 24, a beam splitter 23, a rod lens 31, a light-transmitting and sound-reflecting mirror 33, an ultrasonic transducer 32, an FPGA development board 5 in a control system, a pulse laser 18 in a light source system, a second dichroic mirror 19, an optical fiber coupler 20 and a first collimator 26 in a light path system, a first signal amplifier 8, a first acquisition card 4 and a computer 1 in a data acquisition and reconstruction system.

The OCT imaging module comprises a first collimator 25, a first dichroic mirror 30, a first focusing lens 29, a first reflecting mirror 28, an MEMS mirror 24, a beam splitter 23, a rod lens 31, a light-transmitting and sound-reflecting mirror 33 and an FPGA development board 5 in a control system in the endoscope probe, a continuous laser 10 in a light source system, a coupler 14, a fourth collimator 11, a spectrometer 12, a third collimator 16, a second reflecting mirror 17 and a second collimator 25 in a light path system, a third acquisition card 6 in a data acquisition and reconstruction system and a computer 1.

The fluorescence imaging module comprises a second collimator 26, a first dichroic mirror 30, a first focusing lens 29, a first reflecting mirror 28, an MEMS mirror 24, a spectroscope 23, a rod lens 31, a light-transmitting and sound-reflecting mirror 33 and an FPGA development board 5 in a control system in the endoscope probe, a pulse laser 18 in a light source system, a second dichroic mirror 19, an optical fiber coupler 20, a first collimator 26, a second focusing lens 13 and a photomultiplier 7 in a light path system, a second amplifier 3, a second acquisition card 2 and a computer 1 in a data acquisition and reconstruction system.

In this embodiment, the second dichroic mirror is a long-wavelength pass dichroic mirror, and the cutoff wavelength is 550 nm. And the addition of the second dichroic mirror enables the fluorescence imaging and the photoacoustic imaging to share the optical path. The wavelength of the fluorescence signal and the wavelength of the photoacoustic signal excitation light are 527nm, the excitation light is reflected by the dichroic mirror to enter the optical fiber coupler, the wavelength of the fluorescence signal is higher than that of the fluorescence excitation light, the fluorescence signal is emitted by the optical fiber coupler to pass through the dichroic mirror, light with the wavelength lower than the cut-off wavelength of the dichroic mirror is reflected, and the fluorescence signal is focused by the focusing lens to enter the photomultiplier, so that simultaneous imaging of photoacoustic imaging and fluorescence imaging can be realized.

In the embodiment, rhodamine B is selected as the fluorescent dye, the laser wavelength emitted by the pulse laser is 527nm, and the laser wavelength emitted by the superluminescent diode is 1310 nm. The pulse laser is a semiconductor laser, a solid laser, a dye laser or a gas laser, the wavelength range of the output pulse laser is 400 nm-2500 nm, and the pulse width is 5 ns-50 ns.

In this embodiment, the light guide fiber on the side wall of the endoscope probe is used to transmit illumination white light, the reflected light from the tissue surface passes through the spectroscope via the rod lens, the biconvex lens of the eyepiece group is used to focus the white light, and the CCD collects white light images.

In this embodiment, the first dichroic mirror is a short-wave pass dichroic mirror, and the cutoff wavelength is 950 nm. The addition of the first dichroic mirror enables fluorescence imaging, photoacoustic imaging and OCT imaging to share a light path, the wavelength difference between fluorescence and photoacoustic signal exciting light and OCT exciting light is large, and by selecting the dichroic mirror with proper cut-off frequency, the fluorescence and photoacoustic signal exciting light can be reflected and transmitted through the OCT signal exciting light, so that beam combination of space light is realized.

In the embodiment, pulse laser and continuous laser inside the endoscope probe are combined by the dichroic mirror, focused by the laser focusing light path and reflected to the surface of the MEMS mirror by the reflecting mirror, and the control system controls the MEMS mirror to deflect and adjust the scanning range.

In this embodiment, the probe is guided to the tissue region to be imaged by white light for scanning, and the endoscope probe adopts a forward scanning mode. The working distance can be prolonged by adopting the rod lens, the photoacoustic echo signal can be reflected by adding the light-transmitting and sound-reflecting mirror at the front end of the probe and received by the side-wall high-frequency ultrasonic transducer, the main frequency of the high-frequency ultrasonic transducer is 40M, and the bandwidth is 80%.

In the implementation, the addition of the adjustable-focus collimator can realize the multi-mode imaging confocal point. The first collimator is a focusing collimator, because the wavelength difference between the photoacoustic and fluorescence signal exciting light and the OCT signal exciting light is large, dispersion can occur when the photoacoustic and fluorescence signal exciting light and the OCT signal exciting light pass through the same lens, and by adjusting the focusing collimator, the multimode imaging confocal point is realized, and clear images under different imaging modes can be obtained at the same position.

In this embodiment, the first signal amplifier has an amplification gain of 50dB, and the second signal amplifier has a gain of 46 dB.

In this embodiment, the endoscope housing is a biocompatible metal housing, the working distance is 20cm, and the diameter of the endoscope is 3 mm.

In order to achieve the second object, the invention adopts the following technical scheme:

the invention relates to a method for imaging by using a four-mode imaging device, which comprises the following steps:

triggering a system: the control system generates a trigger signal to control the pulse laser and the continuous laser to generate laser, and simultaneously controls the illumination light source, the pulse laser penetrates through the dichroic mirror to enter the optical fiber coupler and enter the endoscope through the collimator, the continuous laser enters the endoscope through the coupler and the collimator, and the continuous laser and the pulse laser are reflected by the dichroic mirror to form a beam. The combined beam light is reflected by a reflecting mirror and focused by a focusing lens to finally reach the surface of the tissue to be imaged. The illumination light illuminates the tissue surface to be imaged through the optical fiber.

Data acquisition: photoacoustic data acquisition: the photoacoustic signal excited by the pulse laser is reflected by the light-transmitting sound-reflecting mirror, is received by the ultrasonic transducer on the side wall of the endoscope and is converted into an electric signal, the electric signal enters the first amplifier through the coaxial cable, and the amplified electric signal is collected by the first acquisition card and is sent to the computer. OCT data acquisition: the OCT signal excited by the continuous laser is backward scattered light from the surface of the tissue to be imaged, is received by the endoscope, is emitted along an incident light path, enters the coupler through the first collimator, interferes with reflected light from the reference arm, is detected by the spectrometer, is converted into an electric signal, is collected by the third collecting card and is sent to the computer. Fluorescence data acquisition: the fluorescence signal excited by the pulse laser is received by the endoscope, is emitted along an incident light path, enters the optical fiber coupler through the second collimator, is focused by the focusing lens through the dichroic mirror, enters the photomultiplier, converts the fluorescence signal into an electric signal, passes through the second amplifier, is collected by the second acquisition card and is sent to the computer. Optical data acquisition: the optical signal is received by the endoscope, enters the CCD through the rod lens group and the spectroscope, the CCD can convert the incident light signal into electric charge to be output, and the electric signal enters the computer through the data line.

Image processing: the computer uses the collected data for the reconstruction of optical images, photoacoustic images, OCT images and fluorescence images.

It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.

The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such changes, modifications, substitutions, combinations, and simplifications are intended to be included in the scope of the present invention.

Claims (10)

1. An endoscope device integrating optics, optoacoustic, OCT and fluorescence multi-mode forward imaging is characterized by comprising a control system, a light source system, a light path system, an endoscope probe and a data acquisition and reconstruction system, wherein the control system, the light source system, the light path system, the endoscope probe and the data acquisition and reconstruction system are respectively connected with the light source system and the data acquisition system, the light source system, the light path system and the endoscope probe are sequentially connected, and the data acquisition system is also connected with the endoscope probe and the data reconstruction system;

the control system takes an FPGA development board as a control module and generates time sequence pulses to control each imaging submodule;

the light source system comprises a pulse laser, a continuous laser and a white light source; the continuous laser and the white light source control end are connected with the FPGA development board and used for controlling the output of the light source, and the light source output end is connected to the light path system through an optical fiber;

the optical path system comprises an optical fiber coupler, a single-mode optical fiber, a second dichroic mirror, a second focusing lens, a coupler, a third collimator, a fourth collimator, a second reflecting mirror, a spectrometer and a photomultiplier; the output light of the pulse laser is reflected by the second dichroic mirror to enter the optical fiber coupler, the optical fiber coupler is connected with a second collimator in the endoscope probe through a single-mode optical fiber, the fluorescence excitation light is emitted out through the optical fiber coupler and enters the second dichroic mirror, and the emergent light of the second dichroic mirror is focused by the second focusing lens and enters a photoelectric cathode of the photomultiplier; the continuous laser output light is connected with a coupler through a single-mode optical fiber, the coupler divides the input light into two parts, one part enters an endoscope probe through a first collimator, the other part enters a second reflector through the collimation of a third collimator, the sample arm return light emitted by the first collimator interferes with the reflected light from the third collimator in the coupler, and the sample arm return light is emitted out of an incident spectrometer through a fourth collimator; a white light source in the light source system is connected to the side wall optical fiber of the endoscope probe through a single mode optical fiber for tissue illumination;

the endoscope probe is composed of an integrated endoscope and comprises a Charge Coupled Device (CCD), an ocular, a spectroscope, a first collimator, a second collimator, a first dichroic mirror, a first focusing lens, a first reflector, an MEMS mirror, a rod-shaped lens, an optical fiber, an ultrasonic transducer, a light-transmitting and sound-reflecting mirror and an endoscope shell; the output light of the pulse laser enters the integrated endoscope through the second collimator, horizontally enters the integrated endoscope through the first dichroic mirror, the output light of the continuous laser enters the integrated endoscope through the first collimator, is reflected by the first dichroic mirror and then is combined with the pulse light, the combined light is focused by the first focusing lens and then enters the first reflecting mirror, is reflected and then enters the surface of the MEMS mirror, the scanning light is reflected by the beam splitter and then enters the rod-shaped lens, and the scanning light is transmitted inside the rod-shaped lens and passes through the light-transmitting anti-sound mirror and enters the surface of the tissue;

the data acquisition and reconstruction system comprises a first signal amplifier, a first acquisition card, a second signal amplifier, a second acquisition card, a third acquisition card and a computer, and the multi-mode imaging system simultaneously images white light, opto-acoustic, OCT and fluorescence images; in photoacoustic imaging, pulse laser is incident to the surface of a tissue, ultrasonic waves generated by the surface through stimulation are received by a side wall transducer of an endoscope probe, signals are amplified by a first signal amplifier and then enter a first acquisition card, and the first acquisition card sends original data to a computer; in OCT imaging, a spectrometer receives interference signals and sends the interference signals to a computer by a third acquisition card; in fluorescence imaging, a photomultiplier converts a fluorescence signal into a current signal, the current signal is converted into a voltage signal through a second signal amplifier, and a second acquisition card transmits the voltage signal to a computer.

2. The integrated optical, photoacoustic, OCT, and fluorescent multi-modal forward imaging endoscope apparatus according to claim 1, wherein the first dichroic mirror is used to share a light path for fluorescence imaging, photoacoustic imaging, and OCT imaging, the wavelength of the fluorescence and photoacoustic signal excitation light is different from that of the OCT excitation light by a relatively large amount, and the first dichroic mirror with a cutoff frequency of 850nm to 1180nm is selected to reflect the OCT signal excitation light and transmit the fluorescence and photoacoustic signal excitation light, thereby combining the spatial light.

3. The integrated optical, photoacoustic, OCT, and fluorescence multi-modality forward imaging endoscope apparatus according to claim 1, wherein said first dichroic mirror is a short-wavelength-pass dichroic mirror with a cutoff wavelength of 850nm to 1180 nm.

4. The integrated optical, photoacoustic, OCT, and fluorescence multi-modality forward imaging endoscope apparatus according to claim 1, wherein the second dichroic mirror is used to make the fluorescence imaging and the photoacoustic imaging share a common optical path, the fluorescence signal and the photoacoustic signal excitation light are reflected by the second dichroic mirror into the fiber coupler, the fluorescence signal has a higher wavelength than the fluorescence excitation light, the fluorescence signal exits from the fiber coupler through the second dichroic mirror, and the light with a wavelength lower than the cut-off wavelength of the second dichroic mirror is reflected, and the fluorescence signal is focused by the focusing lens into the photomultiplier, so as to realize the simultaneous imaging of the photoacoustic imaging and the fluorescence imaging.

5. The integrated optical, photoacoustic, OCT, and fluorescence multi-modal forward imaging endoscope apparatus according to claim 4, wherein said second dichroic mirror is a long-wave pass dichroic mirror with a cut-off wavelength of 550nm to 780 nm.

6. The integrated optical, photoacoustic, OCT, and fluorescence multi-modality forward imaging endoscope apparatus according to claim 1, wherein the pulsed laser and continuous laser inside the endoscope probe are combined by the first dichroic mirror, focused by the laser focusing optical path and reflected by the mirror to the surface of the MEMS mirror, and the control system controls the MEMS mirror to deflect and adjust the scanning range, so that compared with galvanometer scanning, the MEMS mirror reduces the volume of the endoscope, reduces the mass of the endoscope, and has high reflectivity for incident light.

7. The integrated optical, photoacoustic, OCT, and fluorescence multi-modal forward imaging endoscope apparatus according to claim 1, wherein the first collimator is a focusing collimator, and since the wavelength difference between the photoacoustic and fluorescence signal excitation light and the OCT signal excitation light is large, the first collimator will disperse when passing through the same lens, and the focusing collimator is adjusted to realize a multi-modal imaging confocal point and obtain a clear image in different imaging modes at the same position.

8. The integrated optical, photoacoustic, OCT, and fluorescent multi-modal forward imaging endoscope apparatus according to claim 1, wherein said pulsed laser is one of a semiconductor laser, a solid laser, a dye laser, or a gas laser, and has an output pulsed laser wavelength range of 400nm to 2500nm and a pulse width of 5ns to 50 ns.

9. The integrated optical, photoacoustic, OCT, and fluorescence multi-modality forward imaging endoscope apparatus according to claim 1, wherein said first signal amplifier has an amplification gain of 50dB to 60dB and said second signal amplifier has a gain of 46 dB.

10. An imaging method of an integrated optical, photoacoustic, OCT, fluorescence multi-modality forward imaging endoscope apparatus according to any of claims 1-9, characterized by comprising the steps of:

(1) triggering a system: the control system generates a trigger signal to control the pulse laser to generate pulse light, the continuous laser generates continuous light and controls the illumination light source at the same time, the pulse laser enters the optical fiber coupler through the second dichroic mirror and enters the endoscope through the second collimator, the continuous laser enters the endoscope through the first collimator through the coupler and is reflected by the first dichroic mirror to form a beam with the pulse laser; the combined beam light is reflected by a reflecting mirror and focused by a focusing lens to finally reach the surface of the tissue to be imaged, and the illumination light irradiates the surface of the tissue to be imaged through an optical fiber;

(2) data acquisition: photoacoustic data acquisition: the photoacoustic signal excited by the pulse laser is reflected by a light-transmitting sound reflector, is received by an ultrasonic transducer on the side wall of the endoscope and is converted into an electric signal, the electric signal enters a first amplifier through a coaxial cable, and the amplified electric signal is collected by a first acquisition card and is sent to a computer; OCT data acquisition: the OCT signal excited by the continuous laser is backward scattered light from the surface of the tissue to be imaged, is received by an endoscope, is emitted along an incident light path, enters a coupler through a first collimator, interferes with reflected light from a reference arm, is detected by a spectrometer for interference signal intensity, is converted into an electric signal at the same time, is collected by a third collecting card and is sent to a computer; fluorescence data acquisition: the fluorescence signal excited by the pulse laser is received by the endoscope, is emitted along an incident light path, enters the optical fiber coupler through the second collimator, is focused by the focusing lens through the second dichroic mirror, then enters the photomultiplier, converts the fluorescence signal into an electric signal, passes through the second amplifier, is collected by the second collection card and is sent to the computer; optical data acquisition: the optical signal is received by the endoscope, enters the CCD through the spectroscope via the rod lens group, can convert the incident light signal into electric charge to be output by the CCD, and enters the computer through the data line;

(3) image processing: the computer uses the collected data for the reconstruction of optical images, photoacoustic images, OCT images and fluorescence images.

Images