CN106859579A - A kind of fibre bundle confocal fluorescent endoscopic imaging method and device based on sub-pix - Google Patents

Abstract

The present invention discloses a kind of fibre bundle confocal fluorescent endoscopic imaging method based on sub-pix, including：Illumination light by single-mode fiber outgoing and is focused on sample successively coupled to different single-mode fibers in fibre bundle, is collected the fluorescence that sample produces and is simultaneously focused on and in incident illumination light identical single-mode fiber, and collection fluorescence signal obtains scanning the first two field picture of sample；Then, control the nearly sample end at least movement of fibre bundle once, at position after each movement, illumination light is successively coupled to single-mode fiber different in fibre bundle, by single-mode fiber outgoing and focus on sample, collect the fluorescence of sample generation and focus on and in incident illumination light identical single-mode fiber, gather the second two field picture that fluorescence signal obtains scanning sample；The second two field picture obtained after each displacement and the first two field picture are carried out into image reconstruction, final micro-image is obtained.Invention additionally discloses a kind of fibre bundle confocal fluorescent based endoscopic imaging device based on sub-pix.

Description

A method and device for confocal fluorescence endoscopic imaging of optical fiber bundles based on sub-pixels

technical field

The invention belongs to the field of confocal endoscopic microscopy, in particular to a method and device for confocal fluorescent endoscopic imaging of optical fiber bundles based on sub-pixels.

Background technique

More than ten years ago, most of the applications of optical fiber were mainly in the field of communication. Afterwards, optical fibers were gradually accepted as illumination and detection devices, and were gradually applied to scanning confocal systems. Because of the flexibility of optical fiber devices, special optical fibers have been gradually invented and produced, gradually replacing traditional optical devices in some occasions, and also produced new and unique optical systems. These new optics also bring new advances and advantages to the field of intravital imaging.

The optical fiber confocal endoscopic imaging method combines laser confocal microscopic imaging technology and endoscopic imaging technology. It can not only take advantage of the flexibility of the optical fiber, but also retain the characteristics of high resolution, high contrast and tomography of the confocal microscope. The use of fiber optic bundles to deliver the illumination light and transmit the image allows the scanning mechanism to be placed at the proximal end of the optical system. Fiber optic bundle based systems can be inserted into the working channel of conventional endoscopes due to the smaller size of the distal end. During endoscopic examinations , with the help of fluorescent staining, the structural information of the inspected part is given in real time in vivo. This information can provide an important aid in the diagnosis of disease, localizing the outline of diseased tissue before tissue biopsy and surgery.

In a traditional confocal endoscopic microscope device based on fiber bundles (as shown in Figure 1), it includes a laser 1, a first filter 2, a dichroic mirror 3, a two-axis scanning galvanometer 4, a first Lens 5, second lens 6, objective lens 7, optical fiber bundle 8, microscope objective lens 9, second optical filter 11, photodetector 12 and computer 13.

Using the device shown in Figure 1 to realize the confocal endoscopic microscope based on the fiber bundle, the process is as follows:

(1) The laser 1 emits illuminating light, after the stray light is filtered out by the first filter 2, it is reflected by the dichroic mirror 3 and the biaxial scanning galvanometer 4 to the beam expander system, and the beam expander system is formed by the first lens 5 Composed of the second lens 6, then coupled to an optical fiber in the fiber bundle 8 by the coupling objective lens 7;

(2) The light emitted from the illumination fiber is focused on the fluorescent sample 10 through the microscope objective lens 9;

(3) After the laser illuminates the fluorescence sample 10, the sample is excited to generate fluorescence, which is collected by the microscope objective lens 9 and then focused into the same illumination fiber. The fluorescence emitted by the fiber passes through the coupling objective lens 7, the second lens 6 and the first lens 5 , and reflected by the biaxial scanning galvanometer 4 to the dichroic mirror 3, the fluorescence passes through the dichroic mirror 3, and then filtered by the second filter 11 to filter out the laser light and other stray light reflected by the sample, Only the fluorescence is emitted and collected by the photodetector 12; the photodetector 12 converts the optical signal into an electrical signal, and transmits the electrical signal to the computer 13 to obtain an image corresponding to an object point;

(4) The computer 13 controls the dual-axis scanning galvanometer 4 to couple the exciting illumination light into different optical fibers in the optical fiber bundle 8, and after all the optical fibers are scanned, an image is obtained.

However, in the fiber-optic confocal endoscopic imaging system, the honeycomb arrangement of the fiber bundle itself will bring inherent noise to the image. Since the number of optical fibers limits the number of pixels, which is insufficient to satisfy Nyquist's law, the resulting image resolution will be reduced under the influence of aliasing effects, which is the most common problem in imaging systems. The optical fiber bundle discretizes the continuous images, and each pixel on the imaging surface only represents the light intensity near it. There is a small distance between two pixels, which can be seen as being connected together macroscopically, but there are infinitely smaller things between them on the microscopic level, and this smaller thing is called "sub-pixel". Pixel (Sub-Pixel)", as shown in Figure 2. The key to this problem is to improve the image quality and minimize the blurring of image details when removing the fiber bundle pattern.

Contents of the invention

The present invention provides a method and device for confocal fluorescence endoscopic imaging of optical fiber bundles based on sub-pixels, and proposes that on the basis of confocal endoscopic microscopy based on optical fiber bundles, a dual-axis galvanometer is used to scan the laser beam, The laser beam emerges from the far end of the optical fiber, and is focused on the sample through the distal micro-lens. At the same time, a moving platform is added to the far end of the optical fiber bundle to increase the overall movement of the optical fiber bundle, and the images obtained after each movement are superimposed and processed. In order to obtain high-pixel images and improve resolution, this method is called sub-pixel (Sub-Pixel), which can be used in the field of confocal endoscopic microscopy.

The present invention utilizes the concept of sub-pixel (Sub-Pixel) to improve image quality and obtain high-definition sample images. Compared with other confocal endoscopic microscopes, the device has a simple structure and is easy to operate, and provides a good research method for the field of intravital endoscopic imaging.

Concrete technical scheme of the present invention is as follows:

A sub-pixel-based confocal fluorescence endoscopic imaging method for optical fiber bundles, including: the illumination light is sequentially coupled to different single-mode optical fibers in the optical fiber bundle, emitted from the single-mode optical fibers and focused onto the sample, and the fluorescence generated by the sample is collected and focused Into the same single-mode optical fiber as the incident illumination light, collect the fluorescence signal to obtain the first frame image of the scanned sample;

Control the near-sample end of the fiber bundle to move at least once. At the position after each move, the illumination light is sequentially coupled to different single-mode fibers in the fiber bundle, emitted from the single-mode fiber and focused on the sample, and the fluorescence generated by the sample is collected. And focus it into the same single-mode fiber as the incident illumination light, collect the fluorescence signal to obtain the second frame image of the scanned sample;

Image reconstruction is carried out between the second frame image obtained after each displacement and the first frame image to obtain the final microscopic image.

Further, the near-sample end of the optical fiber bundle is controlled to move in the Z-axis direction to scan the sample in three dimensions.

In the present invention, the more times the near-sample end of the fiber bundle moves, the more the number of second frame images is obtained, which is more conducive to obtaining high-resolution images. Further, the number of times that the end near the sample of the optical fiber bundle moves is at least 2 times, and the position of each movement is different; further, the number of times that the end near the sample of the optical fiber bundle moves is 2 to 36 times.

The present invention also provides a sub-pixel optical fiber bundle confocal fluorescence endoscopic imaging device, which includes for fluorescent samples:

A laser is used to emit excitation light to realize illumination and excitation of fluorescent samples;

The dual-axis scanning galvanometer is used to realize the scanning of the laser beam, that is, to sequentially couple the excitation into different single-mode fibers in the fiber bundle to complete the two-dimensional plane scanning of the sample.

a first lens for coupling excitation illumination light into a single optical fiber;

The optical fiber bundle is composed of a plurality of single-mode optical fibers, and each single-mode optical fiber on the section is arranged in a honeycomb shape (as shown in Figure 4), which is used to emit excitation illumination light and receive fluorescent signals;

The second lens is used to converge the excitation illumination light onto the sample surface, and collect the fluorescence emitted by the fluorescent sample after being excited;

The dichroic mirror is used to transmit the excitation light and the backscattered light generated by the excitation light irradiating the sample, and reflect the fluorescence excited by the sample;

Optical filter, used to filter out the laser light reflected by the sample surface, and only allow the fluorescence emitted by the fluorescent sample to participate in imaging;

The second lens is used for converging the fluorescence emitted by the fluorescent sample onto the photodetector;

A mobile platform that can realize a small displacement, and is used to move the optical fiber bundle as a whole to obtain multiple frames of images at different positions; Electromobility platform or nanotranslation stage.

The photodetector converts the light signal detected at the detection pinhole into an electrical signal and transmits it to the computer;

The computer is used to process the signal of the detector, and simultaneously control the two-axis scanning galvanometer and the micro-displacement mobile platform to complete the three-dimensional plane scanning of the sample.

In the present invention, the mobile platform is used to control the near-sample end of the optical fiber bundle, the position of each movement is different, and the number of movements can be set as required.

The specific implementation steps of the optical fiber bundle confocal fluorescence endoscopic imaging device are as follows:

(1) The laser emits an illuminating beam, passes through a dichroic mirror to a two-axis scanning mirror, and then couples it into a single optical fiber through a lens. The excitation light exits from the other end of the optical fiber and is focused onto the fluorescent sample through a micro lens , to excite the sample;

(2) After the fluorescent sample is excited to emit fluorescence, the obtained fluorescence is collected by a micro-lens and coupled into the same illumination fiber, and then reaches the proximal end of the fiber bundle through the fiber and is collected by the coupling objective lens to reach the main part of the system. The chromatic mirror realizes 90-degree turning to realize the separation from the excitation light. After being focused by the lens and filtering the stray light, it is received by the photodetector;

(3) The photodetector converts the optical signal into an electrical signal and transmits it to the computer, completing the reading and processing of the image information of the first scanning point;

(4) Scanning the laser beam is realized by controlling the dual-axis galvanometer through the computer, so that the sample can complete the scanning of the two-dimensional plane, that is, the first frame of image is obtained;

(5) The piezoelectric mobile platform where the far end of the fiber bundle is located is connected to the computer. After the computer controls the overall movement of the fiber bundle for a small displacement, the scanning is repeated to obtain several frames of images at different positions. After the computer superimposes the images, the final image;

(6) The computer controls the piezoelectric moving platform to move the optical fiber bundle in the Z-axis direction to complete the three-dimensional scanning of the sample.

Principle of the present invention is as follows:

On the basis of the general-purpose confocal endoscopic microscope device, the illumination beam emitted by the laser and transmitted through the dichroic mirror is coupled into a single-mode optical fiber, and the excitation illumination light emitted from the single-mode optical fiber is passed through The objective lens is focused on the surface of the fluorescent sample for total reflection, which excites the sample to emit fluorescence. The excited fluorescence is collected and coupled into the same illumination fiber, and the fluorescence collected by each fiber is imaged at the small hole of the photodetector. The dual-axis galvanometer controls the deflection of the laser beam, which is coupled into different single optical fibers to complete the two-dimensional plane scanning of the sample. Using the concept of sub-pixel (Sub-Pixel), by moving a small distance to collect images at different positions, the obtained optical fiber can realize high-resolution image reconstruction.

In the process of image acquisition, due to undersampling, there is such a relationship between the information recorded by each pixel of the obtained k-th frame image and the information recorded by each pixel under ideal sampling:

Where z _r is the light intensity information recorded by each pixel under ideal sampling, N=L ₁ N ₁ ×L ₂ N ₂ is the high-resolution image size; m=1,2,...,M, M=N ₁ × N ₂ is the size of the low-resolution image; the weight ω _k,m,r (θ _k ,h _k ,v _k ) represents the relationship between the r high-resolution pixels and the mth low-resolution pixel; θ _k , h _k , v _k represent image rotation angle, horizontal and vertical movement respectively.

That is to say, after the low-resolution pixels are moved or rotated relative to the fixed high-resolution pixels, a group of different "virtual" high-resolution pixels constitutes another low-resolution pixel; as shown in Figure 5 As shown, Figure 5(a) is a schematic diagram of another low-resolution pixel composed of "virtual" high-resolution pixels, where the shaded part is a low-resolution pixel; Figure 5(b) is Figure 5(a) Schematic diagram of the corresponding relationship between middle and low resolution pixels and high resolution pixels after rotation and translation. In order to obtain the high pixel information, the low pixel information can be calculated and processed by estimating the weight ω _k,m,r (θ _k ,h _k ,v _k ).

In order to obtain the value of each parameter in the weight, a cost function can be defined, and the high-resolution image is the value obtained when the cost function is minimized. Expressed in a formula that is:

Among them, C(z) is the cost function:

Where m=1,2,...,pM, p is the total number of frames of the image; the right part is a regularization term, the smaller the value, the smoother z is, so define the parameters α _i,j as:

By calculating the partial derivative of C(z), the minimum value of C(z) can be found, which is expressed by the formula:

Through an iterative method, the values of all high-resolution pixels can be calculated by knowing the values of the initial high-resolution pixels, namely:

Among them, ε ⁿ represents the step size of the n-order iteration, and the high-resolution pixels can be calculated by properly selecting the value of ε ⁿ . In this way, a high-resolution image is obtained.

Compared with the prior art, the present invention has the following beneficial technical effects:

(1) Compared with the original confocal endoscopic microscope, multi-frame images are obtained by moving the whole fiber bundle, and the number of pixels is increased.

(2) Realize high-resolution image reconstruction by sub-pixel method, improve image quality and eliminate noise.

(3) The structure of the device is simple and the data processing is convenient.

Description of drawings

Figure 1 is a schematic diagram of a traditional confocal endoscopic microscope device based on a fiber bundle;

Figure 2 is a conceptual schematic diagram of a sub-pixel (Sub-Pixel);

3 is a schematic diagram of a sub-pixel-based confocal fluorescence endoscopic imaging device for optical fiber bundles;

Fig. 4 is a cross-sectional view of an optical fiber bundle;

Figure 5(a) is a schematic diagram of another low-resolution pixel composed of "virtual" high-resolution pixels, where the shaded part is a low-resolution pixel; Figure 5(b) is the low-resolution pixel in Figure 5(a) Schematic diagram of the corresponding relationship between high-resolution pixels and high-resolution pixels after rotation and translation;

detailed description

The present invention will be described in detail below in conjunction with the embodiments and drawings, but the present invention is not limited thereto.

Example 1

As shown in Figure 3, a sub-pixel based optical fiber bundle confocal fluorescence endoscopic imaging device that realizes the movement of the optical fiber bundle by a piezoelectric moving platform includes a laser 1, a dichroic mirror 3, a biaxial scanning galvanometer 4, First lens 5 , fiber bundle 8 , piezoelectric moving platform 14 , second lens 15 , microscope objective 9 , sample 10 , second filter 16 , third lens 17 , photodetector 12 and computer 13 .

The process of confocal fluorescence endoscopic imaging method based on sub-pixel fiber bundles realized by the device shown in Figure 3 is as follows:

(1) The laser 1 emits illumination light, passes through the dichroic mirror 3 and the biaxial scanning galvanometer 4, and is coupled into an optical fiber in the optical fiber bundle 8 by the first lens 5;

(2) The light emitted from the illumination fiber becomes parallel light through the second lens 15, and then focuses on the fluorescent sample 10 through the microscope objective lens 9;

(3) After the laser illuminates the fluorescent sample 10, the sample is excited to generate fluorescence, which becomes parallel light after being collected by the microscope objective lens 9, and is focused into the same illumination optical fiber by the second lens 15, and the fluorescent light emitted by the optical fiber passes through the first lens 5 and the biaxial scanning galvanometer 4, and reflected by the dichroic mirror 3 to the second filter 16, after the second filter 16 filters the laser and other stray light reflected by the sample, only the fluorescence The emitted fluorescent light is converged by the third lens 17 and focused on the photodetector 12; the photodetector 12 converts the optical signal into an electrical signal, and transmits the electrical signal to the computer 13 to obtain an image corresponding to an object point ;

(4) The computer 13 controls the biaxial scanning galvanometer 4 to couple the exciting illumination light into different optical fibers in the fiber bundle 8, and after all the optical fibers are scanned, a frame of image is obtained;

(5) The piezoelectric mobile platform 14 is connected to the computer 13, and the piezoelectric mobile platform is controlled by the computer 13 to move the far end of the optical fiber bundle (near the sample end) twice, and repeat steps 1-4 at the position after each movement , to get 2 frames of images;

(6) The computer performs high-resolution image reconstruction on the obtained 4 frames of images and one frame of images at the initial position to obtain the final high-resolution image.

Example 2

As shown in Figure 3, a sub-pixel based optical fiber bundle confocal fluorescence endoscopic imaging device that realizes the movement of the optical fiber bundle by a piezoelectric moving platform includes a laser 1, a dichroic mirror 3, a biaxial scanning galvanometer 4, First lens 5 , fiber bundle 8 , piezoelectric moving platform 14 , second lens 15 , microscope objective 9 , sample 10 , second filter 16 , third lens 17 , photodetector 12 and computer 13 .

The process of confocal fluorescence endoscopic imaging method based on sub-pixel fiber bundles realized by the device shown in Figure 3 is as follows:

(1) The laser 1 emits illumination light, passes through the dichroic mirror 3 and the biaxial scanning galvanometer 4, and is coupled into an optical fiber in the optical fiber bundle 8 by the first lens 5;

(2) The light emitted from the illumination fiber becomes parallel light through the second lens 15, and then focuses on the fluorescent sample 10 through the microscope objective lens 9;

(3) After the laser illuminates the fluorescent sample 10, the sample is excited to generate fluorescence, which becomes parallel light after being collected by the microscope objective lens 9, and is focused into the same illumination optical fiber by the second lens 15, and the fluorescent light emitted by the optical fiber passes through the first lens 5 and the biaxial scanning galvanometer 4, and reflected by the dichroic mirror 3 to the second filter 16, after the second filter 16 filters the laser and other stray light reflected by the sample, only the fluorescence The emitted fluorescent light is converged by the third lens 17 and focused on the photodetector 12; the photodetector 12 converts the optical signal into an electrical signal, and transmits the electrical signal to the computer 13 to obtain an image corresponding to an object point ;

(4) The computer 13 controls the biaxial scanning galvanometer 4 to couple the exciting illumination light into different optical fibers in the fiber bundle 8, and after all the optical fibers are scanned, a frame of image is obtained;

(5) The piezoelectric mobile platform 14 is connected to the computer 13, and the piezoelectric mobile platform is controlled by the computer 13 to move the far end of the optical fiber bundle (near the sample end) four times, and repeat steps 1-4 at the position after each movement , get 4 frames of images;

(6) The computer performs high-resolution image reconstruction on the obtained two frames of images and one frame of the original image to obtain the final high-resolution image.

Example 3

As shown in Figure 3, a subpixel-based optical fiber bundle confocal fluorescence endoscopic imaging device that realizes the movement of the optical fiber bundle by a nano-translation stage includes a laser 1, a dichroic mirror 3, a two-axis scanning galvanometer 4, and A lens 5, an optical fiber bundle 8, a nano-translation stage 14, a second lens 15, a microscope objective lens 9, a sample 10, a second optical filter 16, a third lens 17, a photodetector 12 and a computer 13.

The process of confocal fluorescence endoscopic imaging method based on sub-pixel fiber bundles realized by the device shown in Figure 3 is as follows:

(1) The laser 1 emits illumination light, passes through the dichroic mirror 3 and the biaxial scanning galvanometer 4, and is coupled into an optical fiber in the optical fiber bundle 8 by the first lens 5;

(2) The light emitted from the illumination fiber becomes parallel light through the second lens 15, and then focuses on the fluorescent sample 10 through the microscope objective lens 9;

(3) After the laser illuminates the fluorescent sample 10, the sample is excited to generate fluorescence, which becomes parallel light after being collected by the microscope objective lens 9, and is focused into the same illumination optical fiber by the second lens 15, and the fluorescent light emitted by the optical fiber passes through the first lens 5 and the biaxial scanning galvanometer 4, and reflected by the dichroic mirror 3 to the second filter 16, after the second filter 16 filters the laser and other stray light reflected by the sample, only the fluorescence The emitted fluorescent light is converged by the third lens 17 and focused on the photodetector 12; the photodetector 12 converts the optical signal into an electrical signal, and transmits the electrical signal to the computer 13 to obtain an image corresponding to an object point ;

(4) The computer 13 controls the biaxial scanning galvanometer 4 to couple the exciting illumination light into different optical fibers in the fiber bundle 8, and after all the optical fibers are scanned, a frame of image is obtained;

(5) The nano-translation stage 14 is connected with the computer 13, and the piezoelectric mobile platform is controlled by the computer 13 to make the far end of the fiber bundle (near the sample end) move thirty-six times, and at the position after each movement, repeat 1-4 Step, obtain 36 frames of images;

(6) The computer performs high-resolution image reconstruction on the obtained 36 frames of images and one frame of images at the initial position to obtain the final high-resolution image.

The above descriptions are only examples of the preferred implementation of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention within.

Claims (7)

1. A subpixel-based optical fiber bundle confocal fluorescence endoscopic imaging method, including: the illumination light is sequentially coupled to different single-mode optical fibers in the optical fiber bundle, emitted from the single-mode optical fiber and focused onto the sample, and the fluorescence generated by the sample is collected And focus into the same single-mode optical fiber as the incident illumination light, collect the fluorescence signal to obtain the first frame image of the scanned sample; it is characterized in that: Control the near-sample end of the fiber bundle to move at least once. At the position after each move, the illumination light is sequentially coupled to different single-mode fibers in the fiber bundle, emitted from the single-mode fiber and focused on the sample, and the fluorescence generated by the sample is collected. And focus it into the same single-mode fiber as the incident illumination light, collect the fluorescence signal to obtain the second frame image of the scanned sample; Image reconstruction is carried out between the second frame image obtained after each displacement and the first frame image to obtain the final microscopic image. 2 . The optical fiber bundle confocal fluorescence endoscopic imaging method according to claim 1 , wherein the end near the sample of the optical fiber bundle is controlled to move in the Z-axis direction to perform three-dimensional scanning on the sample. 3 . 3. The optical fiber bundle confocal fluorescence endoscopic imaging method according to claim 1, characterized in that: the near-sample end of the optical fiber bundle moves at least twice, and the positions of each movement are different. 4 . The optical fiber bundle confocal fluorescence endoscopic imaging method according to claim 3 , wherein the number of times the end near the sample of the optical fiber bundle moves is 2 to 36 times. 5. A fiber bundle confocal fluorescence endoscopic imaging device based on sub-pixels, comprising: a laser for emitting illumination light, an optical fiber bundle for emitting excitation illumination light and receiving fluorescence signals, for coupling the illumination light into Scanning vibrating mirrors in each single-mode optical fiber in the fiber bundle, a photodetector for collecting fluorescent signals, and a computer for processing the output signal of the photodetector; it is characterized in that: A mobile platform that controls the near-sample end of the fiber bundle to move at least once; At each position, the illumination light is sequentially coupled into each single-mode fiber in the fiber bundle to scan and illuminate the sample, and the first frame image of the near-sample end of the fiber bundle at the initial position and the image after each position shift are collected by the photodetector. The second frame image; The computer performs image reconstruction on the second frame image obtained after each shift and the first frame image to obtain the final microscopic image. 6. The optical fiber bundle confocal fluorescence endoscopic imaging device according to claim 5, characterized in that: the moving platform is a piezoelectric moving platform or a nanometer controlled by a computer for moving the fiber bundle near the sample end. translation stage. 7. The optical fiber bundle confocal fluorescence endoscopic imaging device as claimed in claim 6, characterized in that: the number of times the moving platform controls the movement of the near sample end of the optical fiber bundle is at least 2 times, and the positions of each movement are different. same.