CN110840386A - Visible light and near-infrared fluorescence 3D common imaging endoscope system based on single detector - Google Patents
Visible light and near-infrared fluorescence 3D common imaging endoscope system based on single detector Download PDFInfo
- Publication number
- CN110840386A CN110840386A CN201911315523.4A CN201911315523A CN110840386A CN 110840386 A CN110840386 A CN 110840386A CN 201911315523 A CN201911315523 A CN 201911315523A CN 110840386 A CN110840386 A CN 110840386A
- Authority
- CN
- China
- Prior art keywords
- image
- lens
- visible light
- endoscope
- infrared fluorescence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00163—Optical arrangements
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
- A61B1/043—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances for fluorescence imaging
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Surgery (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Biomedical Technology (AREA)
- Optics & Photonics (AREA)
- Pathology (AREA)
- Radiology & Medical Imaging (AREA)
- Biophysics (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Endoscopes (AREA)
Abstract
A visible light and near infrared fluorescence 3D common imaging endoscope system based on a single detector belongs to the technical field of endoscope imaging systems. The problem of how to realize the simultaneous acquisition and display of visible light and near-infrared fluorescence 3D image is solved. The 3D co-imaging endoscopic imaging system comprises a visible light near-infrared excitation light source, a binocular endoscopic imaging system, an optical relay image transfer system, an image sensor module, an image processing and fusing module and a 3D image display system. The endoscope system receives two images with horizontal parallax through the RGB-NIR detector to realize the acquisition of 3D images, so that the simultaneous real-time acquisition of visible light color 3D images and near infrared fluorescence 3D images is realized. On one hand, the system structure is simpler, the volume is smaller, on the other hand, the state switching is carried out without interrupting the operation, the smoothness of the operation process is ensured, on the other hand, a doctor can intuitively feel the position and the size of the pathological change tissue, and the operation success rate is greatly improved.
Description
Technical Field
The invention belongs to the technical field of endoscope imaging systems, and particularly relates to a visible light and near infrared fluorescence 3D co-imaging endoscope system based on a single detector.
Background
The endoscope technology is a comprehensive technology integrating high-precision subjects such as optics, precision manufacturing, image processing, photoelectric information, materials, bioengineering and the like. The appearance of the endoscope is an important milestone in the medical technology development history, which enables minimally invasive surgical examination and operation, greatly reduces the harm of the operation to patients and effectively controls the operation risk. Endoscopes have since appeared to have seen a progression from rigid tube endoscopes, semi-curved endoscopes, fibre endoscopes to electronic endoscopes. In recent years, endoscopes that combine a high-definition video technology and a 3D video technology have been proposed one after another, and the image quality of the endoscopes has been dramatically improved.
With the development of medical level and endoscope technology, endoscopes are more and more widely applied in clinical operations. In the surgical clinical operation, the success rate of the operation can be greatly improved by simultaneously seeing the visible light color 3D image and the focus fluorescent mark. However, no design scheme of a visible light and near infrared fluorescence 3D co-imaging endoscopic imaging system is proposed at present. The current visible light and near-infrared fluorescence endoscope is switched in a working state, and cannot see a near-infrared fluorescence image when working in a visible light state and cannot see a visible light image when working in a near-infrared fluorescence state. The working mode brings trouble to the fluency of the operation and increases the operation time. And the product can only realize the acquisition of 2D images, and lacks depth information.
Disclosure of Invention
The invention provides a visible light and near infrared fluorescence 3D co-imaging endoscope system based on a single detector, which aims to solve the technical problems in the prior art and realize the simultaneous acquisition and display of visible light and near infrared fluorescence 3D images.
The technical scheme adopted by the invention for solving the technical problems is as follows.
The visible light and near infrared fluorescence 3D co-imaging endoscope system based on the single detector comprises a visible light near infrared excitation light source, a binocular endoscopic imaging system, an optical relay image transfer system, an image sensor module, an image processing and fusing module and a 3D image display system;
the visible light near-infrared excitation light source provides visible light illumination and near-infrared fluorescence excitation light illumination for the binocular endoscopic imaging system simultaneously;
the binocular endoscopic imaging system acquires binocular images with horizontal parallax;
the optical relay image transfer system performs effect reduction or amplification on the binocular image, and images the binocular image to an RGB-NIR image sensor after correction;
the image sensor module comprises an RGB-NIR image sensor and an image sensor driving-image collecting module, the RGB-NIR image sensor samples the received image of the optical relay image transfer system into a digital image, the image sensor driving-image collecting module provides a working time sequence for the RGB-NIR image sensor, collects the digital image output by the RGB-NIR image sensor and transmits the collected digital image to the image processing and fusing module;
the image processing and fusing module is used for respectively extracting a visible light color 3D image and a near infrared fluorescence 3D image from a received digital image, respectively carrying out color correction and pseudo color processing on the visible light color 3D image and the near infrared fluorescence 3D image, fusing the processed visible light color 3D image and the near infrared fluorescence 3D image, carrying out 3D coding on the obtained 3D fused image and transmitting the 3D coded image to a 3D image display system;
the 3D image display system displays the received 3D code as a 3D image.
Further, the visible light near-infrared excitation light source is a light source integrating a visible light cold light source and a near-infrared fluorescence excitation light source, the working wavelength band of the visible light cold light source is 400-700nm, and the near-infrared fluorescence excitation light source is 785nm laser.
Further, the working wave band of the binocular endoscopic imaging system is 400-1000 nm.
Furthermore, the binocular endoscopic imaging system comprises an outer endoscope body tube, a second optical fiber, an inner endoscope body tube, a first fixing piece and a single-tube endoscope; the first fixing piece is a cylinder, two axial through holes are formed in the first fixing piece, and the two axial through holes are centrosymmetric relative to the circle center of the radial cross section on the radial cross section of the first fixing piece; the endoscope inner tube and the endoscope outer tube are sequentially sleeved outside the first fixing piece from inside to outside and are coaxially arranged, and the inner wall of the endoscope inner tube is fixed on the outer wall of the first fixing piece; the second optical fibers are fixed between the outer wall of the endoscope body inner tube and the inner wall of the endoscope body outer tube, and the length direction of the second optical fibers is axially arranged along the first fixing piece; the two single-tube endoscopes are respectively fixed in the two axial through holes of the first fixing piece, and the working wave band of the single-tube endoscope is 400-1000 nm; the rear ends of the endoscope outer tube, the second optical fiber, the endoscope inner tube, the first fixing piece and the single-tube endoscope are horizontally aligned, and the front ends are positioned on the same surface.
Furthermore, the 3D common imaging endoscope system further comprises a first optical fiber, a first connecting through hole is formed in the outer wall of the rear portion of the outer tube of the endoscope body, one end of the first optical fiber is connected with the visible light near-infrared excitation light source, and the other end of the first optical fiber penetrates through the first connecting through hole and is connected with a second optical fiber.
Still further, the 3D common imaging endoscope system further comprises a second fixing piece which is of a cylindrical structure and is sleeved and fixed outside the outer wall of the rear portion of the binocular endoscopic imaging system, a second connecting hole matched with the first connecting hole is formed in the second fixing piece, and the other end of the first optical fiber penetrates through the second connecting hole and the first connecting hole in sequence and is connected with the second optical fiber.
Furthermore, the rear end of the optical relay image transfer system is fixedly connected with the image sensor module through a connecting piece.
Further, the working waveband of the optical relay image transfer system is 400-1000 nm.
Further, the optical relay image transfer system comprises a shell, a first lens, a second lens, a third lens, a fourth lens, a fifth lens and a sixth lens, wherein the shell is a hollow cylinder, the first lens, the second lens, the third lens, the fourth lens, the fifth lens and the sixth lens are coaxially arranged in a cavity of the shell from front to back along the propagation direction of light, the diameter of the first lens is 7mm, the center thickness of the first lens is 1.3mm, the diameter of the second lens is 6.4mm, the center thickness of the second lens is 1.5mm, the distance between the first lens and the second lens is 11mm, the diameter of the third lens is 7mm, the center thickness of the third lens is 3.1mm, the distance between the second lens and the third lens is 0.3mm, the diameter of the fourth lens is 3.5mm, the center thickness of the fourth lens is 1.8mm, the distance between the third lens and the fourth lens is 0, the diameter of the fifth lens is 7mm, the center thickness of the fourth lens is 2.1mm, and the distance between the fifth lens is 1.7mm, the diameter of the sixth lens is 7mm, the center thickness is 1.1mm, and the distance between the fifth lens and the sixth lens is 0.5 mm.
Further, the RGB-NIR image sensor allows adjacent four pixels to receive four wavelength bands of light of red, green, blue, and near infrared, respectively, through a mosaic form.
Further, the 3D image display system is a polarized 3D display, 3D helmet glasses, or a shutter type 3D display.
Compared with the prior art, the invention has the beneficial effects that:
the visible light and near infrared fluorescence 3D co-imaging endoscope system based on the single detector receives two images (binocular images) with horizontal parallax through the RGB-NIR image sensor to realize the acquisition of the 3D images, so that the simultaneous real-time acquisition of the visible light color 3D images and the near infrared fluorescence 3D images is realized. On one hand, the system structure is simpler, the size is smaller, on the other hand, the state switching is carried out without interrupting the operation, the smoothness of the operation process is ensured, and on the other hand, the doctor can intuitively feel the position and the size of the pathological change tissue through the visible light color 3D image and the near-infrared fluorescence 3D image, so that the doctor is assisted to make the best diagnosis and treatment scheme, and the operation success rate is greatly improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive efforts.
FIG. 1 is a schematic structural diagram of a visible light and near infrared fluorescence 3D co-imaging endoscope system based on a single detector;
FIG. 2 is a top view of a binocular endoscopic imaging system in the visible light and near infrared fluorescence 3D co-imaging endoscope system based on a single detector according to the present invention;
FIG. 3 is a schematic structural diagram of an optical relay image transfer system in the visible light and near infrared fluorescence 3D co-imaging endoscope system based on a single detector;
FIG. 4 is a schematic diagram of a single detector based RGB-NIR imaging endoscope system for visible and near infrared fluorescence 3D co-imaging simultaneously acquiring visible color 3D images and near infrared fluorescence 3D images, wherein R represents a pixel for receiving red wavelength band, G represents a pixel for receiving green wavelength band, B represents a pixel for receiving blue wavelength band, and NIR represents a pixel for receiving near infrared wavelength band;
in the figure, 1, a visible light near-infrared excitation light source, 2, a binocular endoscopic imaging system, 2-1, an external endoscope tube, 2-2, a second optical fiber, 2-3, an internal endoscope tube, 2-4, a first fixing piece, 2-5, a single-tube endoscope, 3, an optical relay image transfer system, 3-1, a shell, 3-2, a first lens, 3-3, a second lens, 3-4, a third lens, 3-5, a fourth lens, 3-6, a fifth lens, 3-7, a sixth lens, 4, an image sensor module, 5, an image processing fusion module, 6 and 3D image display systems, 7, a second fixing piece, 8, a connecting piece, 9 and the first optical fiber.
Detailed Description
The technical scheme of the invention is further explained by combining the attached drawings.
As shown in fig. 1, the visible light and near-infrared fluorescence 3D co-imaging endoscope system based on a single detector of the present invention includes a visible light near-infrared excitation light source 1, a binocular endoscopic imaging system 2, an optical relay image transfer system 3, an image sensor module 4, an image processing and fusing module 5, a 3D image display system 6, a first fixing member 7, a connecting member 8, and a first optical fiber 9. The binocular endoscopic imaging system 2, the optical relay image transfer system 3 and the image sensor module 4 are main parts of the endoscope system, and the visible light near-infrared excitation light source 1, the image processing and fusion module 5 and the 3D image display system 6 are peripheral equipment of the endoscope system.
In the endoscope system, the visible near-infrared excitation light source 1 is a light source integrating a visible cold light source and a near-infrared fluorescence excitation light source. The working waveband of the visible light cold light source is 400-700nm, and the near-infrared fluorescence excitation light source is 785nm laser. The visible near-infrared excitation light source 1 can be obtained in a manner well known to those skilled in the art. The visible light near-infrared excitation light source 1 provides visible light illumination and near-infrared fluorescence excitation light illumination for the binocular endoscopic imaging system 2 at the same time.
In the endoscope system, the binocular endoscopic imaging system 2 comprises an outer endoscope tube 2-1, a second optical fiber 2-2, an inner endoscope tube 2-3, a first fixing piece 2-4 and a single-tube endoscope 2-5. The first fixing piece 2-4 is a cylinder, two axial through holes are formed in the first fixing piece 2-4, the inner diameters of the two axial through holes are matched with the outer diameters of the two single-tube endoscopes 2-5 respectively, and on the radial cross section of the first fixing piece 2-4, the two axial through holes are centrosymmetric relative to the circle center of the radial cross section; the first fixing member 2-4 may be an integral structure or may be assembled by a plurality of structures. The endoscope inner tube 2-3 and the endoscope outer tube 2-4 are sequentially sleeved outside the first fixing piece 2-4 from inside to outside and are coaxially arranged, the inner wall of the endoscope inner tube 2-3 is fixedly adhered to the outer wall of the first fixing piece 2-4 through glue, and a gap is formed between the outer wall of the endoscope inner tube 2-3 and the inner wall of the endoscope outer tube 2-1. The second optical fibers 2-2 are multiple and can be arranged according to actual needs; the second optical fiber 2-2 is fixed between the outer wall of the endoscope inner tube 2-3 and the inner wall of the endoscope outer tube 2-1 through optical fiber fixing glue, and the length direction of the second optical fiber 2-2 is arranged along the axial direction of the first fixing piece 2-4. The number of the single-tube endoscopes 2-5 is two, and the two single-tube endoscopes 2-5 are respectively fixed in the two axial through holes of the first fixing piece 2-4. The rear ends of the endoscope outer tube 2-1, the second optical fiber 2-2, the endoscope inner tube 2-3, the first fixing piece 2-4 and the single-tube endoscope 2-5 are horizontally aligned, and the front ends are positioned on the same surface; the plane is not particularly limited, and may be a horizontal plane or a slant plane, and the angle between the plane and the binocular endoscopic imaging system 2 is usually 0-90 degrees, such as 0 degree, 30 degrees, or 90 degrees. The outer tube 2-1 of the endoscope body is made of biocompatible steel and can directly contact with a human body. The two single-tube endoscopes 2-5 respectively collect monocular images, so that the binocular endoscopic imaging system 2 collects two images with horizontal parallax like human eyes, and the images are called binocular images. The parameters of the two single-tube endoscopes 2-5 must be as identical as possible in order to acquire two images of different viewing angles but at the same magnification. In order to simultaneously acquire visible light and near-infrared fluorescence 3D images, the working wave bands of the two single-tube endoscopes 2-5 are both 400-1000nm, and visible light and near-infrared fluorescence excitation light can be simultaneously transmitted by plating visible and near-infrared antireflection films on the optical lens group of the single-tube endoscope 2-5. The type of the single-tube endoscope 2-5 is not particularly limited, and a rigid rod endoscope composed of a cylindrical lens can be used, and other types of endoscopes, such as a flexible optical fiber endoscope, can also be used.
In the endoscope system, the rear end of the binocular endoscopic imaging system 2 is fixedly connected with the front end of the shell 3-1 of the optical relay image transfer system 3, the specific connection structure is not particularly limited, and the binocular endoscopic imaging system can be any mechanical connection meeting the optical performance conditions (namely ensuring the functions of the optical relay image transfer system 3) in the prior art, such as threaded connection or clamping through a clamping piece.
In order to realize that the visible light near-infrared excitation light source 1 simultaneously provides visible light illumination and near-infrared fluorescence excitation light illumination for the binocular endoscopic imaging system 2, a first connecting through hole can be arranged on the outer wall of the rear part of the outer tube 2-1 of the endoscope body, one end of a first optical fiber 9 is connected with the visible light near-infrared excitation light source 1, and the other end of the first optical fiber passes through the first connecting through hole and is connected with a second optical fiber 2-2. In order to fix the optical fiber connector of the first optical fiber 9 conveniently, a second fixing member 7 may be further provided, the second fixing member 7 is a cylindrical structure, and is sleeved and fixed outside the outer wall of the rear portion of the binocular endoscopic imaging system 2, a second connecting hole matched with the first connecting hole is formed in the second fixing member 7, and the optical fiber connector of the first optical fiber 9 sequentially penetrates through the second connecting hole and the first connecting hole to be connected with the second optical fiber 2-2. The first optical fiber 9 is a flexible fiber bundle.
In the endoscope system, the connector 8 is used to fixedly connect the rear end of the housing 3-1 of the optical relay image transfer system 3 to the image sensor module 4. The specific structure of the connecting member 8 is not particularly limited, and may be any mechanical connection in the prior art, such as a threaded connection or a clamping member, which satisfies the optical performance condition (i.e., ensures the functions of the optical relay image transfer system 3 and the image sensor module 4).
In the endoscope system, the optical relay image transfer system 3 reduces or amplifies the binocular image obtained by the binocular endoscopic imaging system 2, and images the binocular image after correction onto the RGB-NIR image sensor of the image sensor module 4. The working wave band of the optical relay image transfer system 3 is 400-1000nm, visible light and near infrared light can be transmitted, 785nm filter is adopted to filter 785nm near infrared fluorescence excitation light, and interference of the excitation light on fluorescence imaging is prevented. The present embodiment provides an optical relay image transfer system 3 that doubles and corrects a binocular image and then images on an RGB-NIR image sensor. The structure of the optical relay image transfer system 3 is shown in fig. 3, and comprises a housing 3-1, a first lens 3-2, a second lens 3-3, a third lens 3-4, a fourth lens 3-5, a fifth lens 3-6 and a sixth lens 3-7, wherein the housing 1 is a hollow cylinder, the first lens 3-2, the second lens 3-3, the third lens 3-4, the fourth lens 3-5, the fifth lens 3-6 and the sixth lens 3-7 are coaxially arranged in a cavity of the housing 1 from front to back along the propagation direction of light, the fixing mode is not particularly limited, and the fixing mode can be selected according to actual conditions in the field; the total length of the optical relay image system 3 is 23mm, the diameter of the first lens 3-2 is 7mm, the center thickness is 1.3mm, the diameter of the second lens 3-3 is 6.4mm, the center thickness is 1.5mm, the distance between the first lens 3-2 and the second lens 3-3 is 11mm, the diameter of the third lens 3-4 is 7mm, the center thickness is 3.1mm, the distance between the second lens 3-3 and the third lens 3-4 is 0.3mm, the diameter of the fourth lens 3-5 is 3.5mm, the center thickness is 1.8mm, the distance between the third lens 3-4 and the fourth lens 3-5 is 0, the diameter of the fifth lens 3-6 is 7mm, the center thickness is 2.1mm, the distance between the fourth lens 3-5 and the fifth lens 3-6 is 1.7mm, the diameter of the sixth lens is 7mm, the center thickness is 1.1mm, and the distance between the fifth lens 3-6 and the sixth lens is 0.5 mm. All lenses in the optical relay imaging system 3 are made of K9 glass material,
the image sensor module 4 comprises an RGB-NIR image sensor and an image sensor driver-image acquisition module. The RGB-NIR image sensor samples the received image of the optical relay transfer system 3 into a digital image. The RGB-NIR image sensor is a multispectral detector and comprises three visible light wave bands of red, green and blue and a near infrared wave band. Namely, four adjacent pixels of the RGB-NIR detector 4 are respectively red R, green G, blue B and near-infrared NIR, the three channels of red, green and blue may synthesize a visible light color image, and the near-infrared channel captures a near-infrared fluorescence image, as shown in fig. 4. The RGB-NIR image sensor may be implemented in a manner well known to those skilled in the art. The image sensor driving-image collecting module is connected with the RGB-NIR image sensor and the image processing and fusing module 5, provides a working time sequence for the RGB-NIR image sensor, collects digital images output by the RGB-NIR image sensor and transmits the collected digital images to the image processing and fusing module 5. The image sensor driver-image capture module may also be implemented in a manner well known to those skilled in the art.
In the endoscope system, the image processing and fusing module 5 is connected with the 3D image display system 6, the image processing and fusing module 5 extracts a visible light color 3D image and a near infrared fluorescence 3D image from a received digital image, then carries out color correction and pseudo color processing on the visible light color 3D image and the near infrared fluorescence 3D image respectively, fuses the processed visible light color 3D image and the near infrared fluorescence 3D image, carries out 3D coding on the obtained 3D fused image, and transmits the 3D coded image to the 3D image display system 6. The image processing and fusion module 5 can also be realized by a method known by persons skilled in the art, the hardware part of the image processing and fusion module can adopt an industrial personal computer or an embedded mainboard and the like, and the software part is an image fusion algorithm program.
In the endoscope system, the 3D image display system 6 displays the received 3D code as a 3D image. The 3D image display system 6 refers to a display that can provide a user with a 3D feeling, and may be a polarized 3D display, a shutter type 3D display, a 3D head-mounted eye, other naked eye 3D display devices, and the like.
The visible light and near infrared fluorescence 3D fusion image endoscope system provided by the invention has the working process that: under the irradiation of a visible light near-infrared excitation light source 1, a binocular endoscopic imaging system 2 collects two images (namely binocular images) with horizontal parallax, an optical relay image transfer system 3 appropriately reduces or amplifies the binocular images and images the images on an RGB-NIR image sensor 4 after correction, the RGB-NIR image sensor samples the received binocular images into digital images, an image sensor driving-image collecting module provides a working time sequence for the RGB-NIR image sensor, collects the digital images output by the RGB-NIR image sensor and transmits the collected digital images to an image processing and fusing module 5; the image processing and fusing module 5 extracts a visible light color 3D image and a near infrared fluorescence 3D image respectively, then performs color correction and pseudo color processing on the visible light color 3D image and the near infrared fluorescence 3D image respectively, and finally fuses the processed images, so that one 3D image contains visible light color information and near infrared fluorescence information, 3D codes the 3D fused image and transmits the 3D coded image to the 3D image display system 6, and the 3D image display system 6 displays the 3D coded image.
Claims (10)
1. The visible light and near infrared fluorescence 3D co-imaging endoscope system based on the single detector is characterized by comprising a visible light near infrared excitation light source (1), a binocular endoscopic imaging system (2), an optical relay image transfer system (3), an image sensor module (4), an image processing and fusion module (5) and a 3D image display system (6);
the visible light near-infrared excitation light source (1) is used for simultaneously providing visible light illumination and near-infrared fluorescence excitation light illumination for the binocular endoscopic imaging system (2);
the binocular endoscopic imaging system (2) collects binocular images with horizontal parallax;
the optical relay image transfer system (3) is used for carrying out effect reduction or amplification on the binocular image, and imaging the binocular image to an RGB-NIR image sensor after correction;
the image sensor module (4) comprises an RGB-NIR image sensor and an image sensor driving-image collecting module, the RGB-NIR image sensor samples the received image of the optical relay image transfer system (3) into a digital image, the image sensor driving-image collecting module provides a working time sequence for the RGB-NIR image sensor, collects the digital image output by the RGB-NIR image sensor and transmits the collected digital image to the image processing and fusing module (5);
the image processing and fusing module (5) respectively extracts a visible light color 3D image and a near infrared fluorescence 3D image from the received digital image, respectively performs color correction and pseudo color processing on the visible light color 3D image and the near infrared fluorescence 3D image, fuses the processed visible light color 3D image and the near infrared fluorescence 3D image, performs 3D coding on the obtained 3D fused image, and transmits the 3D coded image to the 3D image display system (6);
the 3D image display system (6) displays the received 3D code as a 3D image.
2. The single-detector-based visible light and near-infrared fluorescence 3D co-imaging endoscope system according to claim 1, characterized in that the visible light near-infrared excitation light source (1) is a light source integrating a visible light cold light source and a near-infrared fluorescence excitation light source, the working wavelength band of the visible light cold light source is 400-700nm, and the near-infrared fluorescence excitation light source is 785nm laser.
3. The visible light and near infrared fluorescence 3D co-imaging endoscope system based on the single detector as claimed in claim 1, characterized in that the working wavelength band of the binocular endoscopic imaging system (2) is 400-1000 nm; the working wavelength band of the optical relay image transfer system (3) is 400-1000 nm.
4. The single-detector based visible light and near-infrared fluorescence 3D co-imaging endoscope system according to claim 1, characterized in that the binocular endoscopic imaging system (2) comprises an outer endoscope tube (2-1), a second optical fiber (2-2), an inner endoscope tube (2-3), a first fixing piece (2-4) and a single-tube endoscope (2-5); the first fixing piece (2-4) is a cylinder, two axial through holes are formed in the first fixing piece (2-4), and on the radial cross section of the first fixing piece (2-4), the two axial through holes are centrosymmetric relative to the circle center of the radial cross section; the endoscope inner tube (2-3) and the endoscope outer tube (2-1) are sequentially sleeved outside the first fixing piece (2-4) from inside to outside and are coaxially arranged, and the inner wall of the endoscope inner tube (2-3) is fixed on the outer wall of the first fixing piece (2-4); the second optical fibers (2-2) are fixed between the outer wall of the endoscope inner tube (2-3) and the inner wall of the endoscope outer tube (2-1), and the length direction of the second optical fibers (2-2) is axially arranged along the first fixing piece (2-4); the two single-tube endoscopes (2-5) are respectively fixed in the two axial through holes of the first fixing piece (2-4), and the working wave band of the single-tube endoscope (2-5) is 400-1000 nm; the rear ends of the endoscope outer tube (2-1), the second optical fiber (2-2), the endoscope inner tube (2-3), the first fixing piece (2-4) and the single-tube endoscope (2-5) are horizontally aligned, and the front ends are positioned on the same surface.
5. The single-detector-based visible light and near-infrared fluorescence 3D co-imaging endoscope system according to claim 4, characterized in that the 3D co-imaging endoscope system further comprises a first optical fiber (9), a first connecting through hole is provided on the outer wall of the rear portion of the outer tube (2-1) of the endoscope body, one end of the first optical fiber (9) is connected with the visible light near-infrared excitation light source (1), and the other end passes through the first connecting through hole and is connected with the second optical fiber (2-2).
6. The visible light and near infrared fluorescence 3D co-imaging endoscope system based on the single detector as claimed in claim 5, characterized in that the 3D co-imaging endoscope system further comprises a second fixing member (7), the second fixing member (7) is a cylindrical structure, and is sleeved and fixed outside the outer wall of the rear portion of the binocular endoscopic imaging system (2), a second connecting hole matched with the first connecting hole is arranged on the second fixing member (7), and the other end of the first optical fiber (9) passes through the second connecting hole and the first connecting hole in sequence and is connected with the second optical fiber (2-2).
7. The single-detector-based visible light and near-infrared fluorescence 3D co-imaging endoscope system according to claim 1, characterized in that the rear end of the optical relay image transfer system (3) is fixedly connected with the image sensor module (4) through a connecting piece (8).
8. The single-detector-based visible light and near-infrared fluorescence 3D co-imaging endoscope system according to claim 1, characterized in that the optical relay image system (3) comprises a housing (3-1), a first lens (3-2), a second lens (3-3), a third lens (3-4), a fourth lens (3-5), a fifth lens (3-6) and a sixth lens (3-7), the housing (3-1) is a hollow cylinder, the first lens (3-2), the second lens (3-3), the third lens (3-4), the fourth lens (3-5), the fifth lens (3-6) and the sixth lens (3-7) are coaxially set in the cavity of the housing (3-1) from front to back along the propagation direction of light, the diameter of the first lens (3-2) is 7mm, the center thickness is 1.3mm, the diameter of the second lens (3-3) is 6.4mm, the center thickness is 1.5mm, the distance between the first lens (3-2) and the second lens (3-3) is 11mm, the diameter of the third lens (3-4) is 7mm, the center thickness is 3.1mm, the distance between the second lens (3-3) and the third lens (3-4) is 0.3mm, the diameter of the fourth lens (3-5) is 3.5mm, the center thickness is 1.8mm, the distance between the third lens (3-4) and the fourth lens (3-5) is 0, the diameter of the fifth lens (3-6) is 7mm, the center thickness is 2.1mm, the distance between the fourth lens (3-5) and the fifth lens (3-6) is 1.7mm, and the diameter of the sixth lens (3-7 mm) is 7mm, the center thickness is 1.1mm, and the distance between the fifth lens (3-6) and the sixth lens (3-7) is 0.5 mm.
9. The single-detector based visible light and near-infrared fluorescence 3D co-imaging endoscope system according to claim 1, characterized in that the RGB-NIR image sensor makes adjacent four pixels receive four bands of red, green, blue and near-infrared light, respectively, through mosaic formation.
10. Single-detector based visible and near-infrared fluorescent 3D co-imaging endoscope systems according to claim 1, characterized by the fact that the 3D image display system (6) is a polarized 3D display, 3D helmet glasses or a shuttered 3D display.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911315523.4A CN110840386A (en) | 2019-12-19 | 2019-12-19 | Visible light and near-infrared fluorescence 3D common imaging endoscope system based on single detector |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911315523.4A CN110840386A (en) | 2019-12-19 | 2019-12-19 | Visible light and near-infrared fluorescence 3D common imaging endoscope system based on single detector |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110840386A true CN110840386A (en) | 2020-02-28 |
Family
ID=69609888
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911315523.4A Pending CN110840386A (en) | 2019-12-19 | 2019-12-19 | Visible light and near-infrared fluorescence 3D common imaging endoscope system based on single detector |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110840386A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111772560A (en) * | 2020-08-06 | 2020-10-16 | 山西医科大学 | Fluorescence endoscope imaging system and method based on dual-channel efficient transmission |
| CN113504636A (en) * | 2021-08-03 | 2021-10-15 | 嘉兴智瞳科技有限公司 | Imaging system with 3D imaging and fluorescence imaging functions |
| CN114298956A (en) * | 2022-03-09 | 2022-04-08 | 广东欧谱曼迪科技有限公司 | Dual-fluorescence endoscope image fusion method, electronic equipment and device |
| CN115316919A (en) * | 2022-09-15 | 2022-11-11 | 广东欧谱曼迪科技有限公司 | Dual-camera 3D optical fluorescence endoscope camera shooting system and method and electronic equipment |
| WO2022257946A1 (en) * | 2021-06-07 | 2022-12-15 | 上海微觅医疗器械有限公司 | Multispectral imaging system and method, and storage medium |
| CN117398043A (en) * | 2023-12-15 | 2024-01-16 | 广东欧谱曼迪科技股份有限公司 | 3D fluorescent endoscope, imaging method, device and debugging method |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101518438A (en) * | 2009-03-27 | 2009-09-02 | 南开大学 | Binocular endoscope operation visual system |
| CN103654699A (en) * | 2013-12-10 | 2014-03-26 | 西安电子科技大学 | Fluorescence excitation binocular endoscopic system |
| CN104656242A (en) * | 2013-11-22 | 2015-05-27 | 深圳先进技术研究院 | Endoscope binocular optical system with double objective lens and single image sensor |
| CN108965654A (en) * | 2018-02-11 | 2018-12-07 | 浙江宇视科技有限公司 | Double spectrum camera systems and image processing method based on single-sensor |
| CN109936690A (en) * | 2017-12-15 | 2019-06-25 | 百度（美国）有限责任公司 | System and method for capturing two or more groups light image simultaneously |
| CN110198653A (en) * | 2017-01-20 | 2019-09-03 | 威里利生命科学有限责任公司 | Simultaneously visible and fluorescence endoscope imaging |
| CN209787294U (en) * | 2019-01-11 | 2019-12-13 | 南京理工大学 | Multispectral three-dimensional imaging system |
-
2019
- 2019-12-19 CN CN201911315523.4A patent/CN110840386A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101518438A (en) * | 2009-03-27 | 2009-09-02 | 南开大学 | Binocular endoscope operation visual system |
| CN104656242A (en) * | 2013-11-22 | 2015-05-27 | 深圳先进技术研究院 | Endoscope binocular optical system with double objective lens and single image sensor |
| CN103654699A (en) * | 2013-12-10 | 2014-03-26 | 西安电子科技大学 | Fluorescence excitation binocular endoscopic system |
| CN110198653A (en) * | 2017-01-20 | 2019-09-03 | 威里利生命科学有限责任公司 | Simultaneously visible and fluorescence endoscope imaging |
| CN109936690A (en) * | 2017-12-15 | 2019-06-25 | 百度（美国）有限责任公司 | System and method for capturing two or more groups light image simultaneously |
| CN108965654A (en) * | 2018-02-11 | 2018-12-07 | 浙江宇视科技有限公司 | Double spectrum camera systems and image processing method based on single-sensor |
| CN209787294U (en) * | 2019-01-11 | 2019-12-13 | 南京理工大学 | Multispectral three-dimensional imaging system |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111772560A (en) * | 2020-08-06 | 2020-10-16 | 山西医科大学 | Fluorescence endoscope imaging system and method based on dual-channel efficient transmission |
| WO2022257946A1 (en) * | 2021-06-07 | 2022-12-15 | 上海微觅医疗器械有限公司 | Multispectral imaging system and method, and storage medium |
| CN113504636A (en) * | 2021-08-03 | 2021-10-15 | 嘉兴智瞳科技有限公司 | Imaging system with 3D imaging and fluorescence imaging functions |
| CN113504636B (en) * | 2021-08-03 | 2022-03-29 | 嘉兴智瞳科技有限公司 | Imaging system with 3D imaging and fluorescence imaging functions |
| CN114298956A (en) * | 2022-03-09 | 2022-04-08 | 广东欧谱曼迪科技有限公司 | Dual-fluorescence endoscope image fusion method, electronic equipment and device |
| CN115316919A (en) * | 2022-09-15 | 2022-11-11 | 广东欧谱曼迪科技有限公司 | Dual-camera 3D optical fluorescence endoscope camera shooting system and method and electronic equipment |
| CN115316919B (en) * | 2022-09-15 | 2023-06-30 | 广东欧谱曼迪科技有限公司 | Dual-camera 3D optical fluorescence endoscope imaging system, method and electronic equipment |
| CN117398043A (en) * | 2023-12-15 | 2024-01-16 | 广东欧谱曼迪科技股份有限公司 | 3D fluorescent endoscope, imaging method, device and debugging method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110840386A (en) | Visible light and near-infrared fluorescence 3D common imaging endoscope system based on single detector | |
| US5603687A (en) | Asymmetric stereo-optic endoscope | |
| US20030233024A1 (en) | Electronic endoscope for stereoscopic endoscope system | |
| US8437629B2 (en) | Dual optical path prism and camera in a minimally invasive surgical system | |
| CN110811498A (en) | Visible light and near-infrared fluorescence 3D fusion image endoscope system | |
| US9549662B2 (en) | Endoscope connector method and apparatus | |
| EP1213988B1 (en) | Miniature endoscope system | |
| US20110043612A1 (en) | Dual-tube stereoscope | |
| JPH11503844A (en) | Objective lens system for stereo video endoscope | |
| US20190298151A1 (en) | Micro cmos scopes for medical imaging | |
| Liu et al. | Bio-inspired multimodal 3D endoscope for image-guided and robotic surgery | |
| US20140055582A1 (en) | Endoscopic calibration method and apparatus | |
| CN116982916B (en) | Wide-field fluorescent endoscopic imaging device and method | |
| CN111772560A (en) | Fluorescence endoscope imaging system and method based on dual-channel efficient transmission | |
| CN216792582U (en) | Endoscope polarized light optical adapter | |
| JP3257641B2 (en) | Stereoscopic endoscope device | |
| CN114521859A (en) | Secondary optical amplification and image transfer relay system of three-dimensional fluorescence endoscope based on double-optical-path design | |
| CN216696844U (en) | Polarized light endoscope optical adapter | |
| US20140055562A1 (en) | Endoscopic synthetic stereo imaging method and apparatus | |
| JPH08122666A (en) | Stereovision endoscope | |
| CN110115557B (en) | Hyperspectral endoscopic imaging device and imaging method | |
| CN201135423Y (en) | Laparoscope | |
| CN114176485A (en) | Endoscope device camera polarized light optical system, camera and endoscope device | |
| CN209750986U (en) | Medical endoscope of virtual reality panorama | |
| JP5818265B2 (en) | Stereoscopic endoscope device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200228 |