CN117281451A - 3D endoscope fluorescence imaging system and imaging method thereof - Google Patents
3D endoscope fluorescence imaging system and imaging method thereof Download PDFInfo
- Publication number
- CN117281451A CN117281451A CN202311511742.6A CN202311511742A CN117281451A CN 117281451 A CN117281451 A CN 117281451A CN 202311511742 A CN202311511742 A CN 202311511742A CN 117281451 A CN117281451 A CN 117281451A
- Authority
- CN
- China
- Prior art keywords
- image
- image data
- visible light
- fluorescence
- display mode
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 71
- 238000000799 fluorescence microscopy Methods 0.000 title claims abstract description 29
- 238000000034 method Methods 0.000 claims abstract description 16
- 230000004927 fusion Effects 0.000 claims abstract description 15
- 238000003780 insertion Methods 0.000 claims description 22
- 230000037431 insertion Effects 0.000 claims description 22
- 238000012632 fluorescent imaging Methods 0.000 claims description 13
- 230000003287 optical effect Effects 0.000 claims description 8
- 238000002073 fluorescence micrograph Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 claims description 2
- 238000005259 measurement Methods 0.000 claims description 2
- 206010028980 Neoplasm Diseases 0.000 abstract description 5
- 210000001519 tissue Anatomy 0.000 description 8
- 238000012545 processing Methods 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 208000005646 Pneumoperitoneum Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000012472 biological sample Substances 0.000 description 1
- 210000004204 blood vessel Anatomy 0.000 description 1
- 239000002872 contrast media Substances 0.000 description 1
- 230000001066 destructive effect Effects 0.000 description 1
- 238000002405 diagnostic procedure Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 210000001165 lymph node Anatomy 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 239000000779 smoke Substances 0.000 description 1
- 238000001356 surgical procedure Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00163—Optical arrangements
- A61B1/00194—Optical arrangements adapted for three-dimensional imaging
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00163—Optical arrangements
- A61B1/00193—Optical arrangements adapted for stereoscopic vision
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
- A61B1/043—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances for fluorescence imaging
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
- A61B1/05—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances characterised by the image sensor, e.g. camera, being in the distal end portion
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Surgery (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Biomedical Technology (AREA)
- Optics & Photonics (AREA)
- Pathology (AREA)
- Radiology & Medical Imaging (AREA)
- Biophysics (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Endoscopes (AREA)
Abstract
The invention discloses a 3D endoscope fluorescence imaging system and an imaging method thereof, wherein the method comprises the following steps: acquiring first image data of a first image sensor, second image data of a second image sensor and third image data of a third image sensor of a 3D endoscope fluorescence imaging system under the first and second light sources; generating a 3D visible light stereoscopic image from the first image data and the second image data; generating a visible light and fluorescence fusion 3D image according to the first image data, the second image data and the third image data; the invention can more clearly distinguish some tissues, such as: tumor and normal tissue.
Description
Technical Field
The invention relates to the technical field of endoscope fluorescence imaging equipment, in particular to a 3D endoscope fluorescence imaging system and an imaging method thereof.
Background
An endoscope is a medical instrument inserted into a body lumen or organ to perform observation and treatment, and is widely used in various surgical and diagnostic procedures. In recent years, with the progress of technology, a 3D endoscope image pickup system gradually replaces a traditional 2D image pickup system, and a more real and stereoscopic visual effect is provided for doctors, so that the operation is more accurate and safer. Fluorescent imaging equipment is an advanced optical instrument widely applied to the fields of biomedicine, biology, material science and the like. Over the past few decades, fluorescence imaging techniques have been rapidly developed and widely used. The advantages are non-destructive, high sensitivity, high spatial-temporal resolution, and multi-parameter information acquisition capability. Thus, fluorescence imaging devices have become an important tool in many biological and medical studies, helping researchers to gain insight into the structure, function, and interaction process of biological samples.
Currently, conventional 3D endoscopes only provide color 3D images, and although 3D endoscopic imaging systems provide a three-dimensional view to the physician, they have insufficient contrast in some cases, especially when viewing blood vessels, lymph nodes, and microscopic structures. Common color 3D imaging systems have difficulty distinguishing certain tissues, such as tumor and normal tissue, especially in the absence of sufficient contrast.
Disclosure of Invention
This section is intended to outline some aspects of embodiments of the invention and to briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section as well as in the description summary and in the title of the application, to avoid obscuring the purpose of this section, the description summary and the title of the invention, which should not be used to limit the scope of the invention.
It is therefore an object of the present invention to provide a 3D endoscopic fluoroscopic imaging system and an imaging method thereof, which can clearly distinguish some tissues, such as: tumor and normal tissue, can provide sufficient contrast.
In order to solve the technical problems, according to one aspect of the present invention, the following technical solutions are provided:
a method of imaging a 3D endoscopic fluoroscopic imaging system, comprising:
acquiring first image data of a first image sensor of the 3D endoscope fluorescent imaging system under the first light source and the second light source, second image data of a second image sensor, third image data of a third image sensor and fourth image data of the second image sensor under the third light source by the 3D endoscope fluorescent imaging system, obtaining depth image data, and generating a 3D visible light stereoscopic image according to the first image data and the second image data;
generating a visible light and fluorescence fusion 3D image according to the first image data, the second image data and the third image data;
generating a fluorescence 3D image according to the third image data and/or the first image data and/or the second image data;
and according to the fourth image data and the 3D visible light stereoscopic image, the three-dimensional scene can be restored, and the depth measurement is performed.
As a preferable mode of the imaging method of the 3D endoscope fluorescent imaging system, the 3D endoscope fluorescent imaging system has a plurality of color 3D image display modes, including a first display mode and/or a second display mode and/or a third display mode and/or a fourth display mode and/or a fifth display mode and/or a sixth display mode and/or a seventh display mode and/or an eighth display mode and/or a ninth display mode;
in the first display mode, according to the 3D visible light stereoscopic image, a system outputs a 3D visible light image;
in the second display mode, according to the 3D image fused by the visible light and the fluorescence, the system outputs the 3D image fused by the visible light and the fluorescence;
in the third display mode, according to the fluorescent 3D image, the system outputs a monochromatic fluorescent 3D image;
in the fourth display mode, according to the fourth image and the 3D visible light stereoscopic image, the system can output the 3D visible light stereoscopic image and the point cloud modeling image at the same time;
in the fifth display mode, generating first and/or second and/or third and/or fourth multi-mode on-screen display 3D image data according to the first image data, the second image data, the third image data and the fourth image data, including but not limited to one or more combinations: 3D visible light images and 3D images fused by visible light fluorescence are displayed on the same screen, 3D visible light images and monochromatic fluorescence 3D images are displayed on the same screen, 3D visible light images, visible light fluorescence are fused, monochromatic fluorescence 3D images and point cloud modeling images are displayed on the same screen, and the like;
in the sixth display mode, according to the 2D visible light stereoscopic image, the system outputs a 2D visible light image;
in the seventh display mode, according to the 2D image fused by the visible light and the fluorescence, the system outputs the 2D image fused by the visible light and the fluorescence;
in the eighth display mode, the system outputs a monochromatic fluorescence 2D image according to the fluorescence 2D image;
in the ninth display mode, generating first and/or second and/or third and/or fourth multi-mode on-screen display 2D image data according to the first, second, third and fourth image data, including but not limited to one or more combinations: 2D visible light image and visible light fluorescence fusion 2D image are displayed on the same screen, 2D visible light image and monochromatic fluorescence 2D image are displayed on the same screen, and 2D visible light image, visible light fluorescence fusion and monochromatic fluorescence 2D image are displayed on the same screen.
As a preferred mode of an imaging method of a 3D endoscope fluoroscopic imaging system of the present invention, wherein the 3D endoscope fluoroscopic imaging system includes an imaging handle including an insertion portion and a handle grip portion, the first image sensor being provided in the insertion portion, the second image sensor and the third image sensor being provided in the handle grip portion to acquire images through an optical system at a front end of the insertion portion for insertion into a site to be observed of a patient to acquire the first image data and/or the second image data and/or the third image data and/or the fourth image data of the site to be observed of the patient;
another setting method further includes: the camera shooting handle comprises an inserting part and a handle holding part, wherein the first image sensor, the second image sensor and the third image sensor are arranged in the handle holding part and acquire images through an optical system at the front end of the inserting part, and the inserting part is used for being inserted into a part to be observed of a patient so as to acquire first image data and/or second image data and/or third image data and/or fourth image data of the part to be observed of the patient.
As a preferred embodiment of the imaging method of a 3D endoscopic fluorescent imaging system according to the present invention, the generating stereoscopic image data from the first image data and/or the second image data includes: and carrying out three-dimensional reconstruction on the first image data and/or the second image data to obtain the 3D visible light image data.
As a preferred embodiment of the imaging method of a 3D endoscopic fluorescent imaging system of the present invention, the generating 3D visible light image data from the first image data and/or the second image data includes:
the parallax of the first image data and the second image data output corresponding to two different channels of the left eye and the right eye may be adjusted to adapt to the difference in interpupillary distances of different users.
As a preferred embodiment of the imaging method of a fluorescence imaging system of a 3D endoscope, according to the 3D image fused by visible light and fluorescence, the system outputs the 3D image fused by visible light and fluorescence, including:
and resolving the third image data, fusing the third image data with the first image data and/or the second image data, superposing the single-channel third image data with the first image data and/or the green channel and/or the blue channel and/or the red channel of the second image data to obtain a new first third fused image M1 and a new second third fused image M2, and performing three-dimensional reconstruction by using the M1 and the M2 to obtain the visible light and fluorescence fused 3D image.
As a preferred embodiment of the imaging method of a fluorescence imaging system of a 3D endoscope of the present invention, the system outputs a monochromatic fluorescence 3D image according to the fluorescence 3D image, including: resolving the third image data, calculating the first image data and/or the second image data, registering the single-channel third image data with the first image data and/or the second image data respectively to obtain new fluorescence image data F1 and F2 respectively, and performing three-dimensional reconstruction by using the F1 and the F2 to obtain the single-color fluorescence 3D image;
according to the fourth image and the 3D visible light stereoscopic image, the system may output the 3D visible light stereoscopic image and the point cloud modeling image at the same time, including: and extracting depth information from the fourth image, and converting the depth image, the first image data and/or the second image data into a three-dimensional point cloud image, wherein a 3D visible light stereoscopic image and a point cloud modeling image can be output at the same time.
As a preferable scheme of the imaging method of the 3D endoscope fluorescent imaging system, the three-dimensional point cloud image can be used for measuring the distance between two points on the image, measuring the length between the marking points and calculating the area of the shape surrounded by any marking point;
the first light source is a white light LED, and the second light source is a laser or a narrow-band LED or a single-wavelength or multi-wavelength LED;
the third light source is a VCSEL or other structured light source, and the wavelength is 850nm or 940nm.
A 3D endoscopic fluoroscopic imaging system, comprising:
the imaging device comprises an imaging handle, an imaging host and a cold light source, wherein the imaging handle comprises an insertion part and a handle holding part, and the insertion part is used for being inserted into a part to be observed of a patient;
the camera shooting handle comprises a first image sensor, a second image sensor and a third image sensor which are respectively used for collecting first image data, second image data, third image data and fourth image data;
the camera handle inserting part further comprises a third light source and a light guide beam light outlet, the third light source is used for being matched with the third image sensor to collect fourth image data, and the light guide beam outlet is used for conducting the first light source and the second light source to be matched with the first image sensor, the second image sensor and the third image sensor to collect first image data, second image data and third image data;
the camera host is used for acquiring the first image data, the second image data, the third image data and the fourth image data from the camera handle so as to execute the method of any one of claims 1-13;
the cold light source comprises a first light source and a second light source, and the first light source and the second light source are conducted to a light guide beam outlet of the camera handle through light guide beams.
As a preferable scheme of the 3D endoscope fluorescent shooting system, the cold light source can receive an external trigger signal sent by the shooting host, the shooting host can respectively control the first second light source and the third light source to sequentially emit light according to a specified time sequence, and the first image data, the second image data, the third image data and the fourth image data are acquired in a short period of time, so that the effect of real-time display is achieved.
Compared with the prior art, the invention has the following beneficial effects: according to the invention, the 3D image fused by visible light and fluorescence can be obtained, the data is converted into the three-dimensional point cloud image, the 3D visible light stereoscopic image and the point cloud modeling image can be simultaneously output, and enough contrast can be provided, so that tissues can be more clearly distinguished, such as: tumor and normal tissue.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following detailed description will be given with reference to the accompanying drawings and detailed embodiments, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained from these drawings without inventive faculty for a person skilled in the art. Wherein:
FIG. 1 shows a schematic block diagram of a 3D endoscopic fluoroscopic imaging system according to an embodiment of the present invention;
FIG. 2 shows a schematic block diagram of a 3D endoscopic fluoroscopic imaging system according to another embodiment of the present invention;
fig. 3 shows a schematic flow chart of an imaging method for a 3D endoscopic fluoroscopic imaging system according to an embodiment of the present invention.
Detailed Description
In order that the above objects, features and advantages of the invention will be readily understood, a more particular description of the invention will be rendered by reference to the appended drawings.
Next, the present invention will be described in detail with reference to the drawings, wherein the sectional view of the device structure is not partially enlarged to general scale for the convenience of description, and the drawings are only examples, which should not limit the scope of the present invention. In addition, the three-dimensional dimensions of length, width and depth should be included in actual fabrication.
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.
The invention provides a 3D endoscope fluorescence imaging system and an imaging method thereof, which can clearly distinguish tissues, such as: tumor and normal tissue, can provide sufficient contrast.
Next, a 3D endoscopic fluorescence imaging system according to an embodiment of the present invention will be described first with reference to fig. 1, fig. 1 showing a schematic block diagram of the 3D endoscopic fluorescence imaging system according to an embodiment of the present invention.
As shown in fig. 1, the 3D endoscope fluoroscopic imaging system includes a small number of imaging handles 10, an imaging main body 20, and a cold light source 30, and the imaging handles 10 include an insertion portion (not shown) for insertion into a portion to be observed of a patient, and a handle grip portion (not shown); the image capturing handle 10 further includes an optical lens assembly 102, a first image sensor 103, a second image sensor 105, a third image sensor 107, a third light source 101, a conductive light beam 104, a beam splitting prism 106, and a high-speed image transmission module 108, where the first image sensor 103 and the second image sensor 105 are respectively configured to collect first image data and second image data, and the third image sensor 107 is configured to obtain third image data and fourth image data in a time sharing manner, and the first image sensor is disposed at an insertion portion of the image capturing handle, and the second image sensor and the third image sensor are disposed at a holding portion of the image capturing handle. The camera host 20 is configured to acquire first image data, second image data, third or fourth image data from the camera handle to perform image processing thereon. The 3D endoscope fluoroscopic imaging system further includes a light source 30, and the light source 30 is connected to the imaging handle via a light guide beam (not shown).
Wherein the light source 101 is used to acquire depth image data in cooperation with the third image sensor 107. The light source 10 may include a visible light source and a multispectral special light source. The light source 1 is illustratively a white LED light source. The light source 2 is a multispectral special light source, which can be a multi-wavelength infrared laser light source for self-fluorescence imaging or fluorescent contrast agent (ICG, melan, etc.) imaging.
The camera host 20 comprises an image receiving module 203, an image processing module 202 and an image output module 201, the image receiving module 203 is configured to receive image data transmitted by the high-speed image transmission module 108, send the image into the image processing module 202 for subsequent image processing, send the processed image into the image output module 201, and the image output module 201 outputs the processed 3D fluorescent image or 3D visible light fluorescent fusion image or point cloud modeling image to a 3D display, and communicates with the camera host through a button on the camera handle 10, so as to control the image output module 201 to switch between display modes, and can switch between a first display mode and/or a second display mode and/or a third display mode and/or a fourth display mode and/or a fifth display mode and/or a sixth display mode and/or a seventh display mode and/or an eighth display mode and/or a ninth display mode.
It should be noted that fig. 1 is only an example of the 3D endoscope fluorescence imaging system, and does not limit the 3D endoscope fluorescence imaging system, and the 3D endoscope fluorescence imaging system may include more or less components than those shown in fig. 1, or may be combined with some components, or different components, for example, the 3D endoscope fluorescence imaging system may further include a 3D display, a pneumoperitoneum machine, a surgical field camera, a 5G relay vehicle, and the like.
It should be noted that fig. 1 is only an example of the 3D endoscope fluorescence imaging system 100, and does not limit the 3D endoscope imaging system 100, and the 3D endoscope fluorescence imaging system 100 may include more or less components than those shown in fig. 1, or may combine some components, or different components, for example, the 3D endoscope fluorescence imaging system 100 may further include a dilator, a smoke control apparatus, an input/output device, a network access device, and the like.
In another embodiment, as shown in fig. 2, the 3D endoscope fluorescence imaging system includes a camera handle 10, a camera host 20, and a cold light source 30, wherein the camera handle 10 includes an insertion portion (not shown) and a handle grip portion (not shown), and the insertion portion is used for being inserted into a portion to be observed of a patient; the image capturing handle 10 further includes a first image sensor 103, a second image sensor 102, a third image sensor 105, a third light source 101, and a high-speed image transmission module 106, where the first image sensor 103 and the second image sensor 102 are respectively configured to collect first image data and second image data, and the third image sensor 105 is configured to obtain third image data and fourth image data in a time-sharing manner, and the first image sensor is illustratively disposed in an insertion portion of the image capturing handle, and the second image sensor and the third image sensor are disposed in a holding portion of the image capturing handle. The camera host 20 is configured to acquire first image data, second image data, third or fourth image data from the camera handle to perform image processing thereon. The 3D endoscope fluoroscopic imaging system further includes a light source 30, and the light source 30 is connected to the imaging handle via a light guide beam (not shown).
Next, an imaging method for a 3D endoscopic fluoroscopic imaging system according to an embodiment of the present invention is described with reference to fig. 3. The method can be implemented by the 3D endoscope fluorescence imaging system described with reference to FIG. 1 or FIG. 2, and can be specifically implemented by an imaging host of the 3D endoscope fluorescence imaging system. FIG. 3 is a schematic flow chart of an imaging method for a 3D endoscopic fluoroscopic imaging system according to an embodiment of the present invention, specifically including the steps of:
in step 40, acquiring first image data of a first image sensor, second image data of a second image sensor and third image data of a third image sensor of the 3D endoscope fluorescence imaging system under the first and second light sources, and acquiring fourth image data of the second image sensor under the third light source by the 3D endoscope fluorescence imaging system;
generating 3D color image data from the first image data, the second image data and outputting an image to a 3D display, displaying a 3D color image from the 3D color image data, step 41;
in step 42, generating 3D color fluorescence fusion image data according to the 3D color image data and the fourth image data, and outputting the image to a 3D display to display a 3D color fluorescence fusion image according to the 3D color fluorescence fusion image data;
generating 3D monochromatic fluorescence image data from the third image data and outputting an image to a 3D display according to the 3D monochromatic fluorescence image in step 43;
generating 3D point cloud modeling data according to the 3D color image data and the fourth image data, outputting the images to a 3D display, and displaying the 3D point cloud images according to the 3D point cloud modeling data in step 44;
in step 45, the screen may be marked according to the point cloud modeling data, and the distance between the marked points, the length, or the area within the marked points may be measured.
11. The 3D endoscope fluorescent camera system is provided with a plurality of color 3D image display modes, including a first display mode and/or a second display mode and/or a third display mode and/or a fourth display mode and/or a fifth display mode and/or a sixth display mode and/or a seventh display mode and/or an eighth display mode and/or a ninth display mode;
in the first display mode, according to the 3D visible light stereoscopic image, a system outputs a 3D visible light image;
in the second display mode, according to the 3D image fused by the visible light and the fluorescence, the system outputs the 3D image fused by the visible light and the fluorescence;
in the third display mode, according to the fluorescent 3D image, the system outputs a monochromatic fluorescent 3D image;
in the fourth display mode, according to the fourth image and the 3D visible light stereoscopic image, the system can output the 3D visible light stereoscopic image and the point cloud modeling image at the same time;
in the fifth display mode, generating first and/or second and/or third and/or fourth multi-mode on-screen display 3D image data according to the first image data, the second image data, the third image data and the fourth image data, including but not limited to one or more combinations: 3D visible light images and 3D images fused by visible light fluorescence are displayed on the same screen, 3D visible light images and monochromatic fluorescence 3D images are displayed on the same screen, 3D visible light images, visible light fluorescence are fused, monochromatic fluorescence 3D images and point cloud modeling images are displayed on the same screen, and the like;
in the sixth display mode, according to the 2D visible light stereoscopic image, the system outputs a 2D visible light image;
in the seventh display mode, according to the 2D image fused by the visible light and the fluorescence, the system outputs the 2D image fused by the visible light and the fluorescence;
in the eighth display mode, the system outputs a monochromatic fluorescence 2D image according to the fluorescence 2D image;
in the ninth display mode, generating first and/or second and/or third and/or fourth multi-mode on-screen display 2D image data according to the first, second, third and fourth image data, including but not limited to one or more combinations: 2D visible light image and visible light fluorescence fusion 2D image are displayed on the same screen, 2D visible light image and monochromatic fluorescence 2D image are displayed on the same screen, and 2D visible light image, visible light fluorescence fusion and monochromatic fluorescence 2D image are displayed on the same screen.
12. The 3D endoscope fluorescence imaging system comprises an imaging handle, wherein the imaging handle comprises an insertion part and a handle holding part, the first image sensor is arranged in the insertion part, the second image sensor and the third image sensor are arranged in the handle holding part, images are acquired through an optical system at the front end of the insertion part, and the insertion part is used for being inserted into a part to be observed of a patient so as to acquire first image data and/or second image data and/or third image data and/or fourth image data of the part to be observed of the patient;
another setting method further includes: the camera shooting handle comprises an inserting part and a handle holding part, wherein the first image sensor, the second image sensor and the third image sensor are arranged in the handle holding part and acquire images through an optical system at the front end of the inserting part, and the inserting part is used for being inserted into a part to be observed of a patient so as to acquire first image data and/or second image data and/or third image data and/or fourth image data of the part to be observed of the patient.
The generating stereoscopic image data from the first image data and/or the second image data includes: and carrying out three-dimensional reconstruction on the first image data and/or the second image data to obtain the 3D visible light image data.
The generating 3D visible light image data from the first image data and/or the second image data comprises:
the parallax of the first image data and the second image data output corresponding to two different channels of the left eye and the right eye may be adjusted to adapt to the difference in interpupillary distances of different users.
According to the 3D image fused by the visible light and the fluorescence, the system outputs the 3D image fused by the visible light and the fluorescence, and the method comprises the following steps:
and resolving the third image data, fusing the third image data with the first image data and/or the second image data, superposing the single-channel third image data with the first image data and/or the green channel and/or the blue channel and/or the red channel of the second image data to obtain a new first third fused image M1 and a new second third fused image M2, and performing three-dimensional reconstruction by using the M1 and the M2 to obtain the visible light and fluorescence fused 3D image.
Based on the fluorescence 3D image, the system outputs a monochromatic fluorescence 3D image comprising: resolving the third image data, calculating the first image data and/or the second image data, registering the single-channel third image data with the first image data and/or the second image data respectively to obtain new fluorescence image data F1 and F2 respectively, and performing three-dimensional reconstruction by using the F1 and the F2 to obtain the single-color fluorescence 3D image;
according to the fourth image and the 3D visible light stereoscopic image, the system may output the 3D visible light stereoscopic image and the point cloud modeling image at the same time, including: and extracting depth information from the fourth image, and converting the depth image, the first image data and/or the second image data into a three-dimensional point cloud image, wherein a 3D visible light stereoscopic image and a point cloud modeling image can be output at the same time.
The three-dimensional point cloud image can be used for measuring the distance between two points on the image, measuring the length between the marking points and calculating the area of the shape surrounded by any marking point, wherein the first light source is a white light LED, the second light source is a laser or a narrow-band LED or a single wavelength or multiple wavelengths, the third light source is a VCSEL or other structured light source, and the wavelength is 850nm or 940nm.
Although the invention has been described hereinabove with reference to embodiments, various modifications thereof may be made and equivalents may be substituted for elements thereof without departing from the scope of the invention. In particular, the features of the disclosed embodiments may be combined with each other in any manner as long as there is no structural conflict, and the exhaustive description of these combinations is not given in this specification merely for the sake of omitting the descriptions and saving resources. Therefore, it is intended that the invention not be limited to the particular embodiment disclosed, but that the invention will include all embodiments falling within the scope of the appended claims.
Claims (10)
1. A method for imaging a 3D endoscopic fluoroscopic imaging system, comprising:
acquiring first image data of a first image sensor of the 3D endoscope fluorescent imaging system under the first light source and the second light source, second image data of a second image sensor, third image data of a third image sensor and fourth image data of the second image sensor under the third light source by the 3D endoscope fluorescent imaging system, obtaining depth image data, and generating a 3D visible light stereoscopic image according to the first image data and the second image data;
generating a visible light and fluorescence fusion 3D image according to the first image data, the second image data and the third image data;
generating a fluorescence 3D image according to the third image data and/or the first image data and/or the second image data;
and according to the fourth image data and the 3D visible light stereoscopic image, the three-dimensional scene can be restored, and the depth measurement is performed.
2. The imaging method of a 3D endoscopic fluorescent imaging system according to claim 1, wherein the 3D endoscopic fluorescent imaging system has a plurality of color 3D image display modes including a first display mode and/or a second display mode and/or a third display mode and/or a fourth display mode and/or a fifth display mode and/or a sixth display mode and/or a seventh display mode and/or an eighth display mode and/or a ninth display mode;
in the first display mode, according to the 3D visible light stereoscopic image, a system outputs a 3D visible light image;
in the second display mode, according to the 3D image fused by the visible light and the fluorescence, the system outputs the 3D image fused by the visible light and the fluorescence;
in the third display mode, according to the fluorescent 3D image, the system outputs a monochromatic fluorescent 3D image;
in the fourth display mode, according to the fourth image and the 3D visible light stereoscopic image, the system can output the 3D visible light stereoscopic image and the point cloud modeling image at the same time;
in the fifth display mode, generating first and/or second and/or third and/or fourth multi-mode on-screen display 3D image data according to the first image data, the second image data, the third image data and the fourth image data, including but not limited to one or more combinations: 3D visible light images and 3D images fused by visible light fluorescence are displayed on the same screen, 3D visible light images and monochromatic fluorescence 3D images are displayed on the same screen, 3D visible light images, visible light fluorescence are fused, monochromatic fluorescence 3D images and point cloud modeling images are displayed on the same screen, and the like;
in the sixth display mode, according to the 2D visible light stereoscopic image, the system outputs a 2D visible light image;
in the seventh display mode, according to the 2D image fused by the visible light and the fluorescence, the system outputs the 2D image fused by the visible light and the fluorescence;
in the eighth display mode, the system outputs a monochromatic fluorescence 2D image according to the fluorescence 2D image;
in the ninth display mode, generating first and/or second and/or third and/or fourth multi-mode on-screen display 2D image data according to the first, second, third and fourth image data, including but not limited to one or more combinations: 2D visible light image and visible light fluorescence fusion 2D image are displayed on the same screen, 2D visible light image and monochromatic fluorescence 2D image are displayed on the same screen, and 2D visible light image, visible light fluorescence fusion and monochromatic fluorescence 2D image are displayed on the same screen.
3. The imaging method of a 3D endoscope fluoroscopic imaging system according to claim 2, wherein the 3D endoscope fluoroscopic imaging system comprises an imaging handle including an insertion portion and a handle grip portion, the first image sensor is disposed in the insertion portion, the second image sensor and the third image sensor are disposed in the handle grip portion, an image is acquired by an optical system at a front end of the insertion portion, the insertion portion is used for being inserted into a site to be observed of a patient to acquire the first image data and/or the second image data and/or the third image data and/or the fourth image data of the site to be observed of the patient;
another setting method further includes: the camera shooting handle comprises an inserting part and a handle holding part, wherein the first image sensor, the second image sensor and the third image sensor are arranged in the handle holding part and acquire images through an optical system at the front end of the inserting part, and the inserting part is used for being inserted into a part to be observed of a patient so as to acquire first image data and/or second image data and/or third image data and/or fourth image data of the part to be observed of the patient.
4. A method of imaging a 3D endoscopic fluoroscopic imaging system according to claim 3, wherein said generating stereoscopic image data from the first image data and/or the second image data comprises: and carrying out three-dimensional reconstruction on the first image data and/or the second image data to obtain the 3D visible light image data.
5. The method of imaging a 3D endoscopic fluoroscopic imaging system of claim 4, wherein generating 3D visible light image data from the first image data and/or the second image data comprises:
the parallax of the first image data and the second image data output corresponding to two different channels of the left eye and the right eye may be adjusted to adapt to the difference in interpupillary distances of different users.
6. The method of imaging a 3D endoscope fluorescence imaging system according to claim 5, wherein the system outputs a 3D image of visible light and fluorescence fusion according to the 3D image of visible light and fluorescence fusion, comprising:
and resolving the third image data, fusing the third image data with the first image data and/or the second image data, superposing the single-channel third image data with the first image data and/or the green channel and/or the blue channel and/or the red channel of the second image data to obtain a new first third fused image M1 and a new second third fused image M2, and performing three-dimensional reconstruction by using the M1 and the M2 to obtain the visible light and fluorescence fused 3D image.
7. The method of imaging a 3D endoscopic fluoroscopic imaging system of claim 6, wherein the system outputs a monochromatic fluoroscopic 3D image from the fluoroscopic 3D image, comprising: resolving the third image data, calculating the first image data and/or the second image data, registering the single-channel third image data with the first image data and/or the second image data respectively to obtain new fluorescence image data F1 and F2 respectively, and performing three-dimensional reconstruction by using the F1 and the F2 to obtain the single-color fluorescence 3D image;
according to the fourth image and the 3D visible light stereoscopic image, the system may output the 3D visible light stereoscopic image and the point cloud modeling image at the same time, including: and extracting depth information from the fourth image, and converting the depth image, the first image data and/or the second image data into a three-dimensional point cloud image, wherein a 3D visible light stereoscopic image and a point cloud modeling image can be output at the same time.
8. The imaging method of the 3D endoscope fluorescence imaging system according to claim 7, wherein the three-dimensional point cloud image can be used for measuring a distance between two points on the image, measuring a length between marking points, and calculating an area of a shape surrounded by any marking point;
the first light source is a white light LED, and the second light source is a laser or a narrow-band LED or a single-wavelength or multi-wavelength LED;
the third light source is a VCSEL or other structured light source, and the wavelength is 850nm or 940nm.
9. A 3D endoscopic fluoroscopic imaging system, implementing a 3D endoscopic fluoroscopic imaging system imaging method according to any one of claims 1 to 8, comprising:
the imaging device comprises an imaging handle, an imaging host and a cold light source, wherein the imaging handle comprises an insertion part and a handle holding part, and the insertion part is used for being inserted into a part to be observed of a patient;
the camera shooting handle comprises a first image sensor, a second image sensor and a third image sensor which are respectively used for collecting first image data, second image data, third image data and fourth image data;
the camera handle inserting part further comprises a third light source and a light guide beam light outlet, the third light source is used for being matched with the third image sensor to collect fourth image data, and the light guide beam outlet is used for conducting the first light source and the second light source to be matched with the first image sensor, the second image sensor and the third image sensor to collect first image data, second image data and third image data;
the camera host is used for acquiring the first image data, the second image data, the third image data and the fourth image data from the camera handle so as to execute the method of any one of claims 1-13;
the cold light source comprises a first light source and a second light source, and the first light source and the second light source are conducted to a light guide beam outlet of the camera handle through light guide beams.
10. The 3D endoscope fluorescent imaging system according to claim 9, wherein the cold light source can receive an external trigger signal sent by the imaging host, the imaging host can respectively control the first second light source and the third light source to sequentially emit light according to a specified time sequence, and the first image data, the second image data, the third image data and the fourth image data are obtained in a shorter period of time, so that the effect of real-time display is achieved.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311511742.6A CN117281451A (en) | 2023-11-14 | 2023-11-14 | 3D endoscope fluorescence imaging system and imaging method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311511742.6A CN117281451A (en) | 2023-11-14 | 2023-11-14 | 3D endoscope fluorescence imaging system and imaging method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117281451A true CN117281451A (en) | 2023-12-26 |
Family
ID=89251926
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311511742.6A Pending CN117281451A (en) | 2023-11-14 | 2023-11-14 | 3D endoscope fluorescence imaging system and imaging method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117281451A (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102006017003A1 (en) * | 2006-04-11 | 2007-10-18 | Friedrich-Alexander-Universität Erlangen-Nürnberg | Endoscope for depth data acquisition in e.g. medical area, has modulation unit controlling light source based on modulation data so that source transmits modulated light signal and evaluation unit evaluating signal to estimate depth data |
| JP6461411B1 (en) * | 2018-07-13 | 2019-01-30 | パナソニック株式会社 | Endoscope system |
| CN111685711A (en) * | 2020-05-25 | 2020-09-22 | 中国科学院苏州生物医学工程技术研究所 | Medical endoscope three-dimensional imaging system based on 3D camera |
| CN111803013A (en) * | 2020-07-21 | 2020-10-23 | 深圳市博盛医疗科技有限公司 | Endoscope imaging method and endoscope imaging system |
| CN113367638A (en) * | 2021-05-14 | 2021-09-10 | 广东欧谱曼迪科技有限公司 | Method and device for acquiring high-precision three-dimensional fluorescence image, storage medium and terminal |
| CN115919239A (en) * | 2022-12-29 | 2023-04-07 | 武汉迈瑞医疗技术研究院有限公司 | Imaging method for 3D endoscopic imaging system and 3D endoscopic imaging system |
| CN116158718A (en) * | 2023-03-13 | 2023-05-26 | 武汉迈瑞医疗技术研究院有限公司 | Imaging and display method for endoscope system and endoscope system |
| CN116172493A (en) * | 2023-03-13 | 2023-05-30 | 武汉迈瑞医疗技术研究院有限公司 | Imaging and display method for endoscope system and endoscope system |
| CN116327094A (en) * | 2023-03-27 | 2023-06-27 | 武汉迈瑞医疗技术研究院有限公司 | Imaging and display method for endoscope system and endoscope system |
-
2023
- 2023-11-14 CN CN202311511742.6A patent/CN117281451A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102006017003A1 (en) * | 2006-04-11 | 2007-10-18 | Friedrich-Alexander-Universität Erlangen-Nürnberg | Endoscope for depth data acquisition in e.g. medical area, has modulation unit controlling light source based on modulation data so that source transmits modulated light signal and evaluation unit evaluating signal to estimate depth data |
| JP6461411B1 (en) * | 2018-07-13 | 2019-01-30 | パナソニック株式会社 | Endoscope system |
| CN111685711A (en) * | 2020-05-25 | 2020-09-22 | 中国科学院苏州生物医学工程技术研究所 | Medical endoscope three-dimensional imaging system based on 3D camera |
| CN111803013A (en) * | 2020-07-21 | 2020-10-23 | 深圳市博盛医疗科技有限公司 | Endoscope imaging method and endoscope imaging system |
| CN113367638A (en) * | 2021-05-14 | 2021-09-10 | 广东欧谱曼迪科技有限公司 | Method and device for acquiring high-precision three-dimensional fluorescence image, storage medium and terminal |
| CN115919239A (en) * | 2022-12-29 | 2023-04-07 | 武汉迈瑞医疗技术研究院有限公司 | Imaging method for 3D endoscopic imaging system and 3D endoscopic imaging system |
| CN116158718A (en) * | 2023-03-13 | 2023-05-26 | 武汉迈瑞医疗技术研究院有限公司 | Imaging and display method for endoscope system and endoscope system |
| CN116172493A (en) * | 2023-03-13 | 2023-05-30 | 武汉迈瑞医疗技术研究院有限公司 | Imaging and display method for endoscope system and endoscope system |
| CN116327094A (en) * | 2023-03-27 | 2023-06-27 | 武汉迈瑞医疗技术研究院有限公司 | Imaging and display method for endoscope system and endoscope system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Boppart et al. | Optical imaging technology in minimally invasive surgery: current status and future directions | |
| US6600861B2 (en) | Fiber bundle and endoscope apparatus | |
| US10182708B2 (en) | Simultaneous acquisition of differing image types | |
| US7123756B2 (en) | Method and apparatus for standardized fluorescence image generation | |
| US8944996B2 (en) | Electronic endoscope system with oxygen saturation enhancing image region | |
| EP2817782B1 (en) | Video endoscopic system | |
| EP3590413A1 (en) | Endoscope system and method for operating same | |
| US9549662B2 (en) | Endoscope connector method and apparatus | |
| EP2638843A1 (en) | Endoscope system, processor device thereof, and exposure control method | |
| US11213189B2 (en) | Endoscopic device and method for endoscopic examination | |
| US20090242797A1 (en) | System and method for multi-mode optical imaging | |
| JPWO2020012563A1 (en) | Endoscope device, processing device and processing method | |
| EP3241481A1 (en) | Dual path endoscope | |
| US20140055582A1 (en) | Endoscopic calibration method and apparatus | |
| WO2017217498A1 (en) | Endoscope expansion device | |
| TWI342201B (en) | ||
| CN117281451A (en) | 3D endoscope fluorescence imaging system and imaging method thereof | |
| JP5809850B2 (en) | Image processing device | |
| US20140055562A1 (en) | Endoscopic synthetic stereo imaging method and apparatus | |
| CN115299914A (en) | Endoscope system, image processing method and device | |
| CN111528770B (en) | Endoscope imaging system, image processing method, and readable storage medium | |
| CN110115557B (en) | Hyperspectral endoscopic imaging device and imaging method | |
| CN217338517U (en) | Wide-spectrum fluorescence endoscope device | |
| Wen et al. | A Miniaturized Endoscopic Device Integrating Raman Spectroscopy and Laser Speckle Technology Via an Image Fusion Algorithm for Intraoperative Identification and Functional Protection of Parathyroid Glands | |
| US20130286407A1 (en) | Apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |