Classifications

• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
  • A61B1/00064—Constructional details of the endoscope body
  • A61B1/00103—Constructional details of the endoscope body designed for single use
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
  • A61B1/00002—Operational features of endoscopes
  • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
  • A61B1/00006—Operational features of endoscopes characterised by electronic signal processing of control signals
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
  • A61B1/00002—Operational features of endoscopes
  • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
  • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
  • A61B1/00002—Operational features of endoscopes
  • A61B1/00043—Operational features of endoscopes provided with output arrangements
  • A61B1/00045—Display arrangement
  • A61B1/00052—Display arrangement positioned at proximal end of the endoscope body
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
  • A61B1/00064—Constructional details of the endoscope body
  • A61B1/00066—Proximal part of endoscope body, e.g. handles
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
  • A61B1/00064—Constructional details of the endoscope body
  • A61B1/00105—Constructional details of the endoscope body characterised by modular construction
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
  • A61B1/012—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor characterised by internal passages or accessories therefor
  • A61B1/015—Control of fluid supply or evacuation
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
  • A61B1/012—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor characterised by internal passages or accessories therefor
  • A61B1/018—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor characterised by internal passages or accessories therefor for receiving instruments
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
  • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
  • A61B1/043—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances for fluorescence imaging
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
  • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
  • A61B1/045—Control thereof
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
  • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
  • A61B1/05—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances characterised by the image sensor, e.g. camera, being in the distal end portion
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
  • A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
  • A61B1/0638—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements providing two or more wavelengths
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
  • A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
  • A61B1/0661—Endoscope light sources
  • A61B1/0676—Endoscope light sources at distal tip of an endoscope
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
  • A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
  • A61B1/0661—Endoscope light sources
  • A61B1/0684—Endoscope light sources using light emitting diodes [LED]

Definitions

• PCT/US 19/36060 filed June 7, 2019; U.S. Patent Appl. Ser. No. 16/363,209 filed March 25, 2019 and published as US Pat. Appl. Publ. No. US2019/0216325; and International Patent Appl. No. PCT/US17/53171 filed September 25, 2017.
• This patent specification generally relates mainly to endoscopes. More particularly, some embodiments relate to portable endoscope devices that include a re-usable handle portion and a disposable or single-use cannula portion.
• Disposable endoscopy is an emerging category of endoscopic instruments. In some cases, endoscopes can be made at a low enough cost for single-use applications. Disposable or single-use endoscopy lessens the risk of cross-contamination and hospital acquired diseases.
• a multi-camera, multi-spectral endoscope comprises: a single-use cannula configured for insertion in a patient; a first camera and a first light source and a second camera and a second light source, all housed at a distal end of the cannula; wherein: the first light source is configured to emit primarily light in a first wavelength range and the second light source is configured to emit light primarily in a second wavelength range that differs from the first wavelength range; the fields of view of the first camera and of the second camera and the fields of illumination of first light source and the second light source overlap at least partly such that both cameras view a same target in a patient essentially at the same time and the same target is illuminated by both light sources essentially at the same time; the first camera includes a first two- dimensional (2D) image sensor and a first color filter and the second camera includes a second 2D sensor and a second color filter that differs from the first color filter in the wavelength allowed to pass through; a processing
• 2D two- dimensional
• the endoscope can further include one or more of the following features: (a) a reusable portion releasably secured to the cannula and carrying said display, wherein said display includes a third camera having a field of view that includes the distal end of the cannula, wherein said display is configured to selectively display images from the third camera system and said composite images, whereby a user can view images of the distal end of the cannula as it is being inserted in a patient and view the composite images after insertion; (b) the first camera has lower spatial resolution but higher sensitivity than the second camera; (c) the first light source emits light for fluorescent imaging and the second light source emits white light, and the first camera and the first color filter are confitures to image primarily fluorescence from a target in a patient and the second camera and second color filter are configured to image primarily reflected white light from the target; (d) the first light source selectively emits light for fluorescent imaging or blue light different from that for fluoroscopic imaging and the second light source emit
• and endoscope comprises: a singleuse cannula configured for insertion in a patient; a first camera system at a distal end of the cannula; a reusable portion that is proximal to and is releasably coupled with the cannula; a display carried by the reusable portion; a second camera system carried by the display, said second camera system having a field of view that includes said distal end of the camera; whereby said display is configured to show images captured with said second camera system and showing the distal end of the cannula end environs thereof as the cannula is being inserted in a patient and to show images captured with said first camera system after insertion of the cannula in the patient.
• the endoscope described in the immediately preceding paragraph ca ⁇ n further include one or more of the following features: (a) said second camera system comprises two cameras spaced from each other in a direction transverse to a longitudinal axis of the cannula and providing depth of field images of the distal end of the cannula and its environs; (b) .the first camera system comprises a first camera capturing images in a first wavelength range and a second camera capturing images in a different wavelength range; and (c) further including a processing system configured to combine aspects of images captured with said first and second cameras into composite images that enhance anatomical features of medical interest.
• an endoscopic method comprises: providing a single-use cannula configured for insertion in a patient; releasably attaching the cannula to a reusable portion that carries a display; concurrently capturing images of patient’s organ with a first camera that is at a distal end of the cannula and captures images in a first range of wavelengths and with a second camera that also is at the distal end of the cannula but captures images in a different, second range of wavelengths; processing the images into composite images that overlay images from the first camera of selected portions of the target that have properties different from the remainder of the target on images of the target taken with the second camera and thereby highlighting said selected portions of the target; and displaying at least some of the received composite images.
• the method can further includeg taking images of the distal end of the cannula with a third camera system carried by said display as the cannula is being inserted in a patient and selectively showing images of the distal end of the camera and its environs at said display.
• the grammatical conjunctions “and”, “or” and “and/or” are all intended to indicate that one or more of the cases, object or subjects they connect may occur or be present. In this way, as used herein the term “or” in all cases indicates an “inclusive or” meaning rather than an “exclusive or” meaning.
• the terms “surgical” or “surgery” refer to any physical intervention on a patient’s tissues, and does not necessarily involve cutting a patient’s tissues or closure of a previously sustained wound.
• FIGs. 1A, 1 B and 1C are side, top and rear views of a portable and ergonomic endoscope with disposable cannula, according to some embodiments;
• FIGs. 2A and 2B are perspective views of a portable and ergonomic endoscope with disposable cannula, according to some embodiments;
• FIGs. 3A-3B are perspective views that illustrate the mating and unmating of reusable and disposable portions of a portable and ergonomic endoscope, according to some embodiments;
• FIGs. 4A and 4B are a perspective and a schematic view of a distal tip including multiple camera and lighting modules used with a portable and ergonomic endoscope, according to some embodiments;
• FIG. 5 is a schematic diagram of a dual camera dual light source system for multi-spectral imaging and surgical applications, according to some embodiments
• FIG. 6 is a conceptual diagram illustrating design aspects of a dual camera dual light source system for multi-spectral imaging and surgical applications, according to some embodiments
• FIG. 7 is a diagram illustrating possible color filter array configurations for a dual camera dual light source system for multi-spectral imaging and surgical applications, according to some embodiments
• FIG. 8 is a plot showing quantum efficiency versus wavelength for Nyxel and conventional pixels
• FIG. 9 is a diagram illustrating further aspects of combining multi-band image data from a dual camera dual light source system, according to some embodiments.
• FIG. 10 is a perspective view in which a combined, spatially registered image displayed to a user on an endoscopy system, according to some embodiments.
• FIG. 11 is a perspective view of a endoscopy system having one or more forward facing cameras, according to some embodiments.
• a portable ergonomic endoscope system includes an imaging system with at least two separate cameras and two separate light sources.
• the camera and light sources are configured to be used to simultaneously image a target object (e.g. tissue).
• a target object e.g. tissue
• a system processor can coordinate the cameras, the light sources and combine the resulting images to display to an operator an enhanced combined image of the object.
• the system can be configured to perform NBI (Narrow Band Imaging) imaging.
• the system can also be configured to perform Fluorescence Imaging.
• Color Filter Array refers to a filter placed on top of a pixel to allow a certain bandwidth(s) to pass.
• Regular consumer cameras such as the cell phone camera uses RGB CFA.
• special CFAs can be designed.
• Narrow-band imaging refers to a color imaging technique for endoscopic diagnostic medical tests, where light of specific blue and green wavelengths is used to enhance the detail of certain aspects of the surface of the mucosa.
• a special filter can be electronically activated by a switch in the endoscope leading to the use of ambient light preferably of wavelengths at or close to 415 nm (blue) and 540 nm (green). Because the peak light absorption of hemoglobin occurs at these wavelengths, blood vessels will appear very dark, allowing for their improved visibility and for the improved identification of other surface structures.
• Fluorescence Imaging refers to fluorescence imaging, sometimes using fluorescent dyes, to mark, highlight or enhance certain biological mechanisms and/or structures. Fluorescence itself, is a form of luminescence that results from matter emitting light of a certain wavelength after absorbing electromagnetic radiation.
• fluorescent dyes Hexvix
• blue light around 405nm
• the camera visualizes the fluorescence emitted from within the object
• NBI the camera visualizes the reflections of various bandwidths of light by the object.
• a novel dual camera and dual light source (DCDL) system is described for multi-spectralor multi-color imaging.
• Embodiments of surgical applications are disclosed with simultaneous white light, fluorescence and infrared images.
• an endoscopy system that includes two separate camera/LED systems that are integrated into the same cannula or endoscope.
• a white light camera referred to as CamW
• a fluorescence camera referred to as CamF
• LightC blue light LEDs
• CamF is used as IR Camera when either or both LightC, LightW are off.
• CamW is optimized for white light endoscopy, where strong and optimal white LEDs are used to illuminate the object, such that high image resolution can be achieved.
• CamF is optimized for sensitivity, because typically a fluorescence light source is weak. To maximize sensitivity and signal to noise of the CMOS sensor pixels for high quality imaging, the following are implemented:
• a special color filter array (CFA) on the pixel array is used (shown in FIG. 7), such that the CMOS sensor array is sensitive to red or IR spectrum (near 600nm or higher).
• CFA color filter array
• preferably relatively large pixels for example 2.2um x 2.2um
• CamF preferably has lower spatial resolution than CamW pixels (for example, 1 ,75um x 1 ,75um or 1 .Oum x 1 .Oum) but much higher sensitivity.
• FIGs. 1A, 1 B and 1C are side, top and rear views of a portable and ergonomic endoscope with disposable cannula, according to some embodiments.
• System 100 is adapted for easy and quick use with minimized patient discomfort and high placement accuracy.
• System 100 is made up of a disposable, or singleuse portion 102 and a re-usable portion 104. The two portions 102 and 104 can be mated and un-mated with each other via connectors as will be shown and discussed infra in further detail.
• Cannula 120 has an imaging and illumination modules on its distal tip 110.
• An electrical cable (not shown) is positioned within the cannula and supplies control signals and power to the camera and LED illumination modules on distal tip 110 and also transmits video image data from the camera module to the hand piece 140 and display 150 for viewing by an operator.
• hand piece 140 includes two control buttons 142 and 144 which can be configured for power on/off and image capture, respectively.
• hand piece 140 is shaped as a pistol grip as shown and includes a rechargeable battery 141 that is accessible via battery door 148.
• battery 141 is an 18650-type lithium-ion battery.
• electronics modules 143 mounted on printed circuit board (PCB) 145.
• Electronics modules 143 and PCB 145 are configured to carry out various processes such as video processing and capture, wi-fi transmission of data to external devices, lighting control, user interface processing, and diagnostics.
• Electronic modules 143 also are configured to include at least one non-volatile memory module for storing captured video and images from the camera module.
• display 150 can both tilt and swivel to provide optimal viewing angle for the operator.
• Swivel joint 152 is configured to provide swiveling of display 150 as shown by the dash dot arrow in FIG. 1C
• hinge joint 154 is configured to provide tilting of display 150 as shown by the dash dot arrows in FIG. 1 B.
• the hinge joint is configured to allow for tilting of display in the distal direction of about 90 degrees, or nearly 90 degrees. Such tilting can be useful, for example, when give the operator an unobstructed or less obstructed view.
• Handle 140 also includes a thumb lever 146 that can be moved upwards or downwards as shown by the dashed arrows.
• the cannula 120 is connected proximally to a fluid hub 172 including in this example two fluid ports 132 and 134.
• a fluid hub 172 Proximal to the fluid hub is a collar 168.
• the collar 168 is configured to rotate so as to allow for a “plug and twist lock” style mating of portions 102 and 104, as will be shown and described in further detail infra.
• at least a portion of fluid hub 172, along with cannula 120 and distal tip 110, are manually rotatable relative to handle 140 along the main longitudinal axis of cannula 120, as shown by solid arrow 124.
• rotating the rotatable portion of hub 172 causes rotation of cannula 120 and distal tip 110 as shown by solid arrow 122.
• the combination of rotating canula 120 and 110 and moving the thumb lever 146, the user can “steer” the direction of distal tip 110 as desired.
• the cannula 120 has a preferred working length of roughly 12 inches but shorter or longer lengths can be used depending on the medical application, and a preferred outer diameter of 5.5 to 6.5 inches but again a greater or a lesser diameter can her used depending on the medical application and developments in camera and lighting technoloty.
• FIGs. 2A and 2B are perspective views of a portable and ergonomic endoscope with disposable cannula, according to some embodiments.
• FIG. 2A shows a syringe 230 used to supply fluid, such as saline, through a fluid lumen (not shown) within cannula 120 via tubing 232, connector 234 and fluid port 134.
• fluid such as saline
• the cannula 120 is semi-rigid.
• the cannula 120 is stiff enough so it does not collapse with longitudinal pushing and pulling forces expected in a medical procedure for which it is intended.
• cannula 120 is flexible enough such that it can bend while it passes through curved anatomy.
• FIGs. 3A-3B are perspective views that illustrate the mating and unmating of reusable and disposable portions of a portable and ergonomic endoscope, according to some embodiments.
• the portions 102 and 104 are connectable and separable via a mechanical and electrical connector.
• the electrical connection is made via a USB-C type plug 302 on single-use portion 102 (visible in FIGs. 3A) and USB-C type receptacle 304 on multiple use portion 104 (FIG. 3B).
• the mechanical connection includes both a structural connection to fixedly attach portions 102 and 104 as well as a steering connection, through which steering input from the steering structure in the re-usable portion 104 can be relayed to the steering components in the single-use portion 102.
• the structural connection in this example, includes a male rounded portion 312 on single-use portion 102 that is shaped to fit snugly into a female socket 314 on multiple-use portion 104.
• the structural connection also includes a twist lock type mechanism wherein a male portion 322 can be inserted past a female opening 324 and then locked by twisting the male portion 322 approximately one quarter turn (90 degrees). The twisting action can be applied manually via textured or knurled ring collar 168. In this way, the connection can be configured as a “plug and twist” type connection.
• the steering connection is provided by meshing the transmission gear 334 on the multiple-use portion 104 with the passive gear 332 on the single-use portion 102.
• FIGs. 4A and 4B are a perspective and schematic view of a distal tip including multiple camera and lighting modules used with a portable and ergonomic endoscope, according to some embodiments.
• the distal tip 110 is shown attached to the distal end of canula 120.
• tip 110 includes a housing piece 410 that is molded separately from and bonded to the distal end of canula 120.
• Housing 410 housed within housing 410 are two camara modules: CamF module 420 and CamW module 430.
• Each of the CamF 420 and CamW 430 modules includes a lens and sensor.
• the sensors for each CamF 420 and CamW 430 include a color sensor, color filter array, and electronics and circuitry as will be described in further detail, infra.
• CamF module 420 On either side of CamF module 420 are two blue LEDs 422 and 424 configured to emit excitation light suitable for fluorescence endoscopy. In some examples, the blue LEDs 422 and 424 are configured to emit light at about 410 nm (violet-blue).
• On either side of CamW module 430 are two white LEDs 430 and 434 configured to emit white light suitable for visible white light endoscopy.
• port 412 is configured to provide fluid (flowing either into or out of the patient) and/or provide an opening through which a tool or other device can pass (e.g.
• FIG 4A shows a total of four LEDs (two white and two blue), in general, other numbers of LEDs may be provided according to factors such as desired lighting quality, endoscope size, and LED characteristics such as size and brightness. In some embodiments three or fewer LEDs can be provided and in some embodiments 10 or more LEDs can be provided. Furthermore, the number of white and blue LEDs does not have to be equal, but also will depend on various factors.
• the LED set can be 3, 4 or more. Other light sources can be substituted, such as optic fibers that deliver light generated elsewhere.
• the example shown includes two separate device/fluid channels 414 and 416. In this case, both have an inner diameter of 2.2 mm.
• channel 414 can be connected to fluid port 134 (shown in FIG. 1A) while channel 416 is connected to fluid port 132 (also shown in FIG. 1A).
• the CMOS sensor of CamF 420 is configured with larger pixels than CamW 430.
• the CamF pixels can be 2.2um x 2.2 urn arranged in a 400x400 matrix size, while the CamW pixels are 1 .Oum x 1 .Oum or 1 ,75umx1 ,75um and arranged in higher spatial resolution matrix size.
• the CamW 430 module can include a CMOS sensor with smaller pixels, such as 1.75 urn x 1.75 urn or 1 um x 1 um, so higher spatial resolutions can be achieved with up to 720x720 matrix size.
• CamF 420 is used for blue light endoscopy, with partial CFA.
• An example is shown in FIG. 7 where only R Filters are used so that blue light and green light are filtered out and the majority of light that reaches the sensor is red.
• an IR camera is used as CamF.
• FIG. 5 is a schematic diagram of a dual camera dual light source system for multi-spectral imaging and surgical applications, according to some embodiments.
• the distal tip 110 includes the camera and lighting modules, namely CamF, LightC, CamW and LightW.
• CamF camera 420 is configured for capturing images of a particular color or bandwidth, such as fluorescence with a narrow band centered around 610 nm. Filters for CamF 420 are designed to block incoming light at other wavelengths, for example by using a specially designed CFA array. CamF can be used for either NBI or Fl depending on the particular application.
• LightC light source (422 and 424) for CamF 420 can be the excitation light in case of fluorescence imaging or simply blue or green light in the case of NBI.
• CamW 430 is regular white light camera such as the camera of a cell phone.
• a typical RGB CFA can be used and in addition an IR filter can also be used.
• Typically an IR filter that filters out 50% of wavelength above 650nm can be used.
• LightW (432 and 434), the light source for CamW can be LED lights with various color tones close to white day light.
• the cannula 120 includes cables 450 and 452.
• ImgF refers to the image captured by CamF, and may be fluorescence or, in the case of NBI, reflections of green or blue lights.
• ImgW refers to the image captured by CamW, which maybe fluorescence or, in the case of NBI, reflections of green or blue lights.
• the endoscope has two cameras that can operate at the same time and with different combination of lighting such as LightC, LightW (or another light band) the system takes advantage of having two “eyes” looking at the same target but seeing different aspects of the target at the same time and thus extracting more information from or about the object and targets. For example, when blue light is on, CamF would see mostly fluorescent emission by CamW sees at the same time reflection (that can be very strong) ofLightC from the object and a little bit of fluorescence. As the two cameras are in sync and also spatially registered relative to each other, composite information of different kinds is delivered to the user to improve the clinical experience over the case of seeing only one of the two kinds of information about the object or target.
• Nyxel technology can be used which has been developed by OmniVision.
• Nyxel pixels can be used for CamF 420 and have significantly improved pixel sensitivity especially with sensitivity to red and near infrared bandwidth. This is particularly useful for detecting fluorescence around 610 nm.
• electronics modules 143 front end processing and main system processing is performed. According to some embodiments, the images are combined for display on display 150.
• FIG. 6 is a conceptual diagram illustrating design aspects of a dual camera dual light source system for multi-spectral imaging and surgical applications, according to some embodiments. In general, it is desirable to obtain multi-color or multi-spectral images of target objects such as human tissue.
• CamF is an optical camera that is sensitive to certain color band, for example Red and IR.
• the output of CamF is ImgF.
• LightC is a light source (band C), other than white light.
• DBI Dual Band Imaging
• LightC can be green or blue.
• fluorescence imaging it can also be a light source that excites the object to fluorescence color.
• CamW is an optical camera that is sensitive to certain color band (B), for example the white light.
• the output of CamW is ImgW.
• LightW is a light source that emits certain color band B, for example the white light.
• FIG. 7 is a diagram illustrating possible color filter array configurations for a dual camera dual light source system for multi-spectral imaging and surgical applications, according to some embodiments.
• CamF uses a Nyxel pixel (from Omnivision) and a ’’Red Only” filter array, the CamF RRRR filter. This arrangement allows for red and/or IR band to pass while filtering out the background blue and green light.
• the CamF can achieve four times the resolution for red compared to that of Nyxel CFA or Old CFA, because one out of four pixels in Nyxel or Old CFA arrangements are used to capture red color. On the other hand, every pixel in CamF arrangement in FIG. 7 is used to capture red color.
• FIG. 8 is a plot showing quantum efficiency versus wavelength for Nyxel and conventional pixels.
• quantum efficiency is shown the new sensor developed by OminiVision, the Nyxel pixel.
• Curve 810 is a Nyxel blue pixel.
• Curve 812 is a conventional blue pixel.
• Curve 820 is a Nyxel green pixel.
• Curve 822 is a conventions green pixel.
• Curve 830 is a Nyxel red pixel.
• Curve 832 is conventional red pixel. It can be seen especially curves 830 and 832 that the Nyxel red pixel has a significantly higher sensitivity to the red or IR band than the regular conventional red pixel.
• FIG. 9 is a diagram illustrating further aspects of combining multi-band image data from a dual camera dual light source system, according to some embodiments.
• CamF can capture image frames under different combinations of LightC and LightW being turned “on” or “off.”
• the resulting captured images are ImgF from CamF and ImgWB from CamW.
• ImgF and ImgWB are spatially registered or correlated. This can be done due to the short time lag (or completely in sync when both cameras capture simultaneously) between images captured by the different cameras.
• ImgWB provides a background image under illumination by LightC, which can be used to correct the background of ImgF.
• the ImgF data combined with ImgWB when only LightC is on produces “elmgB.”
• ImgF has low signal to noise ratio (due to weak fluorescence signal), therefore CMOS sensor with high signal to noise pixels is used.
• ImgW has high signal to noise (due to strong white light), therefore CMOS sensor with smaller pixels can be used to boost spatial resolution.
• CamF is used to capture ImglR with the LightC “off.”
• CamW captures the standard white light image with LightW “on.”
• ImglR provides a “heat map” of the target; it is useful when energy devices such as laser or RF are used for tissue modification.
• ImglR can alert users of hot or cold spots.
• the ImglR and ImgW data can be spatially registered or correlated, again, due to the short time lag (or no time lag) between images captured by the different cameras.
• ImglR and ImgW can also be combined or overlayed to provide a precise location of the hot and cold spots.
• the hot and cold spots can be viewed in the context of an ordinary standard white light image to provide the viewer with locational context of the hot and cold spots.
• Surgical Embodiment 3 ImgW is combined with elmgB.
• the high quality elmgB data is spatially registered with the white light image ImgW.
• the observer is provided with high res ImgW, or fluorescence elmgB or an overlay of both.
• surgeons can employ images available to better visualize their targets.
• the fluorescence Image elmgB, the white light image ImgW and IR Image ImglR seamlessly switch between different visualization modes.
• FIG. 10 is a perspective view in which a combined, spatially registered image is displayed to a user on an endoscopy system, according to some embodiments.
• the ordinary white light image (ImgW) 1020 is displayed over most of the display screen 150.
• the example shown is “Embodiment 3” shown in FIG. 9, where the elmgB image is combined and spatially registered with the standard white color image (ImgW).
• the regions 1010 and 1012 are obtained from the elmgB data and clearly show cancerous tumors.
• the operator can easily view the cancerous regions 1010 and 1012 in spatial registration with the ordinary color image of the surrounding tissue. This blending or combination provides a greatly enhanced view of the target tissue.
• the operator can easily switch between different modes (e.g. Embodiment 1 , 2 or 3) by pressing a toggle button such as button 142, button 144 (shown in FIGs. 1 B and 2B), or by a soft-button 1040 on touch-sensitive display 150.
• a toggle button such as button 142, button 144 (shown in FIGs. 1 B and 2B), or by a soft-button 1040 on touch-sensitive display 150.
• FIG. 11 is a perspective view of an endoscopy system having one or more forward facing cameras, according to some embodiments.
• the example shown has two forward (distally) facing cameras 1140 and 1142.
• the forward facing cameras allow the operator to see precisely where the distal tip is located, without having to move the screen out of the way.
• the operator’s view can be primary focused on the display screen 150.
• With forward facing cameras, 1140 and 1142 the precise location of the distal tip and its surroundings can be viewed on the display 150.
• Image enhancements such as artificially providing a depth of field may be beneficial in some procedures.
• the two cameras or other means may be used to simulate a depth of field centered on the distal tip to enhance usability.

Landscapes

• Health & Medical Sciences (AREA)
• Life Sciences & Earth Sciences (AREA)
• Surgery (AREA)
• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Optics & Photonics (AREA)
• Biomedical Technology (AREA)
• Molecular Biology (AREA)
• Pathology (AREA)
• Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
• Biophysics (AREA)
• Heart & Thoracic Surgery (AREA)
• Medical Informatics (AREA)
• Radiology & Medical Imaging (AREA)
• Animal Behavior & Ethology (AREA)
• General Health & Medical Sciences (AREA)
• Public Health (AREA)
• Veterinary Medicine (AREA)
• Signal Processing (AREA)
• Microelectronics & Electronic Packaging (AREA)
• Endoscopes (AREA)

Applications Claiming Priority (19)

Publications (2)

Family

ID=80629931

Family Applications (1)

Country Status (3)

Family Cites Families (8)

• 2021
  • 2021-09-13 EP EP21867772.2A patent/EP4210552A4/en active Pending
  • 2021-09-13 WO PCT/US2021/050095 patent/WO2022056400A1/en active Application Filing
  • 2021-09-13 CN CN202180004319.XA patent/CN114375173A/zh active Pending

Also Published As

Similar Documents

Legal Events