DE102022125852A1 - Medical imaging device and method for medical imaging - Google Patents

Abstract

The invention relates to a medical imaging device (410), comprising: an illumination unit (412) with at least one light source (414) which is configured to provide illumination light (416) for illuminating an object (418) to be imaged; and an image capture unit (420) which comprises at least one image capture sensor system (422) and at least one observation filter (424). The observation filter (424) is configured to block light in a first spectral range (426) which comprises visible light and to transmit light in a second spectral range (428) which is different from the first spectral range (426). The image capture unit (430) is configured to record fluorescence images (430) of the object (418) to be imaged through the observation filter (424), during the recording of which illumination light (432), the spectrum of which lies at least partially within the first spectral range (426), is used as excitation light and during the recording of which light (434) emitted by the object (418) to be imaged, the spectrum of which lies at least partially within the second spectral range (428), is detected by the image capture sensor system (422). In addition, the image capture unit (430) is configured to record anatomical images (436) of the object (418) to be imaged, during the recording of which illumination light (438) whose spectrum lies at least partially within the second spectral range (428) is used and during the recording of which light (440) remitted by the object (418) to be imaged, whose spectrum lies at least partially within the second spectral range (428), is detected by the image capture sensor system (422). The imaging device further comprises a representation generation unit (442) configured to generate at least one representation (444) from the fluorescence images (430) and the anatomical images (436).The invention also relates to a method for medical imaging, program code and a computer-readable medium (448).

Description

The invention relates to a medical imaging device, in particular an endoscope device, exoscope device and/or microscope device, a method for medical imaging, associated program code and an associated computer program product.

Imaging devices for carrying out fluorescence imaging are known from the state of the art and can record both fluorescence images and white light images. Suitable excitation light is used to specifically excite fluorescent dyes or, where appropriate, natively occurring fluorescent substances and to detect emitted light and use it for imaging. In order to be able to show a user anatomical structures in the image area at the same time, a white light image is often recorded in parallel or sequentially. The user can use the white light image to assess whether the anatomical structure is being imaged. Fluorescence images and white light images can also be superimposed, which means that anatomical information and fluorescence information can be perceived and analyzed by a user at the same time.

Imaging devices such as endoscopic or exoscopic devices that generate multispectral or hyperspectral images are also known from the prior art. In addition to two spatial dimensions, such as a conventional image from a camera, multispectral or hyperspectral images have a spectral dimension. The spectral dimension includes several spectral bands (wavelength bands). Multispectral and hyperspectral images differ essentially in the number and width of their spectral bands. Such systems can in principle also be suitable for taking fluorescence images.

Some imaging devices are known for generating such multispectral or hyperspectral images, especially in the context of medical applications. In DE 20 2014 010 558 U1 For example, a device for recording a hyperspectral image of an examination area of a body is described. The device has an input lens for generating an image in an image plane and a slit-shaped aperture in the image plane for masking out a slit-shaped area of the image. The light passing through the aperture is fanned out by a dispersive element and recorded by a camera sensor. This allows the camera sensor to record a large number of spectra, each with an associated spatial coordinate, along the longitudinal direction of the slit-shaped aperture. The device described is also designed to record further spectra along the longitudinal direction of the slit-shaped aperture in a direction different from the longitudinal direction of the slit-shaped aperture. The method for generating multispectral or hyperspectral images on which this disclosure is based is also known as the so-called pushbroom method.

In addition to the pushbroom method, there are other methods for generating multispectral or hyperspectral images. In the so-called whiskbroom method, the area under investigation or object is scanned point by point and a spectrum is obtained for each point. In contrast, in the staring method, several images are taken with the same spatial coordinates. Different spectral filters and/or illumination sources are used from image to image to resolve spectral information. There are also methods according to which a two-dimensional multi-color image is broken down into several individual spectral images using suitable optical elements such as optical slicers, lenses and prisms, which are recorded simultaneously on different detectors or detector areas. This is sometimes referred to as the snapshot approach.

As in EN 10 2020 105 458 A1 As described, multispectral and hyperspectral imaging devices are particularly suitable as endoscopic imaging devices. In this context, multispectral and/or hyperspectral imaging is a fundamental field of application, for example for diagnostics and for assessing the success or quality of an intervention.

Multimodal imaging devices allow the acquisition of either white light images and/or multispectral images and/or fluorescence images and/or hyperspectral images. Examples of such imaging devices are multimodal endoscopes and multimodal exoscopes.

Regardless of the exact design, tissue is illuminated in a specific wavelength range to capture fluorescence images in order to specifically excite fluorescent dye molecules introduced into certain entities, such as tissue areas. The light emitted with a red-shifted wavelength can be observed through a suitably selected filter, by means of which excitation light can be blocked out. If the wavelength ranges of the excitation light and the emitted light are in the longer wavelength range relative to white light used for parallel or sequential white light imaging, in particular in the In the near-infrared range, i.e. outside of visible white light, white light imaging is possible through the filter despite these observation filters. Recently, however, dyes that can be excited with visible light have been increasingly used. The known method of image generation cannot be implemented here because the filters required for fluorescence imaging impair the white light image by blocking visible light in certain wavelength ranges or usually only or at least primarily transmitting light below a certain wavelength. If fluorescent dyes are used that are to be excited in the visible range, the parallel or sequential white light recording mentioned cannot be implemented. If the white light image is omitted, however, it is difficult for the user to correctly assess anatomical structures.

Based on the state of the art, the invention is based on the object of enabling fluorescence imaging in a wide range of applications.

This object is achieved according to the invention by an imaging device, a method for medical imaging, program code and a computer program product as described herein and defined in the claims.

According to the invention, a medical imaging device, in particular an endoscope device, exoscope device and/or microscope device, comprises an illumination unit with at least one light source, which is designed to provide illumination light for illuminating an object to be imaged. The medical imaging device further comprises an image acquisition unit, which comprises at least one image acquisition sensor and at least one observation filter. The observation filter is designed to block light in a first spectral range, which comprises visible light, and to transmit light in a second spectral range different from the first spectral range. In addition, the image acquisition unit is designed to record fluorescence images of the object to be imaged through the observation filter, during the recording of which illumination light, the spectrum of which lies at least partially within the first spectral range, is used as excitation light and during the recording of which light emitted by the object to be imaged, the spectrum of which lies at least partially within the second spectral range, is detected by the image acquisition sensor. In addition, the image capture unit is configured to record anatomical images of the object to be imaged, during the recording of which illumination light is used, the spectrum of which lies at least partially within the second spectral range, and during the recording of which light remitted by the object to be imaged, the spectrum of which lies at least partially within the second spectral range, is detected by the image capture sensor system. Furthermore, the medical imaging device comprises a representation generation unit, which is configured to generate at least one representation from the fluorescence images and the anatomical images.

A method according to the invention can be carried out in particular with the medical imaging device according to the invention. The method comprises providing illumination light for illuminating an object to be imaged. The method further comprises recording fluorescence images of the object to be imaged through an observation filter which is designed to block light in a first spectral range which includes visible light and to transmit light in a second spectral range different from the first spectral range, wherein when recording the fluorescence images, illumination light whose spectrum lies at least partially within the first spectral range is used as excitation light and light emitted by the object to be imaged whose spectrum lies at least partially within the second spectral range is detected. The method also includes recording anatomical images of the object to be imaged, wherein, when recording the anatomical images, illumination light whose spectrum lies at least partially within the second spectral range is used and light remitted by the object to be imaged whose spectrum lies at least partially within the second spectral range is detected.

The features of the invention allow imaging to be carried out in a wide range of applications. In particular, fluorescent dyes that absorb in the visible range can be used and at the same time a user can be provided with a meaningful anatomical image. This allows different fluorescent dyes to be used equally without having to accept restrictions when creating an anatomical image.

The features according to the invention make it possible to record a fluorescence image and an anatomical image even when fluorescence is excited in the visible range, without having to move filters in and out of the beam path between the individual recordings. In addition, anatomical images do not have to be completely dispensed with, even when fluorescence is present in the visible range. should be excited. In principle, it would also be conceivable to radiate white light and fluorescence excitation light in parallel and to separate fluorescence and white light using the Bayer pattern of a detector. However, due to the associated and expected sensor crosstalk, this requires considerable effort in the evaluation and the expected image quality can be low.

The imaging device can be a microscopic, macroscopic and/or exoscopic imaging device. The imaging device can be designed as a microscope, macroscope and/or exoscope and/or comprise such a device. In some embodiments, the imaging device can be an endoscopic imaging device. The imaging device can be an endoscope device. It can comprise an endoscope and/or an endoscope system and/or be designed as such and/or form at least a part and preferably at least a major part and/or main component of an endoscope and/or an endoscope system. “At least a major part” can mean at least 55%, preferably at least 65%, preferably at least 75%, particularly preferably at least 85% and most preferably at least 95%, in particular with reference to a volume and/or a mass of an object.

In some embodiments, the imaging device is designed to be insertable into a cavity for assessment and/or observation, for example into an artificial and/or natural cavity, such as into the interior of a body, into a body organ, into tissue or the like. The imaging device can also be designed to be insertable into a housing, casing, shaft, pipe or other, in particular artificial, structure for assessment and/or observation.

In particular, if the imaging device is an exoscopic imaging device, it can be configured to record tissue parameters, images of wounds, images of body parts, etc. For example, the imaging device can be configured to image a surgical field.

The image capture sensor system can be configured to detect light in both the visible range and the near-infrared range. In some embodiments, a smallest detectable wavelength can be at most 500 nm, at most 450 nm, or even at most 400 nm. In some embodiments, a largest detectable wavelength can be at least 800 nm, at least 900 nm, or even at least 1000 nm. The image capture sensor system can, for example, comprise at least one white light image sensor and at least one near-infrared image sensor. In some embodiments, the imaging device comprises a white light camera and/or sensors for white light image capture. The imaging device can be configured for white light imaging. The anatomical images can be recorded using the white light camera and/or the sensors for white light image capture.

The image capture unit can have a filter unit with optical observation filters. The filter unit can define several fluorescence modes that are defined by different observation filters. For example, different edge filters can be used that absorb/block the respective spectrum of the associated light element used for excitation and at least essentially only transmit fluorescent light. The observation filter that blocks light in the first spectral range is then part of the filter unit. In some embodiments, the observation filters can also be switchable between a multispectral mode and a fluorescence mode.

The imaging device and in particular an optics and/or the image capture sensor system can be set up for multispectral and/or hyperspectral imaging, in particular for capturing and/or generating multispectral and/or hyperspectral image data. Multispectral imaging or multispectral image data can refer in particular to imaging in which at least two, in particular at least three, and in some cases at least five spectral bands can be and/or are captured independently of one another. Hyperspectral imaging or hyperspectral image data can refer in particular to imaging in which at least 20, at least 50 or even at least 100 spectral bands can be and/or are captured independently of one another. The imaging device can work according to the pushbroom method and/or the whiskbroom method and/or the staring method and/or a snapshot principle.

For some applications, it can be advantageous to be able to use a high spectral resolution. Hyperspectral imaging is then a good option. This can be combined with white light imaging. This enables real-time observation via a white light image, even if the acquisition of spectrally resolved image data is only essentially real-time, i.e. several seconds are needed to create a spectrally resolved image. For some applications, it can be advantageous to generate spectral image data in real-time. This For example, it involves generating a spectrally resolved image in less than a second or even several times per second. In this case, it may be useful to use multispectral imaging. A possibly lower spectral resolution is then offset by a higher frame rate. Depending on the application, it may be sufficient to consider only a few different spectral ranges and/or wavelengths, for example two or three or four or generally less than ten. In this case, additional white light imaging can optionally be dispensed with. Spectrally resolved image data that is acquired in real time or provides several images per second can also be used for surveillance purposes, whereby an image for a user to display does not necessarily have to be created, but the image data can also be processed in the background.

The medical imaging device can have at least one proximal section, one distal section and/or one intermediate section. The distal section is designed in particular to be introduced into and/or located in a cavity to be examined in an operating state, for example during the diagnostic and/or therapeutic action. The proximal section is designed in particular to be arranged outside the cavity to be examined in an operating state, for example during the diagnostic and/or therapeutic action. “Distal” is to be understood as facing a patient and/or facing away from a user, in particular during use. “Proximal” is to be understood as facing away from a patient and/or facing away from a user, in particular during use. In particular, proximal is the opposite of distal. The medical imaging device has in particular at least one, preferably flexible, shaft. The shaft can be an elongated object. Furthermore, the shaft can at least partially and preferably at least to a large extent form the distal section. An "elongated object" is to be understood in particular as an object whose main extension is at least a factor of five, preferably at least a factor of ten and particularly preferably at least a factor of twenty larger than a largest extension of the object perpendicular to its main extension, i.e. in particular a diameter of the object. A "main extension" of an object is to be understood in particular as its longest extension along its main extension direction. A "main extension direction" of a component is to be understood in particular as a direction which runs parallel to a longest edge of a smallest imaginary cuboid which just completely encloses the component.

The image capture unit can be arranged at least partially and preferably at least for the most part in the area of the proximal section and/or form this. In other embodiments, the image capture unit can be arranged at least partially and preferably at least for the most part in the distal section and/or form this. Furthermore, the image capture unit can be arranged at least partially distributed over the proximal section and the distal section. The image capture sensor system has in particular at least one image sensor. Furthermore, the image capture sensor system can also have at least two and preferably several image sensors, which can be arranged one behind the other. Furthermore, the two and preferably several image capture sensors can have spectral detection sensitivities that are different from one another, so that, for example, a first sensor in a red spectral range, a second sensor in a blue spectral range and a third sensor in a green spectral range is particularly sensitive or comparatively more sensitive than the other sensors. The image sensor can be designed as a CCD sensor and/or a CMOS sensor.

An optics of the image capture unit can comprise suitable optical elements such as lenses, mirrors, gratings, prisms, optical fibers, etc. The optics can be configured to guide object light coming from an imaged object to the image capture sensor system, for example to focus and/or project it.

The image capture unit is in particular designed to generate at least two-dimensional spatial image data. The image capture unit can have a spatial resolution in such a way that it delivers a resolution of at least 100 pixels, preferably of at least 200 pixels, preferably of at least 300 pixels and advantageously of at least 400 pixels in at least two different spatial directions. The image data is preferably at least three-dimensional, with at least two dimensions being spatial dimensions and/or with at least one dimension being a spectral dimension. Several spatially resolved images of the image area can be obtained from the image data, each of which is assigned to different spectral bands. The spatial and spectral information of the image data can be such that an associated spectrum can be obtained from it for several spatial pixels.

In some embodiments, the image acquisition unit is configured to generate continuously updated image data. The image acquisition unit can, for example, be configured to generate the image data substantially in real time, which includes, for example, generating updated image data at least every 30 seconds, in some cases at least every 20 seconds, and in some cases even at least every 10 seconds or at least every 5 seconds. Preferably, the image acquisition unit is configured to generate at least the anatomical images and the fluorescence images as well as the representation based thereon in real time, for example with a frame rate of at least 5 fps, at least 10 fps, at least 20 fps, or even at least 30 fps.

The lighting unit can be multimodal and comprise a plurality of independently selectable activatable lighting elements which are designed to emit light according to different emission spectra in order to provide the illumination light.

The term "visible light" can refer here in particular to light with a wavelength between 400 nm and 750 nm. In general, this means light that is perceptible to the human eye. The first spectral range in particular only includes a partial range of visible light. The second spectral range can also include visible light. The observation filter can be an edge filter, in particular a high-pass filter. The edge of the observation filter or, generally speaking, the first spectral range can be selected to match the fluorescent dye used. If, for example, fluorescein or Cy5 is used as the dye, excitation occurs in the middle of the visible range, in particular at a distance from its edges, for example at around 430 nm for fluorescein or at around 660 nm for Cy5. In order not to distort fluorescence images with excitation light, the excitation light must be blocked accordingly. The first spectral range must then be selected in such a way that the excitation light is blocked at least. In the case of fluorescein, for example, an edge filter with an edge at about 450 nm could be selected. In the case of Cy5, for example, an edge filter with an edge at about 680 nm could be selected. The first spectral range and the second spectral range preferably border on one another at least substantially. A spacing between the spectral ranges can, for example, be defined merely by a width of the filter edge. There can be a third spectral range which lies between the first spectral range and the second spectral range and in which the observation filter and in particular the transmittance of the observation filter changes greatly depending on the wavelength. This can, as mentioned, be the filter edge. The first spectral range and/or the second spectral range can comprise a plurality of sub-ranges which can be spaced apart from one another.

In some embodiments, the second spectral range extends into the near-infrared range beyond the visible range. In particular, the light detected when recording the anatomical images can also be at least in the near-infrared range, for example in a wavelength range beyond 800 nm, beyond 850 nm, or even beyond 900 nm.

The fluorescence images and the anatomical images can be recorded in parallel or sequentially. Preferably, both the fluorescence images and the anatomical images are moving images.

Images can be recorded particularly easily and at short intervals if the image capture unit is set up to record the anatomical images of the object to be imaged through the observation filter. The observation filter can therefore be permanently in the beam path for both the anatomical images and the fluorescence images during the recording and display of a superimposed anatomical and fluorescence image. It can be changed, for example, if a different excitation wavelength and/or a different imaging mode is selected. The observation filter is located in particular in front of an optical input of an image capture sensor system of the imaging device. As mentioned, this can include sensors that respond in different spectral ranges; the observation filter is nevertheless expediently arranged in front of all of these sensors, i.e. in front of the image capture sensor system. This makes it easy to use different filters without the filters having to be integrated directly into the sensor system.

In other embodiments, the observation filter can be integrated into the image capture sensor system. In particular for multi-chip image sensors, different image sensors can be equipped with different observation filters. The capture of the anatomical images and the capture of the fluorescence images can then be carried out with disjoint groups of image sensors. Images can be recorded in parallel. In this case, the observation spectral range used for the fluorescence image capture is not available for the anatomical image capture.

A smallest wavelength of the first spectral range can be greater than 430 nm and in particular greater than 500 nm. A largest wavelength of the first spectral range can be less than 800 nm and in particular less than 700 nm. In some embodiments, the illumination light used as excitation light for fluorescence imaging is narrow-band. For example, the illumination light can originate from at least one colored LED, laser diode or another light element that emits at least substantially monochromatically.

In some embodiments, the medical imaging device comprises a lighting device that comprises the lighting unit. The lighting device can comprise an optical interface for optically connecting an imaging device. The lighting unit can be configured to supply illumination light to the optical interface. The lighting unit can be multimodal and comprise a plurality of independently selectably activatable lighting elements that are configured to emit light according to different emission spectra in order to supply the illumination light. The lighting unit can be operable in at least one multispectral mode in which a first group of the lighting elements is at least temporarily activated and in which the lighting unit supplies illumination light for multispectral imaging. Furthermore, the lighting unit can be operable in at least one fluorescence mode in which a second group of the lighting elements is at least temporarily activated and in which the lighting unit supplies illumination light for fluorescence imaging. The lighting elements can comprise at least one lighting element that is contained in both the first group and the second group.

In addition, a method for generating illumination light for an imaging device by means of an illumination device can be provided. The illumination device comprises an optical interface for optically connecting an imaging device and an illumination unit which is designed to supply illumination light to the optical interface, wherein the illumination unit comprises a plurality of independently selectably activatable lighting elements which are designed to emit light according to different emission spectra in order to supply the illumination light. The method comprises the step of at least temporarily activating a first group of the lighting elements in order to supply illumination light for multispectral imaging and the step of at least temporarily activating a second group of the lighting elements in order to supply illumination light for fluorescence imaging. At least one of the lighting elements is at least temporarily activated both when the first group of the lighting elements is at least temporarily activated and when the second group of the lighting elements is at least temporarily activated.

The optical interface can be either detachable or connectable. In addition, the optical interface can be combined with a mechanical interface so that an optical connection is automatically established, for example, when the imaging device is mechanically coupled.

The lighting elements can comprise single-color LEDs (light-emitting diodes) and/or laser diodes. Furthermore, at least one of the lighting elements can be a white light LED or another white light source. In some embodiments, the lighting unit comprises at least one blue lighting element, at least one red lighting element, at least one dark red lighting element and at least one near-IR lighting element (near-infrared lighting element), in particular LEDs or laser diodes. In addition, the lighting unit can comprise at least one white light LED or another white light source.

The first group can comprise at least two light elements that emit spectrally differently. A high degree of efficiency in multispectral imaging can be achieved if the multispectral mode comprises different states in which a specific light element or a specific type of light element is activated at least temporarily. This allows targeted illumination in a specific spectral range, whereby different spectral images can be captured. Different light elements that are activated in different states can serve as different support points for the multispectral imaging. At least one of these support points can be selected such that it is adapted to characteristic points of absorption spectra of physiologically relevant components, for example to an isosbestic point of the hemoglobin oxygenation curve. The multispectral imaging can additionally comprise the use of suitable observation filters.

Furthermore, the second group can comprise at least two light elements that emit spectrally differently. The fluorescence mode can comprise different sub-modes and/or states in which a specific light element or a specific type of light element is activated at least temporarily. This allows targeted excitation in a specific spectral range, so that fluorescence imaging, for example, for a specifically selected dye. In other words, the at least one light element that is contained in both the first group and the second group can be used for both the multispectral mode and the fluorescence mode.

In some embodiments, the first group comprises only some but not all of the light elements. Alternatively or additionally, in some embodiments, the second group comprises only some but not all of the light elements. In the multispectral mode, in particular, only light elements of the first group are activated at least temporarily, whereas light elements that do not belong to the first group are deactivated. In the fluorescence mode, in particular, only light elements of the second group are activated at least temporarily, whereas light elements that do not belong to the second group are deactivated. In general, it is understood that the light elements can comprise different light element types and that, in particular, exactly one light element can be present in each of the different light element types. It is understood that mixed operating modes can also occur according to the invention, in which the modes mentioned are used sequentially. For example, multispectral imaging and fluorescence imaging can be carried out sequentially.

Synergy with regard to the use of a light element for different modes and associated efficiency gains can be achieved in particular if at least one light element contained in both the first group and the second group emits light in the red spectral range, in particular in a spectral range between 600 nm and 680 nm, for example between 610 nm and 650 nm or between 620 and 660 nm or between 630 and 670 nm. The spectral range can be narrowband and include the wavelength 660 nm. "Narrowband" can include a spectral width of at most 80 nm, in particular of at most 40 nm or even of at most 20 nm. This at least one light element can be designed to excite dyes absorbing in the red spectral range and to contribute to the illumination in the red spectral range for multispectral imaging.

In some embodiments, the illumination unit can be operated in at least one white light mode in which the illumination unit supplies illumination light for white light imaging. The illumination light for white light imaging can be broadband white light. Alternatively, the illumination light for white light imaging can comprise several narrow wavelength bands that are separated from one another, for example a blue, a red and a dark red band. "Dark red" is to be understood in the sense of "longer wavelength than red" and refers to the spectral position, not the light intensity. The illumination light for white light imaging can be mixed from light from different lighting elements.

In the white light mode, a third group of the lighting elements can be activated at least temporarily to provide the illumination light for the white light imaging. The lighting elements can comprise at least one lighting element that is contained in both the first group and/or the second group and the third group. In some cases, the third group can comprise only some but not all of the lighting elements. In the white light mode, in particular, only lighting elements of the third group are activated at least temporarily, whereas lighting elements that do not belong to the third group are deactivated. In other words, the lighting unit can comprise lighting elements that serve one, two or all three of the aforementioned lighting modes. This allows multiple lighting elements to be used multiple times.

At least one light-emitting element contained in both the first group and/or the second group and the third group can emit light in the red spectral range, in particular in a spectral range between 600 nm and 680 nm, for example between 610 nm and 650 nm or between 620 and 660 nm or between 630 and 670 nm. The advantages of using light-emitting elements together are particularly evident when at least one red light-emitting element can be used for all three modes.

At least one light-emitting element contained in both the first group and/or in the second group and in the third group can emit light in the blue spectral range, in particular in a spectral range between 440 and 480 nm. At least one blue light-emitting element can expediently be used both in fluorescence mode and in white light mode.

Generally speaking, the lighting elements can comprise at least one, in particular blue, lighting element that emits light in a spectral range between 440 and 480 nm. In addition, the lighting elements can comprise at least one, in particular red, lighting element that emits light in a spectral range between 600 and 680 nm, for example between 610 nm and 650 nm or between 620 and 660 nm or between 630 and 670 nm. Alternatively or in addition, the lighting elements can comprise at least one, in particular dark red, lighting element that emits light in a spectral range between 750 and 790 nm. Alternatively or additionally, lighting elements can comprise at least one, in particular near-IR-emitting, lighting element that emits light in a spectral range between 920 and 960 nm. In addition, the lighting elements can comprise a white light lighting element. A compact and versatile lighting unit can be provided in particular if at least one lighting element of each of the above-mentioned lighting element types is present. For example, in fluorescence mode, the blue and red lighting elements, and in the case of suitable dyes, possibly also the dark red lighting element, can be used. In multispectral mode, the dark red and near-IR-emitting lighting elements can be used. In white light mode, the white light lighting element can be used. In white light mode, this can be supplemented by the blue lighting element and possibly also the red lighting element. This allows wavelength ranges to be supplemented by means of colored light elements in which the white light element delivers a reduced intensity, for example due to its construction but especially due to filters and optical elements of the lighting unit. In addition, the colored light elements can be used to set a color temperature for white light imaging.

In some embodiments, the second group comprises a single light element and/or a single type of light element. For example, a white light light element, a red light element and an IR-emitting light element can be provided, with particular reference being made to the above values with regard to possible spectral ranges. The first group can then comprise, for example, the red and the IR-emitting light element. The second group can comprise the IR-emitting light element, in particular as the only light element or as the only type of light element.

A favorable arrangement of lighting elements is made possible in particular if the lighting unit comprises at least one crossed beam splitter, by means of which light can be deflected from opposite input sides to an output side, with at least one of the lighting elements being arranged on the opposite input sides of the crossed beam splitter. In some embodiments, two or more crossed beam splitters can be provided, which are optically arranged one behind the other. The at least one crossed beam splitter can comprise two beam splitter elements, the permeability of which is adapted to the respectively assigned lighting element. The beam splitter elements each comprise in particular a notch filter, so that they each reflect in a narrow spectral band, but otherwise transmit. The spectral position and/or width of the corresponding notch can be adapted to the spectral range of the respectively assigned lighting element, so that its light is deflected, but light from other lighting elements is at least largely transmitted.

In some embodiments, the lighting elements can comprise at least four narrow-band emitting single-color lighting elements, each with different spectral ranges, and at least one broadband emitting white light lighting element. In this regard, reference is also made to the above statements on the colored lighting elements.

A wide range of functions in combination with a compact design and the exploitation of synergy effects when using lighting elements can be achieved in particular if the lighting unit can be operated in at least one hyperspectral mode in which several lighting elements are activated, the emission spectra of which together cover at least a spectral range from 450 nm to 850 nm, and in which the lighting unit supplies illumination light for hyperspectral imaging. This can in particular be all of the lighting elements.

It is understood that, particularly when laser diodes are used, suitable polarization filters can be used for the optical filters mentioned here. Furthermore, particularly when laser diodes are used, at least one crossed beam splitter can be used, the beam splitter elements of which are provided with polarization filters. Selective permeability can then be achieved by combining different polarizations.

The representation generated by the representation generation unit from the fluorescence images and the anatomy images can comprise a combination representation, in particular an overlay representation based on a combination of fluorescence images and anatomy images. A user can then particularly easily assign fluorescence information to anatomy information. In particular, the fluorescence images and the anatomy images can be directed at the same object, in particular the same anatomy. Image sections can differ from one another.

The imaging device can comprise a display and/or be provided for connection to a display. A corresponding interface can be present for this purpose. The representation generation unit can be set up to generate the representations for the display. The display can, for example, comprise a screen on which the representation is made available to a user.

In some embodiments, the image capture unit is configured to record the fluorescence images and the anatomy images in real time. Furthermore, the representation generation unit can be configured to generate the at least one representation in real time. In this way, an anatomical and a fluorescence-based moving image can be made available to a user, even if excitation occurs in the visible range.

As mentioned, in some embodiments the image acquisition unit can be set up to record the fluorescence images and the anatomy images sequentially. It can then be advantageous if the representation generation unit is set up to assign sequentially recorded fluorescence images and anatomy images to one another, in particular in terms of time, and to generate the at least one representation based on assigned fluorescence images and anatomy images. The assignment can be frame-synchronous. “Frame-synchronous” is to be understood here as meaning that fluorescence images and anatomy images assigned to one another are correlated in time. This does not necessarily mean simultaneity in the narrow sense. In addition, the term “frame-synchronous” also includes situations in which a first number of fluorescence images and a second number of anatomy images that differ from them are recorded alternately. For example, according to the invention it can be provided that an update rate for anatomy images is greater than an update rate for fluorescence images. The recording is still frame-synchronous within the meaning of this disclosure if the assignment is made in such a way that there is a temporal connection between the images displayed. In other words, the combination display can be based on at least one fluorescence image and at least one anatomical image that were recorded within a predetermined period of time, for example within a period of at most 1000 ms, at most 500 ms, at most 100 ms or even at most 50 ms.

In addition to the above statements on the image capture sensor system, it can generally be provided that the image capture sensor system comprises at least one first image capture sensor, which is set up to capture image information for the fluorescence images, and that the image capture sensor system also comprises at least one second image capture sensor, which is set up to capture image information for the anatomical images. The image capture sensors can be combined by means of at least one optical element in such a way that detected light always or at least selectively falls on several or all image capture sensors at the same time. The first image capture sensor can comprise at least one near-infrared image sensor. The second image capture sensor can comprise a white light image capture chip. Alternatively or additionally, the second image capture sensor can comprise different color channels, for example a red, a green and a blue color channel. This can be done, for example, by using a color filter with a suitable pattern. In some embodiments, for example, a Bayer sensor is used, but other arrangements are also possible according to the invention. Furthermore, it can be provided that a separate two-dimensionally resolving image sensor is used for each color channel. It is understood that in some embodiments a fluorescence image can additionally or alternatively also be generated by means of the first image acquisition sensor and/or that an anatomy image can additionally or alternatively also be generated by means of the second image acquisition sensor. The anatomy images can, for example, comprise multiple anatomy images and/or be based on multiple individual images that are assigned to different color channels, such as images from a red color channel, a green color channel, a blue color channel and/or a near-infrared color channel. Alternatively or additionally, a single anatomy image can comprise multiple color channels, for example a red color channel, a green color channel, a blue color channel and/or a near-infrared color channel.

In other embodiments, the image capture sensor system comprises a single image capture sensor, in particular a single-chip image capture sensor and/or a single-chip camera head that images both in the visible range and in the near-infrared range.

In some embodiments, the image capture sensor system can be configured to record fluorescence images and anatomical images simultaneously. “Simultaneously” here means in particular a temporal simultaneity, which differs, for example, from a sequential recording of fluorescence images and anatomical images. In this case, for example, the recording of an anatomical image and at the same time the recording of a fluorescence image can take place within a period of at most 500 ms, at most 100 ms or even as little as 20 ms. However, "simultaneous" may mean that the corresponding images take different amounts of time to capture, for example when different exposure times are used.

Available image information can be used in a targeted manner and to generate a meaningful anatomical image, in particular if the representation generation unit is set up to generate at least one anatomical image from several, in particular single-colored, individual images and/or from several color channels of anatomical images. The anatomical image can be synthesized in this way from several color channels, whereby the loss of a certain color channel and/or a certain spectral range due to the observation filter can be at least partially compensated. Depending on the image capture sensor system used, individual image sensors can also provide individual images, for example single-colored individual images, such as a blue, green, red and/or near-infrared individual image, or also different and possibly partially multi-colored individual images, such as an RGB individual image and a near-infrared individual image.

In some embodiments, the anatomical image corresponds to a white light image with multiple color channels, in which at least one color channel from the visible range is replaced by at least one color channel that lies outside the visible range, in particular in the near-infrared range. This makes it possible to generate an almost natural anatomical image, even if spectral information is lost due to the observation filter.

The invention also relates to program code which, when executed in a processor, is designed to effect implementation of a method according to the invention.

Furthermore, the invention relates to a program code comprising a computer-readable medium on which the program code according to the invention is stored.

The devices and systems according to the invention and the methods according to the invention should not be limited to the application and embodiment described above. In particular, in order to fulfill a function described herein, they can have a number of individual elements, components and units as well as method steps that differs from the number stated herein. In addition, in the value ranges specified in this disclosure, values within the stated limits should also be considered disclosed and can be used as desired.

It is particularly pointed out that all features and properties described in relation to a device, but also procedures, can be transferred to methods and used in the sense of the invention and are deemed to be disclosed. The same applies in the opposite direction. This means that structural features mentioned in relation to methods, i.e. device-related features, can also be taken into account and claimed within the scope of the device claims and can also be counted as part of the disclosure.

The present invention is described below by way of example with reference to the attached figures. The drawing, the description and the claims contain numerous features in combination. The person skilled in the art will expediently also consider the features individually and use them in combination as appropriate within the scope of the claims.

If there is more than one example of a particular object, only one of them may be provided with a reference symbol in the figures and in the description. The description of this example can be transferred to the other examples of the object accordingly. If objects are named in particular using numerical words, such as first, second, third object, etc., these serve to name and/or assign objects. Accordingly, for example, a first object and a third object, but not a second object, can be included. However, a number and/or sequence of objects could also be derived using numerical words.

• 1 eine schematische Darstellung einer Bildgebungsvorrichtung mit einer Beleuchtungsvorrichtung;
• 2 eine schematische Darstellung der Beleuchtungsvorrichtung;
• 3 schematische Transmissionskurven von Strahlteilerelementen der Beleuchtungsvorrichtung;
• 4 eine schematische Darstellung der Bildgebungsvorrichtung;
• 5 eine schematische Darstellung einer weiteren Ausführungsform der Bildgebungsvorrichtung;
• 6 eine schematische Darstellung noch einer weiteren Ausführungsform der Bildgebungsvorrichtung;
• 7 eine schematische perspektivische Darstellung einer weiteren Ausführungsform der Bildgebungsvorrichtung;
• 8 ein schematisches Ablaufdiagramm eines Verfahrens zur Erzeugung von Beleuchtungslicht für ein Bildgebungsgerät mittels einer Beleuchtungsvorrichtung;
• 9 ein schematisches Ablaufdiagramm eines Verfahrens zum Betrieb einer Bildgebungsvorrichtung;
• 10 ein schematisches Ablaufdiagramm eines Verfahrens zum Betrieb einer Bildgebungsvorrichtung;
• 11 eine schematische Darstellung einer medizinischen Bildgebungsvorrichtung;
• 12 schematische Spektren eines Fluoreszenzfarbstoffs sowie eine Transmissionskurve eines zugehörigen Beobachtungsfilters;
• 13 schematische Spektren des Fluoreszenzfarbstoffs sowie eine Transmissionskurve eines alternativen zugehörigen Beobachtungsfilters;
• 14 schematische Darstellungen einer Reihe von Fluoreszenzbildern und Anatomiebildern;
• 15 eine schematische Darstellung von Farbkanälen eines Anatomiebilds;
• 16 ein schematisches Diagramm zur Veranschaulichung eines ersten Beispiels zur Berechnung von Fluoreszenzbildern und Anatomiebilder;
• 17 ein schematisches Diagramm zur Veranschaulichung eines zweiten Beispiels zur Berechnung von Fluoreszenzbildern und Anatomiebilder;
• 18 ein schematisches Diagramm zur Veranschaulichung eines dritten Beispiels zur Berechnung von Fluoreszenzbildern und Anatomiebilder;
• 19 ein schematisches Diagramm zur Veranschaulichung eines vierten Beispiels zur Berechnung von Fluoreszenzbildern und Anatomiebilder;
• 20 ein schematisches Ablaufdiagramm eines Verfahrens zur medizinischen Bildgebung; und
• 21 eine schematische Darstellung eines Computerprogrammprodukts.

Show it:

• 1 a schematic representation of an imaging device with an illumination device;
• 2 a schematic representation of the lighting device;
• 3 schematic transmission curves of beam splitter elements of the lighting device;
• 4 a schematic representation of the imaging device;
• 5 a schematic representation of another embodiment of the imaging device;
• 6 a schematic representation of yet another embodiment of the imaging device;
• 7 a schematic perspective view of another embodiment of the imaging device;
• 8th a schematic flow diagram of a method for generating illumination light for an imaging device by means of an illumination device;
• 9 a schematic flow diagram of a method for operating an imaging device;
• 10 a schematic flow diagram of a method for operating an imaging device;
• 11 a schematic representation of a medical imaging device;
• 12 schematic spectra of a fluorescent dye and a transmission curve of an associated observation filter;
• 13 schematic spectra of the fluorescent dye and a transmission curve of an alternative corresponding observation filter;
• 14 schematic representations of a series of fluorescence images and anatomical images;
• 15 a schematic representation of color channels of an anatomical image;
• 16 a schematic diagram illustrating a first example of calculating fluorescence images and anatomical images;
• 17 a schematic diagram illustrating a second example for calculating fluorescence images and anatomical images;
• 18 a schematic diagram illustrating a third example for calculating fluorescence images and anatomical images;
• 19 a schematic diagram illustrating a fourth example of calculating fluorescence images and anatomical images;
• 20 a schematic flow diagram of a method for medical imaging; and
• 21 a schematic representation of a computer program product.

1 shows a schematic representation of an imaging device 10. In the exemplary case shown, the imaging device 10 is an endoscopic imaging device, specifically an endoscope device. Alternatively, the imaging device 10 could be an exoscopic, a microscopic or a macroscopic imaging device. The imaging device 10 is shown as an example as a medical imaging device. The imaging device 10 is intended, for example, for examining a cavity.

The imaging device 10 has a medical imaging device 14. In the case shown, this is an endoscope.

The imaging device 10 further comprises an illumination device 12 with an optical interface 16 and an illumination unit 18. The imaging device 14 can be optically connected to the optical interface 16. The optical interface 16 can be part of an optical-mechanical interface that can be optionally connected and detachable. The illumination device 14 can optionally be decoupled from the illumination device 12. The illumination unit 18 is designed to supply illumination light to the optical interface 16. When imaging using the imaging device 14, the illumination unit 18 can accordingly provide the required illumination light, which is guided to the illumination device 14 and from there coupled out onto an object to be imaged, such as a site.

In the case shown, the imaging device 10 further comprises a display unit on which images can be displayed that are based on image data that were captured by means of the imaging device 14. These can be video images, still images, overlays of different images, partial images, image sequences, etc.

The imaging device 10 is multimodal. By way of example, the imaging device can be operated in three basic modes, a multispectral mode, a fluorescence mode and a white light mode. Furthermore, it can be provided that the imaging device 10 can be operated in a hyperspectral mode in addition to or as an alternative to the multispectral mode.

The lighting device 12 is multimodal. The lighting device 12 can be operated in different lighting modes in which it provides light for different imaging modes. In the present case, the lighting device 12 can be operated in three basic modes, a multispectral mode, a fluorescence mode and a white light mode. The imaging device 14 can also be operated in different operating modes, specifically also at least in a multispectral mode, a fluorescence mode and a white light mode. In the corresponding operating mode of the imaging device 10, the modes of the lighting device 12 are coordinated with one another.

2 shows a schematic representation of the lighting device 12. The lighting unit 18 comprises several independently activatable lighting elements 20, 22, 24, 26, 28. These are designed to emit light according to different emission spectra in order to provide illumination light, ie the respective emission spectrum differs from lighting element to lighting element.

For example, the light elements 20, 22, 24, 26, 28 are designed as LEDs. Specifically, a first light element 20 is designed as a red LED, a second light element 22 as a dark red LED, a third light element 24 as a blue LED and a fourth light element 26 as a near-IR LED. The colored light elements 20, 22, 24, 26 each emit in a narrow band, for example with an emission peak at wavelengths of around 660 nm (first light element 20), 770 nm (second light element 22), 460 nm (third light element 24) and 940 nm (fourth light element 26).

Furthermore, a fifth light element 28 is provided, which in the present case is a white light element, for example a white light LED. The fifth light element 28 emits, for example, in a spectral range of approximately 400 to 700 nm. In other embodiments, laser diodes can also be used, in particular as colored light elements.

Depending on the lighting mode, some of the lighting elements 20, 22, 24, 26, 28 are activated at least temporarily, whereas other lighting elements 20, 22, 24, 26, 28 may not be used in the lighting mode in question.

In the present case, a first group comprises the first light element 20 and the fourth light element 26. The first group can additionally comprise the light element 22 and/or the light element 24. The first group is used for multispectral imaging, with the light elements 20, 26 and possibly 22 and 24 each serving as a support point. In multispectral mode, for example, the first light element 20 is first illuminated and an image is taken. The fourth light element 26 is then illuminated and an image is taken. The images are each based on remission, i.e. the light scattered back from the object to be imaged is observed. Spectral information about the object to be imaged can be obtained from the two different support points. For example, this can be used to assess certain types of tissue, a perfusion state, a tissue condition or the like.

A second group also includes the first light element 20, the second light element 22 and the third light element 24. The second group is used for illumination in fluorescence imaging. For example, objects colored with suitably selected dyes can be viewed in a targeted manner. Different dyes can also be introduced into different types of tissue or the like, which are viewed at the same time. By specifically exciting a certain dye, it is excited to fluoresce. The fluorescent light is then imaged. The first light element 20 is suitable, for example, for exciting the dye cyanine 5.5 (Cy 5.5). The second light element 22 is suitable for exciting the dye indocyanine green (ICG). The third light element 24 is suitable for exciting the dye fluorescein.

Furthermore, a third group comprises the fifth light element 28. In the present embodiment, the third group also comprises the first light element 20 and the third light element 24. The third group serves to provide illumination light for white light imaging. For this purpose, white light from the fifth light element 28 can be mixed with light from certain colored light elements, whereby spectral losses can be compensated and/or a color temperature can be set in a targeted manner.

It can be seen that some of the light elements 20, 22, 24, 26, 28 are assigned to several groups, for example the first light element 20 to all three groups and the third light element 24 and possibly also the second light element 22 to the second and third groups.

Alternatively or additionally, it can also be provided that some or all of the light elements 20, 22, 24, 26, 28 are used in a hyperspectral mode. A broad excitation spectrum is then generated. In combination with a suitable hyperspectral detector, spectral information about the object to be imaged can then be recorded over the entire visible and near-IR spectrum. For this purpose, the imaging device 14 can comprise a pushbroom arrangement as a hyperspectral detector. In other embodiments, a whiskbroom arrangement, a staring arrangement and/or a snapshot arrangement is used. The imaging device 14 can be a hyperspectral imaging device. Regarding different methods of hyperspectral imaging and the components required for this, please refer to the specialist article "Review of spectral imaging technology in biomedical engineering: achievements and challenges" by Quingli Li et al. Published in Journal of Biomedical Optics 18(10), 100901 , October 2013, and on the Technical article "Medical hyperspectral imaging: a review" by Guolan Lu and Baowei Fei, published in Journal of Biomedical Optics 19(1), 010901 , January 2014.

The lighting unit 18 comprises two crossed beam splitters 30, 32. These each comprise an output side 42, 44, an input side 37, 41 opposite the output side 42, 44 and two input sides 34, 36, 38, 40 opposite each other. All input sides 34, 36, 37, 38, 40, 41 guide incident light to the corresponding output side 42, 44. The output side 42 of a first crossed beam splitter 30 faces an input side 41 of the second crossed beam splitter 32. The output side 44 of the second crossed beam splitter 32 faces the optical interface 16. The two crossed beam splitters 30, 32 are preferably arranged coaxially to each other and/or to the optical interface.

The lighting unit 18 may comprise suitable optical elements such as lenses and/or mirrors (not shown). 2 several lenses 78, 80, 82, 84, 86, 88 are shown. A lens 78 is assigned to the optical interface 16 and couples light coming from the output side 44 of the second crossed beam splitter 32 into the optical interface 16. Furthermore, each of the lighting elements 20, 22, 24, 26, 28 can be assigned a lens 80, 82, 84, 86, 88. A particularly high degree of compactness can be achieved in particular if the lighting elements 20, 22, 24, 26, 28 are each arranged on input sides 34, 36, 37, 38, 40 of the at least one crossed beam splitter 30, 32 without an intermediate mirror. The lighting elements 20, 22, 24, 26, 28 can then be moved very close to at least one crossed beam splitter 30, 32.

The crossed beam splitters 30, 32 each comprise two beam splitter elements 90, 92, 94, 96. These can in principle be partially transparent, so that light from all input sides 34, 36, 37, 38, 40, 41 is redirected to the respective output side 42, 44. In the present embodiment, the beam splitter elements 90, 92, 94, 96 are selectively transparent. This is explained with further reference to 3 The beam splitter elements 90, 92, 94, 96 can be filters that only reflect in a defined area, but otherwise have a high transmission. In 3 Transmission curves 98, 100, 102, 104 of the beam splitter elements 90, 92, 94, 96 of the two crossed beam splitters 30, 32 are shown. Each of the colored light elements 20, 22, 24, 26 or each of the opposite input sides 34, 36, 38, 40 is assigned one of the beam splitter elements 90, 92, 94, 96. The beam splitter elements 90, 92, 94, 96 are selected such that they each reflect in the wavelength range in which the associated light element 20, 22, 24, 26 emits, but also largely transmit. For this purpose, notch filters can be used in the medium wavelength range, which can have the transmission spectra 100 and 102, for example. At spectral edges, high-pass or low-pass filters can be used instead of notch filters, see transmission spectra 98 and 104.

Due to the specific transmission spectra 98, 100, 102, 104 of the crossed beam splitters 30, 32, light from the fifth light element 28 is spectrally clipped. It can therefore be expedient, in the manner already mentioned, to supplement the light blocked by the beam splitters 30, 32 in a targeted manner using the light elements 20 and 24, possibly also 22 and/or 26. This can be supplemented specifically in those spectral ranges in which the beam splitters 30, 32 absorb and/or reflect light from the fifth light element 28, but in any case do not transmit it to the optical interface 16. The additionally used light elements 20, 24 and possibly 22 are preferably operated with reduced power or with adjusted power. The aim here can be to at least largely restore the original spectrum of the fifth light element 28.

In some embodiments, the fifth light element 28 can alternatively be a green light element, or generally speaking, a colored light element that emits primarily in the spectral range that the at least one beam splitter 30, 32 transmits. For example, the fifth light element 26 in such embodiments can be an LED with an emission peak at about 530 nm. A green laser diode is also possible for this. It can be provided that color mixing takes place in white light mode and in particular no individual white light source such as a white light LED is used, but white light from separate light elements is specifically mixed.

It is understood that in the case of suitable dyes, such a green light element can also be used in fluorescence mode. Alternatively or additionally, it could be used in multispectral mode.

The lighting unit 18 defines a common optical path 54 into which emitted light from the lighting elements 20, 22, 24, 26, 28 can be coupled. The common optical path 54 extends from the output side 44 of the second crossed beam splitter 32 to the opti ical interface. The common optical path 54 is arranged coaxially with the fifth light element 26.

In the embodiment shown, the lighting elements 20, 26 of the first group are arranged such that light emitted by the lighting elements 20, 26 travels a light path of at least substantially the same length from the respective lighting element 20, 26 to the optical interface 16. The lighting elements 20, 26 of the first group each have a light-emitting surface 56, 58. The light-emitting surfaces 56, 62 are arranged equidistantly with respect to the common optical path 54. In the present case, this is achieved in that the two lighting elements 20, 26 are arranged at the same distance from the beam splitter 32 assigned to them (in the present case, the second beam splitter 32 by way of example), in particular from its opposite input sides 38, 40. The light is coupled into the common optical path 54 by the crossed beam splitter 32.

The beam splitters 30, 32 are in particular arranged such that light-emitting surfaces 56, 58, 60, 62, 64 of the lighting elements 20, 22, 24, 26, 28 are each arranged equidistantly with respect to their associated crossed beam splitter 30, 32.

By using crossed beam splitters 30, 32 and lighting elements 20, 22, 24, 26, 28 that can be used together for different modes, the lighting unit 18 or the lighting device 12 has a high degree of compactness. In addition, the equidistant arrangement makes it possible to ensure that no spectral shifts occur when the imaging device 14 or its light guide is rotated relative to the optical interface 16.

It is understood that a different number of light elements 20, 22, 24, 26, 28 and/or a different number of crossed beam splitters 30, 32 may be used. The use of crossed beam splitters 30, 32 has proven to be particularly useful. In other embodiments, however, other types of beam splitters and/or other optical elements may be used to couple light from the light elements 20, 22, 24, 26, 28 into the optical interface 16.

4 shows a schematic representation of the imaging device 10. The imaging device 14 is optically coupled to the optical interface 16, for example via a light guide 106 such as at least one optical fiber.

The imaging device 10 has a controller 66 that is designed to automatically coordinate an operating state of the imaging device 14 and a lighting mode of the lighting unit 18. In the present case, a user can specify the operating mode of the imaging device 14 through a user action. The controller 66 then sets the appropriate lighting mode of the lighting unit 18. Alternatively or additionally, the user can set a specific lighting mode of the lighting unit 18 through a user action. The controller 66 can then set an appropriate operating mode of the imaging device 14. The lighting device 12 and/or the imaging device 10 has, for example, a user interface via which the user can enter corresponding commands.

The imaging device 14 comprises a camera unit 68 and a distal shaft 76. The distal shaft 76 is optically coupled to the camera unit 68. The camera unit 68 can have a connection for the distal shaft 76, wherein the distal shaft 76 can be selectively coupled and decoupled. The distal shaft 76 can also be permanently optically and/or mechanically coupled to the camera unit 68. The camera unit 68 is arranged proximally with respect to the shaft 76. The camera unit 68 comprises imaging sensors 108, in the present case, for example, a white light sensor 110 and a near-IR sensor 112. The imaging sensors 108 can, generally speaking, have one or more light sensors/image sensors with at least spatial resolution, for example at least one CMOS sensor and/or at least one CCD sensor. The shaft 76 comprises optical elements (not shown) by means of which light can be guided to the camera unit 68 in order to be able to optically capture the object to be imaged. Furthermore, the shaft 76 comprises at least one light path 114, for example defined by a light guide such as an optical fiber, which leads to a distal section 116 of the shaft 76 and by means of which the illumination light originating from the optical interface 16 of the illumination device 12 can be coupled out to the object to be imaged.

The camera unit 68 has different operating states, specifically for example at least one multispectral operating state and one fluorescence operating state and in the present embodiments additionally a white light operating state and possibly a hyperspectral operating state. The controller 66 automatically adapts the lighting mode of the lighting unit 18 to the current operating state of the camera unit 68. In this case, the controller 66 can make settings to the image recording behavior of the camera unit 68. For example, the controller 66 sets the exposure time, sensitivity/amplification/gain and/or other operating parameters of the camera unit 68 or, in particular, its image capture sensor system 108 and, if applicable, its optics, and thereby defines different operating states of the imaging device 14. In the present case, the controller 66 triggers the lighting unit 18 synchronously with the camera.

The imaging device 14 comprises a filter unit 46 with optical filters 48, 50, 52. Three optical filters are shown as an example, but it is understood that a different number can be used. The filter unit 46 can be switched between a multispectral mode and a fluorescence mode. Furthermore, the filter unit 46 can also be switched to a white light mode and/or a hyperspectral mode. The optical filters 48, 50, 52 can optionally be introduced into an observation beam path 70 of the camera unit 68, thereby defining different observation modes. In the present case, these define the operating states of the camera unit 68.

A basic imaging mode can be assigned to several optical filters 48, 50, 52. In particular, for fluorescence imaging, a different suitable optical filter can be used depending on the light element 20, 22, 24, 26, 28 used for excitation. For example, in the present case the first light element 20 (red) is combined with an optical filter that transmits wavelengths greater than 730 nm, but blocks shorter wavelengths. This can in particular ensure that only fluorescent light and not the excitation light itself is detected. For example, this optical filter can absorb at least in the range 600 nm to 730 nm. Furthermore, in the present case, for example, the second light element 22 (dark red) is combined with a filter that absorbs in the range from 700 to 850 nm or that only transmits significantly above 850 nm.

The user can select a specific filter 48, 50, 52 and thereby immediately selects an associated observation mode or operating state of the camera unit 68. For this purpose, the camera unit 68 has a filter sensor 72 that can automatically detect an optical filter currently inserted into the observation beam path 70. The user can thus manually insert a selected filter 48, 50, 52 into the observation beam path 70. In the example shown, the optical filters 48, 50, 52 are attached to a filter carrier 118. This can be moved into different positions, whereby one of the optical filters 48, 50, 52 can be selected at a time. The filter sensor 72 then detects the currently selected optical filter 48, 50, 52. The control can then determine the current operating state of the camera unit 68 and thus of the imaging device 14 based on a sensor signal from the filter sensor 72 and automatically adapt the lighting mode of the lighting unit 18 accordingly. The user thus puts the entire imaging device 10 into the desired mode by a simple user action such as manually selecting an optical filter 48, 50, 52. In principle, a user can combine different filters with different lighting modes and thereby generate different types of contrast.

In the case shown, the imaging device 14 and in particular the shaft 76 comprises a broadband transmitting optic 77, which can be used uniformly in the different illumination modes. In the present case, the broadband optic 77 is designed for a spectral range of at least 400 nm to 1000 nm. It can be used uniformly for different illumination and/or observation spectral ranges.

In some embodiments, the imaging device 14 can be designed as a stereo endoscope that includes a stereoscopic eyepiece with two sides. Different optical filters can be connected to these sides independently of one another, whereby different contrast images can be superimposed on one another.

In the following, in the context of further embodiments and modifications, the same reference numerals as above are used for identical or similar components. With regard to their description, reference is generally made to the above statements, whereas the following primarily explains differences between the embodiments. In addition, reference numerals have been partially omitted in the following figures for reasons of clarity.

5 shows a schematic representation of a further embodiment of the imaging device 10. The imaging device 10 comprises an illumination device 12 with an optical interface 16 and an illumination unit 18 as well as an imaging device 14 which is connected to the optical interface 16. The imaging device 14 comprises a camera unit 68 with an automated filter unit 210. The automated filter unit 210 comprises a plurality of optical filters 48, 50, 52 which can be automatically introduced into an observation beam path 70 of the camera unit 68 in accordance with an observation mode specified by a user.

The automated filter unit 210 comprises a filter drive 212, which is designed to automatically move the optical filters 48, 50, 52 into the observation beam path 70 or out of the observation beam path 70. The optical filters 48, 50, 52 can be mounted on a filter carrier 118, which is connected to the filter drive 212. The filter drive 212 can be designed to move the filter carrier 118, for example to shift and/or rotate and/or pivot it.

The imaging device 14 has a user interface 214 by means of which the user can set a desired observation mode. For example, a desired position of the filter carrier 118 can be specified by means of the user interface 214.

The imaging device 14 also has a controller 66. The controller 66 is coupled to the filter drive 212 and the user interface 214. The controller 66 is particularly designed to process a user specification of an observation mode and to control both the filter unit 210 and the lighting unit 18 in accordance with this user specification. The controller 66 can thus set an operating state of the imaging device 14 and a lighting mode of the lighting unit 18 that is coordinated with this in accordance with an observation mode selected by the user.

6 shows a schematic representation of yet another embodiment of the imaging device 10. The imaging device 10 comprises an illumination device 12 with an optical interface 16 and an illumination unit 18 as well as an imaging device 14 which is connected to the optical interface 16. The imaging device 14 comprises a proximal base unit 310. The proximal base unit 310 is connected to the optical interface 16 of the illumination device 12. Illumination light generated by the illumination device 12 can thus be fed to the proximal base unit 310. The imaging device 14 further comprises a controller 66 which can be integrated into the base unit 310 in some embodiments.

Different interchangeable shafts 312, 314 can optionally be optically or electronically coupled to the proximal base unit 310. The base unit 310 has an interface 316 for coupling different interchangeable shafts 312, 314. This interface 316 supplies the illumination light coming from the illumination device 12 to a coupled interchangeable shaft 312, 314. Furthermore, the interface 316 is designed to electrically supply a coupled interchangeable shaft 312, 314 and/or to connect it electronically to the controller 66 of the imaging device 14.

The interchangeable shafts 312, 314 each have an integrated camera 318, 320 and integrated optical filters 322, 324. The integrated cameras 318, 320 are designed as tipcams. In the present case, the integrated camera 318 of a first interchangeable shaft 312 is set up for multispectral imaging. Furthermore, the integrated camera 310 of a second interchangeable shaft 314 is set up for fluorescence imaging. The optionally available optical filters 322, 324 can be adapted to this.

In other embodiments, interchangeable shafts can also be used that only contain optical filters but no integrated camera. These can then be coupled to a proximal camera unit. The proximal camera unit can then in some cases be designed without an additional filter unit. The selection of a specific optical filter or a specific observation mode can be made by choosing a suitably equipped interchangeable shaft.

The controller 66 is configured to detect a coupled interchangeable shaft 312, 314. This can be done software-based, mechanically and/or by sensor detection. Depending on the interchangeable shaft 312, 314 detected, the controller 66 can then determine in which operating state or in which observation mode the imaging device 14 should be operated. The control unit 66 is also configured to set an illumination mode of the illumination unit 18. The control unit 66 is thus configured to set an illumination mode of the illumination unit 18 depending on the observation mode defined by a currently coupled interchangeable shaft 312, 314.

In the present case, the interchangeable shafts 312, 314 and the imaging device 10 are part of a medical imaging system 316. The medical imaging system 316 allows a user to select a suitable interchangeable shaft 312, 314, to couple it to the base unit 310, and thus to set a mode for the entire imaging device 10. By simply changing the interchangeable shaft 312, 314, the illumination device 18 is automatically adapted to the image acquisition mode to be used.

7 shows a schematic perspective view of another embodiment of an imaging device 10'. The reference numerals of this embodiment are provided with inverted commas for differentiation. The imaging In this embodiment, device 10' is designed as an exoscopic imaging device. It comprises an illumination device 12' and an imaging device 14'. Their basic functionality corresponds to that described above, but in this embodiment the imaging device 14' is designed as an exoscope.

Aspects of the above description can also be summarized or described as follows. 8th shows a schematic flow diagram of a method for generating illumination light for an imaging device 14 by means of an illumination device 12. The sequence of the method also follows from the above explanations. The illumination device 12 comprises an optical interface 16 for optically connecting an imaging device 14 and an illumination unit 18 which is designed to supply illumination light to the optical interface 16, wherein the illumination unit 18 comprises a plurality of independently selectively activatable light elements 20, 22, 24, 26, 28 which are designed to emit light according to different emission spectra in order to supply the illumination light.

The method comprises a step S11 of at least temporarily activating a first group of the light elements 20, 22, 24, 26, 28 to provide illumination light for multispectral imaging. The method further comprises a step S12 of at least temporarily activating a second group of the light elements 20, 22, 24, 26, 28 to provide illumination light for fluorescence imaging. One of the light elements 20, 22, 24, 26, 28 is at least temporarily activated both when the first group of the light elements 20, 22, 24, 26, 28 is at least temporarily activated and when the second group of the light elements 20, 22, 24, 26, 28 is at least temporarily activated.

9 shows a schematic flow diagram of a method for operating an imaging device 10. The sequence of the method also follows from the above explanations. In a step S21, an imaging device 10 with an imaging device 14 is provided. In a step S22, illumination light is supplied to the imaging device 14. The supply of illumination light to the imaging device 14 takes place according to a method as described with reference to 8th was described.

10 shows a schematic flow diagram of a method for operating an imaging device 10. The sequence of the method also follows from the above explanations. The method comprises a step S31 of providing a lighting device 12 for providing illumination light for an imaging device 14. The imaging device 14 comprises an optical interface 16 for optically connecting an imaging device 14 and a lighting unit 18 which is designed to supply illumination light to the optical interface 16. The lighting unit 18 is multimodal and can be operated in several different lighting modes. The method further comprises a step S32 of providing an imaging device 14 which can be connected to the optical interface 16 of the lighting device 12. The method also comprises a step S33 of automatically coordinating an operating state of the imaging device 14 and a lighting mode of the lighting unit 18.

The following describes an aspect concerning the generation of representations from fluorescence images and anatomical images. 11 shows a schematic representation of a medical imaging device 410 according to this aspect. The medical imaging device 410 can basically be constructed and/or designed like the imaging device 10 described above or also like the above imaging device 10'. In particular, reference is made to the above description with regard to the functionality of the components and details of the design of the imaging device 410. To explain the present aspect, it is expedient to explain the technical facts with reference to the purely schematic 11 and describe the other characters.

In the specific example, the imaging device 410 is an endoscope device, but can also be an exoscope device and/or a microscope device. The imaging device 410 comprises an illumination unit 412 with at least one light source 414. The illumination unit 412 can be designed, for example, as described above with reference to the illumination device 12. For the following description, it is assumed that the illumination unit 412 is designed in this way. However, this is to be understood purely as an example. Basically, the illumination unit 412 is designed to provide illumination light 432, 438, by means of which an object 418 to be imaged can be illuminated. This will be discussed in more detail below.

The imaging device 410 further comprises an image capture unit 420 with suitable image capture sensor system 422. The image capture sensor system 422 is configured in the present case to be able to capture images in both the visible range and the near infrared range. For example, the image capture sensor system 422 at least in one Sensitive in the range between 450 nm and 950 nm, in some embodiments in a range between 400 nm and 1000 nm.

The object 418 to be imaged is, for example, an anatomical structure, for example in a patient's cavity. The anatomical structure comprises an area 454 marked with a fluorescent dye. Using the imaging device 410, a user can view the anatomical structure and the marked area 454 and use the fluorescence of the marked area 454 to distinguish it from the surrounding tissue. As an example application, reference is made to freeing up the marked area 454, although the person skilled in the art is also familiar with other applications of fluorescent marking of tissue.

In the present case, the marked area 454 is marked with a fluorescent dye that absorbs light in the visible range and also fluoresces at least partially or exclusively in the visible range. An exemplary absorption spectrum and an exemplary emission spectrum are shown in the upper region of the 12 as a solid or dotted line in a diagram in which an intensity is plotted against the wavelength. To excite the dye, light is irradiated with a wavelength that lies in the absorption range of the dye. This is shown in 12 by a dashed line, with the relevant wavelength marked as a point on the x-axis. This is the illumination light 432, which is 11 shown as an arrow.

The dye is, for example, fluorescein. This dye can be conveniently excited at a wavelength of 430 nm or 460 nm. In particular, this can be done using the third light element 24 described above. Another example of the dye is Cy5. This dye can be conveniently excited at a wavelength of 660 nm. In particular, this can be done using the first light element 20 described above.

In order to be able to carry out fluorescence imaging, the image acquisition unit 420 comprises an observation filter 424. This is mounted in front of the image acquisition sensor system 422 and/or integrated into it and ensures that the illumination light 432 is not remitted to the image acquisition sensor system 422 or at least not to individual sensors thereof, but exclusively or at least primarily light 434 that is emitted by the object 418 due to the fluorescence excitation of the dye.

In the lower part of 12 a suitable transmission spectrum of the observation filter 424 is shown, which allows such fluorescence imaging. The observation filter 424 is designed to block light in a first spectral range 426 and to transmit it in a second spectral range 428 that is different from the first spectral range 426. The observation filter 420 is suitable for selection depending on the dye used or the absorption spectrum and emission spectrum. As described above, the imaging device 410 can comprise a plurality of optical filters that can be used selectively as observation filters. As a result, fluorescence imaging for different fluorescent dyes can be carried out using the imaging device 410.

The observation filter 424 with the 12 The transmission shown is a high-pass edge filter. In the area of the edge, the transmission behavior of the observation filter 424 changes abruptly, although the corresponding edge can still have a certain width. The area of the edge may not fall into either of the two spectral ranges 426, 428.

Depending on the position, width and shape of the absorption peak of the dye in question, an observation filter with two edges can be used instead of an observation filter 424 with a single edge. This is exemplified in 13 illustrated. The first spectral range 426' lies here between two sub-ranges of the second spectral range 428'. Light is therefore transmitted both above and below the first spectral range 426'. In this case, the observation filter only blocks out a sub-range from the observable spectral range that lies in the visible range.

The image capture unit 420 is further configured to operate in a white light mode or an anatomy mode in which the object 418 is illuminated by means of illumination light 438 which lies at least partially within the second spectral range 428 and in which light 440 remitted by the object 418 is detected. As in 11 As shown schematically, a portion of the remitted light 440 is blocked by the observation filter 424 and the portion that lies outside the first spectral range 426 or in the second spectral range 428 is passed through to the image capture sensor 422. If the first spectral range 426 is selected to lie in the visible range due to the dye used, a complete white light image cannot be recorded, even if white light is irradiated. This can be done, for example, by means of a white light luminous element, as described above.

In the following, additional reference is made to 14 The image capture unit 420 is configured to record fluorescence images 430 of the object to be imaged through the observation filter 424 in the manner described. Furthermore, the image capture unit 420 is configured to record anatomical images 436 of the object 418 to be imaged in the manner described. In the present case, the anatomical images 436 are images in which broadband illumination light is irradiated and remitted light is detected. In 14 the case is shown in which several anatomical images 436 and several fluorescence images 430 are recorded over time or obtained from corresponding image data. Specifically, these are either recorded sequentially by alternately recording one or more anatomical images 436 and then one or more fluorescence images 430 (or vice versa) according to a specific logic, or by recording anatomical images 436 and fluorescence images 430 simultaneously. This depends on the design and operating mode of the image acquisition sensor system 422.

The anatomical images 436 are not true white light images due to the recording through the observation filter 424. Rather, they are obtained by processing individual images and/or color channels according to one or more of the principles described below. The imaging device 410 further comprises a representation generation unit 442, which is set up to generate a representation 444 from the fluorescence images 430 and the anatomical images 436. In the present case, this comprises a combination representation and/or an overlay representation, which is based on a combination or overlay of one or more anatomical images 436 and one or more fluorescence images 430.

The representation generation unit 442 is set up to assign recorded fluorescence images 430 and recorded anatomy images 436 to one another. This can be done both in the case of sequential image recording and in the case of simultaneous image recording. In the case shown, this is a temporal assignment. The assignment is based on the information about the time at which the images were recorded, so that the representation 444 contains temporally related anatomy information and fluorescence information. For example, a moving image is made available to a user in real time.

As in 11 As shown schematically, the imaging device 410 comprises a display 452 via which the generated representation 444 can be displayed to a user. This embodiment is to be understood purely as an example. In particular, in other embodiments, a display can be connected to the imaging device 410 only via an interface but is not necessarily part of the same.

As mentioned, even with broadband illumination, image information is not available over the entire visible range when recording the anatomical images 436 due to the observation filter 424. If regular white light imaging were performed, the white light image in question would be distorted in color or would possibly not correctly display certain anatomical structures.

It is therefore intended that, in addition to image information in the visible range, image information from the near-infrared range is also used to generate the anatomical image 436. In this way, the information that may not be available due to the observation filter 424 can be provided elsewhere.

This is purely exemplary in 15 for four color channels of an image, as would result from illumination with white light and near-infrared light. For the purposes of explanation, it should be assumed that certain anatomical structures or other image information can only be recognized in the blue (B), green (G) or red (R) color channel and/or individual image. If these color channels and/or individual images B, G, R were fully combined, a correct white light image of the object 418 would be created. However, since the imaging takes place through the observation filter 424, the image information from the blue color channel and/or blue individual image, for example, is missing. If only the green and red color channels and/or the green and red individual images were combined, an incomplete image would result. The present aspect, on the other hand, provides for the additional addition of a near-infrared color channel (NIR) and/or a near-infrared individual image. This may make it possible to recognize details of the anatomical structure that would be recognizable in the unavailable spectral range (here, for example, in the blue). If this additional image information is now combined with the available color channels and/or individual images, an anatomical image 436 can be obtained that correctly represents the anatomy, although it is not a true white light image.

Anatomical images 436 obtained in this way are then combined with fluorescence images 34 in the manner described to generate the representation 444, which contains information regarding anatomy of the object 418 as well as regarding fluorescence of the marked region 454.

Below, various possibilities are described as to how anatomical images 436 can be obtained for different configurations of the image capture sensor system 422 and/or different approaches to the utilization of available information. In each case, it is assumed that a multimodal lighting unit is used. This has, for example, and in accordance with the embodiment described above, a white light image lighting element and a blue lighting element that emits, for example, at 430 nm or 460 nm, a red lighting element that emits, for example, at 660 nm, a dark red lighting element that emits, for example, at 765 nm or 770 nm, and a near-infrared lighting element that emits, for example, at 940 nm. As already mentioned, however, this is primarily for understanding, and lighting units with different configurations can also be used.

16 illustrates a first example in which the image capture sensor system 422 comprises a two-chip image sensor. Imaging light is distributed between two sensors, as already described above by way of example. A first image sensor (VIS) operates in the visible range, for example at wavelengths up to about 700 nm, and a second image sensor (NIR) operates in the near-infrared range, for example at wavelengths from about 700 nm. In this example, fluorescein is used as the dye. This is excited using the blue light element. Due to the observation filter 424, blue light cannot be detected. In addition to the illumination for exciting the dye, illumination is provided, for example, using the dark-red light element and/or the near-infrared light element.

Fluorescence images 430 are obtained using the VIS image sensor. Anatomy images 436 are obtained using the NIR image sensor. The anatomy image 436 is output as a grayscale image, so that the capture in the near infrared range, which is not perceptible to the user, does not represent an obstacle. The representation generation unit 442 thus generates a representation 444 that superimposes a grayscale anatomy image 436 and a fluorescence image 434 on one another. Both the fluorescence images 430 and the anatomy images 436 can be recorded in real time, for example at at least 24 fps, at least 30 fps or even at least 60 fps. Since different image sensors are used to capture the fluorescence image 430 and the anatomy image 436, they can also be captured simultaneously. Sequential recording is not required.

17 illustrates a second example in which the image capture sensor system 422 also comprises a 2-chip image sensor. As before, a fluorescence image 430 is recorded using the VIS image sensor. Unlike in the first example, however, anatomy images 436 and fluorescence images 430 are recorded sequentially. This makes it possible to use not only the NIR image sensor but also the VIS image sensor for the anatomy image 436.

The VIS image sensor provides three color channels 436B, 436G, 436R, which can generally be used for the anatomical image 436. Due to the observation filter 424, the blue color channel 436B can hardly or not at all be used, since little to no light remitted by the object 418 passes through the observation filter 424 in the corresponding spectral range. The blue color channel 436B is therefore not taken into account here. In contrast, both the green color channel 436G and the red color channel 436R of the VIS image sensor image data and a near-infrared single image 436NIR of the NIR image sensor are used to obtain the anatomical image 436. The anatomical image 36 is therefore reconstructed by replacing the blue channel with the near-infrared channel or the near-infrared single image. A colored anatomical image can thus be generated. However, due to the sequential acquisition of anatomical images 436 and fluorescence images 430, the frame rate is halved in this example.

In some use cases, a lower frame rate may be acceptable for the fluorescence information than for the anatomy information. Starting from an available frame rate of 60fps, for example, several anatomy images, such as two or three or five, can always be recorded one after the other before a fluorescence image is recorded. The representation generation unit then assigns the most recently recorded fluorescence image to each of the several anatomy images. For example, with this approach, 50 anatomy images and 10 fluorescence images can be recorded per second. The actual frame rate of the fluorescence information in this example is then only 10 fps, but this may not be noticeable to the user due to the higher frame rate of the anatomy information.

18 illustrates a third example. In this example, the image capture sensor system 422 comprises a one-chip image sensor (OC). This uses a red, a green and a blue channel and is also able to detect near-infrared light on all three channels. In the example, the dye Cy5 is excited. This is done, for example, by means of a red light-emitting element.

Anatomical images 436 and fluorescence images 430 are recorded sequentially. The image data of the image sensor can be used unchanged for the fluorescence image 430.

In this case, the observation filter 424 blocks out a first spectral range 426 that extends from about 600 nm to about 700 nm. The second spectral range 428 lies both above and below the first spectral range 426 (see the exemplary illustration in 13 ). Light in the first spectral range 28 is not detected due to the observation filter 424. If an anatomical image 436 is recorded, the red channel still provides a signal because near-infrared light is detected by all three channels of the image sensor. The anatomical image 436 is calculated in this example by combining the three channels in a suitable manner. An anatomical image 436 with the color channels R'G'B' is obtained, for example, by using the red channel of the image sensor as the red channel of the anatomical image 436, i.e. R'=R, by using a difference between the green and red channels of the image sensor as the green channel G', i.e. G'=GR, and by using a difference between the blue and red channels of the image sensor as the blue channel of the anatomical image 436, i.e. B'=BR.

It is understood that any other combinations, in particular linear combinations, can be used. These can also be weighted if necessary depending on the response behavior of the image sensor in question and/or depending on the observation filter 424 used. As a further example, in the case of excitation of fluorescein, in which blue light would be blocked accordingly, the color channels of an anatomical image 436 could be calculated as follows: R'=R-B; G'=G-B; B'=B.

19 illustrates a fourth example. In this example, the image capture sensor system 422 comprises a four-chip image sensor with separate image sensors for a blue channel (B), a green channel (G), a red channel (R) and a near infrared channel (NIR). Light coming from the object 418 is divided between these four sensors. In this example, an illumination unit can also be used which, instead of a white light illumination element, comprises a combination of three or more illumination elements by means of which light is mixed in such a way that it can be used instead of white light. For example, a combination of a red, green and blue illumination element can be used. Although these only emit in certain and possibly very narrow-band wavelength ranges, illumination can be provided for recording a red, green and blue image so that an image can be obtained that is perceived by a user as a white light image. In this case, for example, the single-colored illumination elements mentioned above are present and in addition the mentioned combination of colored illumination elements that are used instead of the white light illumination element.

In principle, this configuration can be operated analogously to the first and second examples described above. Then, for example, the observation filter 424 is arranged in front of the four-chip image sensor. A further variant will be explained below which can be implemented using the system according to the fourth example.

Fluorescein, for example, is again used as a dye. To excite the dye, light is irradiated using a blue light element. Remitted light is in the green range and is detected by the green image sensor. A filter integrated into the green image sensor and/or arranged in front of the image sensor, which makes it selective for green light, serves as an observation filter 424. The image of the green image sensor can thus be used as a fluorescence image 430, since the blue excitation light does not fall on the green image sensor.

In order to obtain an additional anatomical image, for example, illumination is provided by a near-infrared light element. The object thus remits light in both the blue and near-infrared ranges. In these ranges, the dye used also does not fluoresce. The dye fluorescein emits predominantly in the green range, and its emission spectrum also extends into the red with a low intensity. The green and red image sensors are therefore not used for the anatomical image 436. The anatomical image 436 is thus obtained by combining a blue individual image 436B recorded by the blue image sensor and a near-infrared individual image 436NIR recorded by the near-infrared image sensor.

It is understood that an analogous procedure can be followed if another dye is used which, for example, can be excited with green light and emits red light.

20 shows a schematic flow diagram of a method for medical imaging. The sequence of the method also follows from the above explanations. In a step S41, illumination light 438 is provided for illuminating an object 418 to be imaged. In a step S42, fluorescence images 430 of the object 480 to be imaged are recorded through an observation filter 424, which is designed to capture light in a first spectral range 426. blocking light which comprises visible light and transmitting light in a second spectral range 428 which is different from the first spectral range 426, wherein when recording the fluorescence images 430, illumination light 432, the spectrum of which lies at least partially within the first spectral range 426, is used as excitation light and light 434 emitted by the object 418 to be imaged, the spectrum of which lies at least partially within the second spectral range 428, is detected. The method further comprises a step S43 in which anatomical images 436 of the object 418 to be imaged are recorded, wherein when recording the anatomical images 436, illumination light 438, the spectrum of which lies at least partially within the second spectral range 428, is used and light 440 remitted by the object 418 to be imaged, the spectrum of which lies at least partially within the second spectral range 428, is detected.

21 shows a schematic representation of a computer program product 448 with a computer-readable medium 450. The computer-readable medium has program code stored thereon that is configured to cause one and/or all of the described methods to be carried out when executed in a processor.

List of reference symbols

10

Imaging device

12

Lighting device

14

Imaging device

16

optical interface

18

Lighting unit

20

Light element

22

Light element

24

Light element

26

Light element

28

Light element

30

Beam splitter

32

Beam splitter

34

Home page

36

Home page

37

Home page

38

Home page

40

Home page

41

Home page

42

Home Page

44

Home Page

46

Filter unit

48

filter

50

filter

52

filter

54

optical path

56

light-emitting surface

58

light-emitting surface

60

light-emitting surface

62

light-emitting surface

64

light-emitting surface

66

steering

68

Camera unit

70

Observation beam path

72

Filter sensor

74

Display unit

76

shaft

77

optics

78

lens

80

lens

82

lens

84

lens

86

lens

88

lens

90

Beam splitter element

92

Beam splitter element

94

Beam splitter element

96

Beam splitter element

98

Transmission spectrum

100

Transmission spectrum

102

Transmission spectrum

104

Transmission spectrum

106

Light guide

108

Imaging sensors

110

White light sensor

112

Near-IR sensor

114

Light path

116

distal section

210

Filter unit

212

Filter drive

214

User interface

310

Base unit

312

Changeable

314

Changeable

316

Imaging system

318

camera

320

camera

322

filter

324

filter

410

Imaging device

412

Lighting unit

414

Light source

416

Illumination light

418

object

420

Image capture unit

422

Image capture sensors

424

Observation filter

426

first spectral range

428

second spectral range

430

Fluorescence image

432

Illumination light

434

Light

436

Anatomy picture

438

Illumination light

440

Light

442

Representation generation unit

444

Depiction

448

Computer program product

450

computer-readable medium

452

Advertisement

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions.

Cited patent literature

• DE 202014010558 U1 [0004]
• DE 102020105458 A1 [0006]

Cited non-patent literature

• "Review of spectral imaging technology in biomedical engineering: achievements and challenges" by Quingli Li et al. Published in Journal of Biomedical Optics 18(10), 100901 [0082]
• "Medical hyperspectral imaging: a review" by Guolan Lu and Baowei Fei, published in Journal of Biomedical Optics 19(1), 010901 [0082]

Claims (16)

Medical imaging device (410), in particular endoscope device, exoscope device and/or microscope device, comprising: - an illumination unit (412) with at least one light source (414) which is designed to provide illumination light (416) for illuminating an object (418) to be imaged; and - an image capture unit (420) which comprises at least one image capture sensor system (422) and at least one observation filter (424), wherein the observation filter (424) is designed to block light in a first spectral range (426) which comprises visible light and to transmit light in a second spectral range (428) which is different from the first spectral range (426); wherein the image capture unit (420) is configured to record fluorescence images (430) of the object (418) to be imaged through the observation filter (424), during the recording of which illumination light (432), the spectrum of which lies at least partially within the first spectral range (426), is used as excitation light and during the recording of which light (434) emitted by the object (418) to be imaged, the spectrum of which lies at least partially within the second spectral range (428), is detected by the image capture sensor system (422), and wherein the image capture unit (420) is configured to record anatomical images (436) of the object (418) to be imaged, during the recording of which illumination light (438), the spectrum of which lies at least partially within the second spectral range (428), is used and during the recording of which light (440) remitted by the object (418) to be imaged, whose spectrum lies at least partially within the second spectral range (428), is detected by the image capture sensor system (422); and - a representation generation unit (442) which is configured to generate at least one representation (444) from the fluorescence images (430) and the anatomical images (436). The medical imaging device (410) of claim 1, wherein the image acquisition unit (24) is configured to capture the anatomical images (436) of the object (84) to be imaged through the observation filter (424). Medical imaging device (410) according to one of the preceding claims, wherein a smallest wavelength of the first spectral range (426) is greater than 430 nm and in particular greater than 500 nm. Medical imaging device (410) according to one of the preceding claims, wherein the at least one representation (444) comprises a combination representation, in particular an overlay representation, which is based on a combination of fluorescence images (34) and anatomical images (436). Medical imaging device (410) according to one of the preceding claims, wherein the image acquisition unit (420) is configured to record the fluorescence images (430) and the anatomical images (436) in real time, and wherein the representation generation unit (442) is configured to generate the at least one representation (444) in real time. Medical imaging device (410) according to one of the preceding claims, wherein the image acquisition unit (24) is configured to record the fluorescence images (430) and the anatomical images (436) sequentially, wherein the representation generation unit (442) is configured to assign sequentially recorded fluorescence images (430) and anatomical images (436) to one another, in particular in terms of time, and to generate the at least one representation (444) based on assigned fluorescence images (430) and anatomical images (436). Medical imaging device (410) according to one of the preceding claims, wherein the image acquisition sensor system (422) comprises at least one first image acquisition sensor (VIS, NIR, OC, R, G, B) which is configured to acquire image information for the fluorescence images (430), and wherein the image acquisition sensor system (422) comprises at least one second image acquisition sensor (VIS, NIR, OC, R, G, B) which is configured to acquire image information for the anatomical images (436). Medical imaging device (410) according to one of the preceding claims, wherein the image acquisition sensor system (422) is configured to record fluorescence images (430) and anatomical images (436) simultaneously. Medical imaging device (410) according to one of the preceding claims, wherein the representation generation unit (442) is configured to generate at least one anatomical image (436) from a plurality of, in particular single-colored, individual images (436B', 436NIR`) and/or from a plurality of color channels of anatomical images (436B, 436G, 436R, 436NIR). Medical imaging device (410) according to one of the preceding claims, wherein the anatomical image (436) comprises at least two color channels, of which at least one is in the visible range and at least one is outside the visible range. range, particularly in the near infrared range. Medical imaging device (410) according to one of the preceding claims, wherein the anatomical image (436) corresponds to a white light image with multiple color channels, in which at least one color channel from the visible range is replaced by at least one color channel that lies outside the visible range, in particular in the near-infrared range. Medical imaging device (410) according to one of the preceding claims, wherein the illumination unit (412) is multimodal and comprises a plurality of independently selectable activatable lighting elements (20, 22, 24, 26, 28) which are configured to emit light according to different emission spectra in order to provide the illumination light (438). Medical imaging device (410) according to one of the preceding claims, wherein the image acquisition unit (420) is configured for multispectral and/or hyperspectral imaging. Method for medical imaging, in particular with a medical imaging device (410) according to one of the preceding claims, comprising: Providing illumination light (438) for illuminating an object (418) to be imaged; Recording fluorescence images (430) of the object (480) to be imaged through an observation filter (424) which is designed to block light in a first spectral range (426) which includes visible light and to transmit light in a second spectral range (428) different from the first spectral range (426), wherein when recording the fluorescence images (430), illumination light (432) whose spectrum lies at least partially within the first spectral range (426) is used as excitation light and light (434) emitted by the object (418) to be imaged, whose spectrum lies at least partially within the second spectral range (428), is detected; and recording anatomical images (436) of the object (418) to be imaged, wherein when recording the anatomical images (436), illumination light (438) whose spectrum lies at least partially within the second spectral range (428) is used and light (440) remitted by the object (418) to be imaged, whose spectrum lies at least partially within the second spectral range (428), is detected. Program code which, when executed in a processor, is designed to perform a method according to Claim 14 to effect. Computer program product (448) comprising a computer-readable medium (450) on which program code according to Claim 15 is stored.

Images