DE102022121505A1 - Method, computer program and data processing unit for preparing the observation of a fluorescence intensity, method for observing a fluorescence intensity and optical observation system - Google Patents

Abstract

A method is provided for preparing the observation of a fluorescence intensity of fluorescence radiation of a fluorescent dye in an observation object (3), which comprises object regions (3A-H) that differ from one another in their depth and/or orientation, wherein the observation is carried out with a optical observation system (100) should be carried out, with which fluorescence radiation can be observed, provided that it has a certain minimum intensity. The method comprises the steps: - determining the parameter value of at least one parameter influencing the observation of the fluorescence intensity, and - simulating the fluorescence intensity to be expected for the respective object areas (3A-H) on the basis of the determined parameter value of the at least one parameter and a model for the Influence of the at least one parameter on the fluorescence intensity. As part of the simulation, a minimum concentration of the fluorescent dye is specified and the simulation is used to determine the fluorescence intensity to be expected with the minimum concentration for each object area (3A-H). In addition, a computer program, a computer-implemented one, is used Method, a data processing unit (6) and an optical observation system (100) for preparing the observation of a fluorescence intensity of fluorescence radiation of a fluorescent dye in an observation object (3) are provided.

Description

The present invention relates to a method for preparing the observation of a fluorescence intensity of fluorescence radiation of a fluorescent dye in an observation object and a method for observing a fluorescence intensity of fluorescence radiation. The invention also relates to an optical observation system. In addition, a computer program, a computer-implemented method and a data processing unit are provided for preparing the observation of a fluorescence intensity of fluorescence radiation of a fluorescent dye in an observation object.

In modern optical observation systems, which can include optical observation devices such as surgical microscopes and endoscopes, various fluorescence options can be integrated that enable observation of fluorescence. If a fluorescence measurement is to be carried out, this is often based purely on the color impression and the perceived brightness when observed with the eye. However, this color impression and brightness are very dependent on the various parameters during the recording, which does not allow for reliable quantitative measurement. In order to make fluorescence measurements easier to compare, there are recommendations to set the working distance and the illumination brightness to a specific value.

For example, fluorescence images during neuro-surgery depend on a variety of parameters that influence the perceptible fluorescence intensity. It is often necessary to change parameters such as the working distance, the zoom position, etc. from time to time during the neuro-surgery. With every change, the observed or measured fluorescence intensity also changes. This means that a tumor marked with a fluorescent dye sometimes shines brighter and sometimes darker, depending on the current set of parameters. However, since at least in some fluorescence methods the luminosity of the fluorescence is included in the diagnosis, the aim is to achieve comparability between several fluorescence measurements in different surgical situations, by different users and in different clinics. In particular, quantitative methods for fluorescence measurement are also sought.

Quantitative measurements are possible with contact measurements at tissue points, but such point measurements with hand-held contact devices are not practical for visualizing fluorescence in a larger area or even live during resection. Such contact measurements are, for example, in Quantitative fluorescence in intracranial tumor: implications for ALA-induced PpIX as an intraoperative biomarker", Roberts et al, Neurosurg. 2011 July ; 115(1): 11-17. doi:10.3171/2011.2.JNS101451. described

The US 2019/227288 A1 describes a method for normalizing fluorescence intensities. In the method, parameter values of the observation beam path of an optical observation device, in particular the setting values of the magnification and the working distance, are recorded. Based on the recorded parameter values and the influence of the corresponding parameters on the fluorescence intensity, an exposure parameter for image recording is set so that the influence of a changed magnification or a changed working distance on the recorded fluorescence intensities is compensated.

In US 2016/278678 A1 Fluorescence images are recorded, which are corrected using a 3D surface model. In particular, fluorescence images are recorded, with which quantification of surface and near-surface dyes is possible. Image deformations, which are based on the settings of the image recording and the surface orientation of the object under observation, are determined and taken into account using a suitable image distortion.

For example, with the in US 2019/227288 A1 and US 2016/278678 A1 Quantitative fluorescence measurements are possible using the method described, but here too the prerequisite is that the fluorescence intensity is sufficient to be detected. There is therefore the possibility that, despite a suitable method for quantitative fluorescence measurement, the quantitative measurement fails because, for example, after a change in the magnification or the working distance, the fluorescence intensity is no longer sufficient to be detected, for example, by an image sensor used.

It is therefore an object of the invention to provide a method, a computer program and a data processing unit for preparing the observation of a fluorescence intensity of fluorescence radiation and to provide a method for observing fluorescence intensity and an optical observation system with which the probability of failure in a fluorescence measurement can be reduced.

This object is achieved according to the invention by a method for preparing the observation of a fluorescence intensity according to claim 1, by a method for observing a fluorescence intensity according to claim 11, by an optical observation system according to claim 12, by a computer-implemented method according to claim 20, by a computer program according to claim 21 and a data processing unit according to claim 22 solved. The dependent claims contain advantageous embodiments of the invention.

• - Ermitteln des Parameterwertes wenigstens eines die Beobachtung der Fluoreszenzintensität beeinflussenden Parameters. Das Ermitteln des Parameterwertes kann dabei bspw. durch Erfassen des Parameterwertes mittels eines geeigneten Sensors oder durch Abrufen des Parameterwertes aus einer Steuerung erfolgen. Es besteht aber auch die Möglichkeit, den Parameterwert zu ermitteln, indem er anhand eines Modells aus wenigstens einem anderen erfassten oder abgerufenen Parameterwert berechnet wird.
• - Simulieren der für die jeweiligen Objektbereiche zu erwartenden Fluoreszenzintensität auf der Basis des ermittelten Parameterwertes des wenigstens einen Parameters und eines Modells für den Einfluss des wenigstens einen Parameters auf die Fluoreszenzintensität.

According to the invention, a method for preparing the observation of a fluorescence intensity of fluorescence radiation of a fluorescent dye in an observation object is provided. The observation object, which can also be an area of interest of a larger object, has object areas that differ from one another in their depth and/or orientation. The observation of the fluorescence intensity should be carried out using an optical observation system with which fluorescence radiation can be observed, provided that it has a certain minimum intensity. The minimum intensity can be determined by the sensitivity of an image sensor used in the optical observation system. The image sensor can, for example, be part of an optical observation device used in the optical observation system, such as a camera, a surgical microscope, an endoscope, etc. But even in the case of a purely visual observation of the fluorescence with the optical observation device used, a certain minimum intensity of the fluorescence radiation must be achieved so that the eye can perceive it. The procedure includes the steps:

• - Determining the parameter value of at least one parameter influencing the observation of the fluorescence intensity. The parameter value can be determined, for example, by detecting the parameter value using a suitable sensor or by retrieving the parameter value from a controller. However, it is also possible to determine the parameter value by calculating it using a model from at least one other recorded or retrieved parameter value.
• - Simulating the fluorescence intensity expected for the respective object areas on the basis of the determined parameter value of the at least one parameter and a model for the influence of the at least one parameter on the fluorescence intensity.

The method according to the invention is characterized in that a minimum concentration of the fluorescent dye is specified as part of the simulation and the fluorescence intensity to be expected with the minimum concentration is determined for each object area on the basis of the simulation.

Based on the simulation of the fluorescence intensity to be expected with the determined parameter values for the minimum concentration of the fluorescent dye, it is possible to determine in advance which areas of the observation object make it possible to detect the minimum concentration of the fluorescent dye with the currently set parameters. This can be used to check whether the expected fluorescence intensity determined is sufficient to be detected by the optical observation system with the given sensitivity. By checking, it is possible to prevent fluorescence marking from being carried out which does not allow the minimum concentration of the fluorescent dye to be detected in the areas of interest of the observation object. Loss of time due to inappropriate fluorescence measurements during an operation can thus be avoided. In order to prevent an ineffective fluorescence measurement, an optical and/or acoustic warning can be issued, for example, if a check shows that the expected fluorescence intensity determined for the minimum concentration of the fluorescent dye is not sufficient in every object area to match the given sensitivity of the optical Observation system can be detected by this.

Advantageously, a graphical representation can be generated which shows in which object areas the expected fluorescence intensity determined for the minimum concentration of the fluorescent dye is sufficient to be able to be detected by the optical observation system with the given sensitivity. This enables the user to decide whether the expected fluorescence intensity determined for the minimum concentration of the fluorescent dye in the object areas relevant to him is sufficient to be able to carry out the fluorescence observation in a targeted manner.

In addition, the method also offers the possibility of using the simulation to determine an improved parameter value, in particular an optimized parameter value, for the at least one parameter in such a way that the expected fluorescence intensity simulated with the optimized parameter value for the minimum concentration of the fluorescent dye is in as many object areas as possible, in particular If possible in all object areas, is sufficient to be detected by the optical observation system with the given sensitivity. The improved or optimized parameter value for the at least one parameter can then, for example, be displayed for manual setting or output to an automatic setting unit that. current parameter value of the at least one parameter is automatically set to the improved or optimized parameter value. In this way, the proportion of non-effective fluorescence measurements can be reduced to those fluorescence measurements in which an improvement of the parameter value of the at least one parameter value is not possible.

As part of the simulation, in particular information about the depth distribution of the object areas and/or information about the orientation of the object areas can be used. The information mentioned can be present, for example, in the form of a 3D model of the observation object, in the form of a depth map of the observation object, etc. This information makes it possible, within the framework of the simulation, to take into account the dependence of the fluorescence intensity on the distance of the lighting system and/or the optical observation system from the respective object areas and/or the orientation of the lighting system and/or the optical observation system with respect to the respective object areas. In particular, it is also possible to take shading effects into account.

• - Abstand eines optischen Beobachtungsgeräts des optischen Bobachtungssystems von den Objektbereichen. Dieser Abstand kann bspw. anhand der Information über die Tiefenverteilung der Objektbereiche und der Information über die Orientierung der Objektbereiche sowie der Position und Orientierung des optischen Beobachtungsgeräts in Bezug auf das Beobachtungsobjekt ermittelt werden. Zum Ermitteln der Position und Orientierung des optischen Beobachtungsgeräts in Bezug auf das Beobachtungsobjekt kann bspw. ein Navigationssystem, Stereografie, etc. zum Einsatz kommen.
• - Orientierung des optischen Beobachtungsgeräts in Bezug auf die Objektbereiche. Die Orientierung kann wie der Abstand bspw. anhand der Information über die Tiefenverteilung der Objektbereiche und der Information über die Orientierung der Objektbereiche sowie der Position und Orientierung des optischen Beobachtungsgeräts in Bezug auf das Beobachtungsobjekt ermittelt werden.
• - Zoomeinstellung des optischen Beobachtungsgeräts.
• - Objektschnittweite des optischen Beobachtungsgeräts.
• - Blendeneinstellung des optischen Beobachtungsgeräts.
• - Gain eines im optischen Beobachtungsgerät verwendeten Bildsensors.
• - Belichtungsdauer eines im optischen Beobachtungssystem verwendeten Bildsensors.
• - Nichtlinearitäten eines im optischen Beobachtungssystem verwendeten Bildsensors.
• - Abstand eines Beleuchtungssystems von den Objektbereichen. Dieser Abstand kann bspw. anhand der Information über die Tiefenverteilung der Objektbereiche und der Information über die Orientierung der Objektbereiche sowie der Position und Orientierung des Beleuchtungssystems in Bezug auf das Beobachtungsobjekt ermittelt werden. Zum Ermitteln der Position und Orientierung des Beleuchtungssystems in Bezug auf das Beobachtungsobjekt kann bspw. ein Navigationssystem, Stereografie, etc. zum Einsatz kommen.
  • - Orientierung eines Beleuchtungssystems in Bezug auf die Objektbereiche. Die Orientierung kann wie der Abstand bspw. anhand der Information über die Tiefenverteilung der Objektbereiche und der Information über die Orientierung der Objektbereiche sowie der Position und Orientierung des Beleuchtungssystems in Bezug auf das Beobachtungsobjekt ermittelt werden.
• - Intensität einer Beleuchtungslichtquelle.
• - spektrale Intensitätsverteilung einer Beleuchtungslichtquelle.
• - Zoomstellung eines Beleuchtungs-Zooms.
• - Stellung einer Beleuchtungsblende.

At least the parameter value of one of the following parameters can be determined as the at least one parameter value and taken into account in the simulation:

• - Distance of an optical observation device of the optical observation system from the object areas. This distance can be determined, for example, based on the information about the depth distribution of the object areas and the information about the orientation of the object areas as well as the position and orientation of the optical observation device in relation to the observation object. For example, a navigation system, stereography, etc. can be used to determine the position and orientation of the optical observation device in relation to the observation object.
• - Orientation of the optical observation device in relation to the object areas. The orientation, like the distance, can be determined, for example, based on the information about the depth distribution of the object areas and the information about the orientation of the object areas as well as the position and orientation of the optical observation device in relation to the observation object.
• - Zoom adjustment of the optical observation device.
• - Object focal length of the optical observation device.
• - Aperture adjustment of the optical observation device.
• - Gain of an image sensor used in the optical observation device.
• - Exposure duration of an image sensor used in the optical observation system.
• - Nonlinearities of an image sensor used in the optical observation system.
• - Distance of a lighting system from the object areas. This distance can be determined, for example, based on the information about the depth distribution of the object areas and the information about the orientation of the object areas as well as the position and orientation of the lighting system in relation to the object being observed. For example, a navigation system, stereography, etc. can be used to determine the position and orientation of the lighting system in relation to the object under observation.
  • - Orientation of a lighting system in relation to the object areas. The orientation, like the distance, can be determined, for example, based on the information about the depth distribution of the object areas and the information about the orientation of the object areas as well as the position and orientation of the lighting system in relation to the object being observed.
• - Intensity of an illuminating light source.
• - spectral intensity distribution of an illuminating light source.
• - Zoom position of a lighting zoom.
• - Position of a lighting panel.

All of the parameters mentioned influence the fluorescence intensity, which can be detected when the minimum concentration of fluorescent dye is present, for example by the eye or an image sensor of the optical observation device. The more these parameters are taken into account in the simulation, the more accurate the simulation is. In practice, the available computer capacity and/or the equipment of the observation system can limit the number of usable parameters to the parameters necessary or most important for the intended fluorescence observation.

A light source for fluorescence excitation, typically a xenon lamp or an LED, is subject to aging, which causes the spectral intensity distribution of the illuminating light source to change over time. It is therefore advantageous if the spectral intensity distribution of the illumination light source is determined based on the value of a lifespan counter of the illumination source and its nominally set intensity using a degradation model for the illumination source.

In an advantageous development of the method according to the invention for preparing the observation of a fluorescence intensity, at least one reference measurement is carried out with a reference concentration of the fluorescent dye using a reference parameter value for the at least one parameter influencing the observation of the fluorescence intensity in order to obtain a reference value for the fluorescence intensity at the reference concentration of the fluorescent dye .

A simulation of the fluorescence intensity to be expected for the respective object areas is then carried out, in which a change in the fluorescence intensity in comparison to the reference intensity is determined for a deviation of the parameter value of the at least one parameter influencing the observation of the fluorescence intensity from the reference parameter value.

Finally, a compensation factor is determined with which the change in the fluorescence intensity caused by the deviation of the parameter value of the at least one parameter influencing the observation of the fluorescence intensity from the reference parameter value can be compensated for in a digital image.

The reference measurement and the compensation factor make it possible to display surface areas with the same concentration of fluorescent dye regardless of the distance and orientation of a surface area in relation to the illumination system of the optical observation system and in relation to the optical observation device of the optical observation system in a digital image with the same intensity of fluorescence radiation .

Advantageously, the at least one reference measurement is carried out with reference parameter values for each parameter influencing the observation of the fluorescence intensity, which is later relevant when observing a fluorescence intensity. In this way, the reference measurement provides more accurate results. The reference measurement can be carried out, for example, on a reference observation object with a known surface geometry, in particular with a flat surface, with the optical observation device of the optical observation system and the illumination system of the optical observation system each being in a reference position and a reference orientation with respect to the reference observation object. As part of the development described, it is also possible to carry out several reference measurements with several reference concentrations that differ from one another in order to obtain reference values for the fluorescence intensities assigned to the different reference concentrations. This means that the reference intensity from which the recorded fluorescence intensity has the smallest deviation can be used to calculate the compensation factors. As a result, the accuracy of the calculated compensation factors can be increased.

According to a second aspect of the invention, a method for observing a fluorescence intensity of fluorescence radiation of a fluorescent dye in an observation object is provided. The observation object, which can also be an area of interest of a larger object, has object areas that differ from one another in their depth and/or orientation. The observation of the fluorescence intensity is carried out using an optical observation system with which fluorescence radiation can be observed, provided that it has a certain minimum intensity. The Min The minimum intensity can be determined by the sensitivity of an image sensor used in the optical observation system. The image sensor can be part of an optical observation device used in the optical observation system, such as a camera, a surgical microscope, an endoscope, etc. But even in the case of purely visual observation of the fluorescence, a certain minimum intensity of the fluorescence radiation must be achieved so that the eye can perceive it. As part of the method for observing a fluorescence intensity, the method according to the invention is used to prepare the observation of a fluorescence intensity of fluorescence radiation of a fluorescent dye in an observation object or one of its advantageous developments.

With the help of the method for preparing the observation of a fluorescence intensity of fluorescence radiation of a fluorescent dye, it is possible to determine in advance which areas of the observation object make it possible to detect the minimum concentration of the fluorescent dye with the currently set parameters. In this way, before observing the fluorescence intensity, it can be assessed whether the measurement makes sense. If the measurement does not make sense, the parameter value of the at least one parameter influencing the observation of the fluorescence intensity can be improved or optimized with a view to achieving a fluorescence intensity detectable with the optical observation system in as many surface areas as possible with the minimum concentration of fluorescent dye. The improved or optimized parameter value of the at least one parameter influencing the observation of the fluorescence intensity can, for example, contain at least one of the following changes compared to the original parameter value: a changed intensity of the illumination radiation, a changed position of the optical observation device used in the optical observation system and/or of the optical observation device Observation system used lighting system in relation to the observation object, a changed orientation of the optical observation device used in the optical observation system and / or the lighting system used in the optical observation system in relation to the observation object, a changed zoom setting of the optical observation device used in the optical observation system and / or in optical observation system used, lighting system, etc.

According to a third aspect of the invention, an optical observation system is provided which enables the observation of a fluorescence intensity of fluorescence radiation from a fluorescent dye in an observation object which comprises object regions that differ from one another in their depth and/or orientation, provided that this has a certain minimum intensity . The minimum intensity can be determined by the sensitivity of an image sensor used in the optical observation system. But even in the case of purely visual observation of the fluorescence, a certain minimum intensity of the fluorescence radiation must be achieved so that the eye can perceive it. The optical observation system can include, for example, a camera, a surgical microscope, an endoscope, etc. as an observation device.

The optical observation system comprises a determination device for determining the parameter value of at least one parameter influencing the observation of the fluorescence intensity and a simulation device for simulating the fluorescence intensity to be expected for the respective object areas on the basis of the determined parameter value of the at least one parameter and a model for the influence of the at least one Parameters on the fluorescence intensity. The parameter value can be determined, for example, by detecting the parameter value using a suitable sensor or by retrieving the parameter value from a controller. However, it is also possible to determine the parameter value by calculating it using a model from at least one other recorded or retrieved parameter value. In addition, the optical observation system includes an evaluation device which is designed to determine the fluorescence intensity to be expected with the minimum concentration for each object area for a minimum concentration of the fluorescent dye specified in the simulation based on the simulation.

With the optical observation system, the method for observing a fluorescence intensity of fluorescence radiation of a fluorescent dye, including the method for preparing the observation of a fluorescence intensity of fluorescence radiation of a fluorescent dye, so that the advantages described with respect to these two methods can be realized with the optical observation system.

In advantageous developments of the optical observation system, this is designed to carry out the advantageous developments of the method according to the invention for preparing the observation of a fluorescence intensity of fluorescence radiation of a fluorescent dye, so that the with Reference to the advantageous developments of the method according to the invention for preparing the observation of a fluorescence intensity of fluorescence radiation of a fluorescent dye can be realized with the optical observation system. The advantages achieved with the advantageous developments of the optical observation system will therefore not be described again. Instead, reference is made to the description of the advantages of the developments of the method according to the invention for preparing the observation of a fluorescence intensity of fluorescence radiation from a fluorescent dye.

In a first of the advantageous developments, the evaluation device is designed to carry out a check for each object area as to whether the expected fluorescence intensity determined is sufficient to be able to be detected by the optical observation system with the given sensitivity.

In a second of the advantageous developments, the simulation device is designed to use information about the depth distribution of the object areas and/or information about the orientation of the object areas as part of the simulation.

• - Abstand eines optischen Beobachtungsgeräts des optischen Beobachtungssystems von den Objektbereichen. Dieser Abstand kann bspw. anhand der Information über die Tiefenverteilung der Objektbereiche und der Information über die Orientierung der Objektbereiche sowie der Position und Orientierung des optischen Beobachtungsgeräts in Bezug auf das Beobachtungsobjekt ermittelt werden. Zum Ermitteln der Position und Orientierung des optischen Beobachtungsgeräts in Bezug auf das Beobachtungsobjekt kann bspw. ein Navigationssystem, Stereografie, etc. zum Einsatz kommen.
• - Orientierung des optischen Beobachtungsgeräts in Bezug auf die Objektbereiche. Die Orientierung kann wie der Abstand bspw. anhand der Information über die Tiefenverteilung der Objektbereiche und der Information über die Orientierung der Objektbereiche sowie der Position und Orientierung des optischen Beobachtungsgeräts in Bezug auf das Beobachtungsobjekt ermittelt werden.
• - Zoomeinstellung des optischen Beobachtungsgeräts.
• - Objektschnittweite des optischen Beobachtungsgeräts.
• - Blendeneinstellung des optischen Beobachtungsgeräts.
• - Gain eines im optischen Beobachtungsgerät verwendeten Bildsensors.
• - Belichtungsdauer eines im optischen Beobachtungssystem verwendeten Bildsensors.
• - Nichtlinearitäten eines im optischen Beobachtungssystem verwendeten Bildsensors.
• - Abstand eines Beleuchtungssystems von den Objektbereichen. Dieser Abstand kann bspw. anhand der Information über die Tiefenverteilung der Objektbereiche und der Information über die Orientierung der Objektbereiche sowie der Position und Orientierung des Beleuchtungssystems in Bezug auf das Beobachtungsobjekt ermittelt werden. Zum Ermitteln der Position und Orientierung des Beleuchtungssystems in Bezug auf das Beobachtungsobjekt kann bspw. ein Navigationssystem, Stereografie, etc. zum Einsatz kommen.
• - Orientierung eines Beleuchtungssystems in Bezug auf die Objektbereiche. Die Orientierung kann wie der Abstand bspw. anhand der Information über die Tiefenverteilung der Objektbereiche und der Information über die Orientierung der Objektbereiche sowie der Position und Orientierung des Beleuchtungssystems in Bezug auf das Beobachtungsobjekt ermittelt werden.
• - Intensität einer Beleuchtungslichtquelle.
• - spektrale Intensitätsverteilung einer Beleuchtungslichtquelle.
• - Zoomstellung eines Beleuchtungs-Zooms.
• - Stellung einer Beleuchtungsblende.

In a third of the advantageous developments, the determination device is designed to determine at least the parameter value of one of the following parameters, and the simulation device is designed to take this at least one parameter value into account in the simulation:

• - Distance of an optical observation device of the optical observation system from the object areas. This distance can be determined, for example, based on the information about the depth distribution of the object areas and the information about the orientation of the object areas as well as the position and orientation of the optical observation device in relation to the observation object. For example, a navigation system, stereography, etc. can be used to determine the position and orientation of the optical observation device in relation to the observation object.
• - Orientation of the optical observation device in relation to the object areas. The orientation, like the distance, can be determined, for example, based on the information about the depth distribution of the object areas and the information about the orientation of the object areas as well as the position and orientation of the optical observation device in relation to the observation object.
• - Zoom adjustment of the optical observation device.
• - Object focal length of the optical observation device.
• - Aperture adjustment of the optical observation device.
• - Gain of an image sensor used in the optical observation device.
• - Exposure duration of an image sensor used in the optical observation system.
• - Nonlinearities of an image sensor used in the optical observation system.
• - Distance of a lighting system from the object areas. This distance can be determined, for example, based on the information about the depth distribution of the object areas and the information about the orientation of the object areas as well as the position and orientation of the lighting system in relation to the object being observed. For example, a navigation system, stereography, etc. can be used to determine the position and orientation of the lighting system in relation to the object under observation.
• - Orientation of a lighting system in relation to the object areas. The orientation, like the distance, can be determined, for example, based on the information about the depth distribution of the object areas and the information about the orientation of the object areas as well as the position and orientation of the lighting system in relation to the object being observed.
• - Intensity of an illuminating light source.
• - spectral intensity distribution of an illuminating light source.
• - Zoom position of a lighting zoom.
• - Position of a lighting panel.

In a fourth of the advantageous developments, the evaluation device is designed to generate a graphical representation that shows in which object areas the determined one is expected tending fluorescence intensity is sufficient to be detected by the optical observation system with the given sensitivity.

In a fifth of the advantageous developments, the optical observation system comprises an optimization unit which is designed to use the simulation to determine an improved parameter value, in particular an optimized parameter value, for the at least one parameter in such a way that the expected one simulated with the improved or optimized parameter value Fluorescence intensity in as many object areas as possible (if possible in all object areas) is sufficient to be detected by the optical observation system with the given sensitivity.

In a sixth of the advantageous developments, the optical observation system comprises a control unit for controlling the optical observation system, which is connected to the optimization unit to receive the improved or optimized parameter value and is designed to set the at least one parameter to the improved or optimized parameter value.

In a seventh of the advantageous developments, the optical observation system comprises a compensation factor determination unit, which is designed to determine a compensation factor with which a deviation of the parameter value of the at least one parameter influencing the observation of the fluorescence intensity is determined in a digital image recorded with an image sensor The change in fluorescence intensity caused by the reference parameter value can be compensated for. The compensation factor determination unit is further designed to determine the compensation factor on the basis of at least one reference value for the fluorescence intensity determined for a reference concentration of the fluorescent dye and for a reference parameter value and a simulation of the fluorescence intensity to be expected for the respective object areas, wherein in the simulation for a deviation of the Parameter value of the at least one parameter influencing the observation of the fluorescence intensity is determined from the reference parameter value, a change in the fluorescence intensity compared to the reference intensity.

• - Empfangen oder Abrufen des Parameterwertes wenigstens eines die Beobachtung der Fluoreszenzintensität beeinflussenden Parameters, und
• - Simulieren der für die jeweiligen Objektbereiche zu erwartenden Fluoreszenzintensität auf der Basis des ermittelten Parameterwertes des wenigstens einen Parameters und eines Modells für den Einfluss des wenigstens einen Parameters auf die Fluoreszenzintensität.

Furthermore, according to the invention, a computer-implemented method for preparing the observation of a fluorescence intensity of fluorescence radiation of a fluorescent dye in an observation object which comprises object areas that differ from one another in their depth and/or orientation, the observation being carried out with an optical observation system with which Fluorescent radiation can be observed, provided it has a certain minimum intensity. The procedure includes the steps:

• - Receiving or retrieving the parameter value of at least one parameter influencing the observation of the fluorescence intensity, and
• - Simulating the fluorescence intensity expected for the respective object areas on the basis of the determined parameter value of the at least one parameter and a model for the influence of the at least one parameter on the fluorescence intensity.

As part of the simulation, a minimum concentration of the fluorescent dye is specified and the fluorescence intensity to be expected with the minimum concentration is determined for each object area using the simulation.

• - den Parameterwert wenigstens eines die Beobachtung der Fluoreszenzintensität beeinflussenden Parameters zu empfangen oder abzurufen, und
• - die für die jeweiligen Objektbereiche zu erwartenden Fluoreszenzintensität auf der Basis des empfangenen oder abgerufenen Parameterwertes des wenigstens einen Parameters und eines Modells für den Einfluss des wenigstens einen Parameters auf die Fluoreszenzintensität zu simulieren.

In addition, according to the invention, a computer program for preparing the observation of a fluorescence intensity of fluorescence radiation of a fluorescent dye in an observation object which comprises object areas that differ from one another in their depth and / or orientation, the observation being carried out with an optical observation system, with which fluorescent radiation can be observed, provided it has a certain minimum intensity. The computer program includes instructions that, when executed on a computer, cause it to:

• - to receive or retrieve the parameter value of at least one parameter influencing the observation of the fluorescence intensity, and
• - to simulate the fluorescence intensity to be expected for the respective object areas on the basis of the received or retrieved parameter value of the at least one parameter and a model for the influence of the at least one parameter on the fluorescence intensity.

As part of the simulation, a minimum concentration of the fluorescent dye is specified and the fluorescence intensity to be expected with the minimum concentration is determined for each object area using the simulation.

• - den Parameterwert wenigstens eines die Beobachtung der Fluoreszenzintensität beeinflussenden Parameters zu empfangen oder abzurufen, und
• - die für die jeweiligen Objektbereiche zu erwartenden Fluoreszenzintensität auf der Basis des empfangenen oder abgerufenen Parameterwertes des wenigstens einen Parameters und eines Modells für den Einfluss des wenigstens einen Parameters auf die Fluoreszenzintensität zu simulieren.

Finally, according to the invention, a data processing unit for preparing the observation of a fluorescence intensity of fluorescence radiation of a fluorescent dye in an observation object which comprises object areas that differ from one another in their depth and/or orientation, the observation being to be carried out with an optical observation system with which fluorescence radiation can be observed, provided it has a certain minimum intensity. The data processing unit comprises a memory and a processor, the processor being designed to do so using a computer program stored in the memory.

• - to receive or retrieve the parameter value of at least one parameter influencing the observation of the fluorescence intensity, and
• - to simulate the fluorescence intensity to be expected for the respective object areas on the basis of the received or retrieved parameter value of the at least one parameter and a model for the influence of the at least one parameter on the fluorescence intensity.

As part of the simulation, a minimum concentration of the fluorescent dye is specified and the fluorescence intensity to be expected with the minimum concentration is determined for each object area using the simulation.

The advantages of the computer-implemented method according to the invention, the computer program according to the invention and the data processing unit according to the invention are apparent from the method according to the invention and are therefore not described further at this point. The computer-implemented method according to the invention, the computer program according to the invention and the data processing unit according to the invention can also be developed in such a way that they enable the further developments of the method according to the invention to be carried out.

• 1 ein Ausführungsbeispiel für ein optisches Beobachtungssystem zur Beobachtung einer Fluoreszenzintensität von Fluoreszenzstrahlung.
• 2 zeigt ein die optischen Komponenten eines Operationsmikroskops, wie es als optisches Beobachtungsgerät im optischen Beobachtungssystem Verwendung finden kann, in einer schematischen Darstellung.
• 3 zeigt schematisch den grundsätzlichen Aufbau eines Varioskopobjektivs.
• 4 zeigt die optischen Komponenten eines digitalen Operationsmikroskops, wie es als optisches Beobachtungsgerät im optischen Beobachtungssystem Verwendung finden kann, in einer schematischen Darstellung.
• 5 zeigt ein exemplarisches Ausführungsbeispiel für ein Verfahren zur Vorbereitung der Beobachtung einer Fluoreszenzintensität.
• 6 zeigt ein Beispiel für ein Beobachtungsobjekt.
• 7 zeigt das Beobachtungsobjekt aus 6 mit einer Markierung, die anzeigt, wo die zu erwartende Fluoreszenzintensität nicht ausreicht, um mit der gegebenen Empfindlichkeit des optischen Beobachtungsgeräts von diesem detektiert werden zu können.

Further features, properties and advantages of the present invention result from the following description of exemplary embodiments with reference to the accompanying figures.

• 1 an exemplary embodiment of an optical observation system for observing a fluorescence intensity of fluorescent radiation.
• 2 shows the optical components of a surgical microscope, as it can be used as an optical observation device in the optical observation system, in a schematic representation.
• 3 shows schematically the basic structure of a varioscope lens.
• 4 shows the optical components of a digital surgical microscope, as it can be used as an optical observation device in the optical observation system, in a schematic representation.
• 5 shows an exemplary embodiment of a method for preparing the observation of a fluorescence intensity.
• 6 shows an example of an observation object.
• 7 shows the observation object 6 with a marking that indicates where the expected fluorescence intensity is not sufficient to be detected by the optical observation device with the given sensitivity.

1 shows an exemplary embodiment of an optical observation system 100 according to the invention in a highly schematic representation. In the present exemplary embodiment, the optical observation system 100 comprises a surgical microscope 2 suspended on a tripod 1, typically a robotic tripod, which serves as an optical observation device with which an observation object 3 is observed. The observation object 3 has an irregular structure with object areas 3A-H that have different orientations and are located at different depths. As a result, the object areas 3A-H have different distances and different orientations with respect to the surgical microscope 2. It should be noted at this point that in 1 For the sake of clarity, the object areas are only shown in part of the observation object 3.

The optical observation system 100 includes a controller 4 and a data processing unit 6, which processes digital image data recorded with an image sensor present in the surgical microscope 2 for display on a monitor 8, which can in particular be a 3D monitor, and outputs it to the monitor 8. Instead of the monitor 8 or in addition to the monitor, further display units, for example an HMD (Head Mounted Display) or a digital view of the surgical microscope 2, can be present, to which the data processing unit 6 sends the digital image data prepared for display. In the present exemplary embodiment, the data processing unit 6 can be set up by loading a computer program to carry out a method for preparing the observation of a fluorescence intensity of fluorescent radiation of a fluorescent dye in the observation object 3. Alternatively, the data processing unit 6 can contain an application-specific integrated circuit (ASIC) in which the steps for executing the method are stored.

The optical observation system 100 also includes an illumination system 40 with which the observation object 3 can be illuminated with illumination light. In the present exemplary embodiment, the lighting system 40 is able to illuminate the observation object 3 with a special wavelength, hereinafter referred to as the excitation wavelength, which stimulates a fluorescent dye introduced into the observation object 3 to emit fluorescent radiation. The surgical microscope 2 is set up to detect the fluorescent radiation emitted by the fluorescent dye. For this purpose, the surgical microscope 2 can in particular have a digital image sensor with sufficient sensitivity for the wavelength of the fluorescence of the fluorescent dye. In addition, in the present exemplary embodiment, there is at least one filter that can be inserted into the observation beam path, with which light reflected from the observation object 3 with the excitation wavelength can be removed from the observation beam path. The excitation usually occurs with an intensity that would result in the fluorescent radiation being outshone by the light with the excitation wavelength. This overradiation can be prevented with at least one filter that can be inserted into the observation beam path.

Of course, the fluorescence does not necessarily have to be observed via a display such as the monitor 8, but it can also be observed purely optically through eyepieces of the surgical microscope. Regardless of whether the fluorescence is observed using eyepieces or with the help of image sensors, the fluorescence intensity must exceed a certain minimum level in order to be detected. With an image sensor, the detection threshold in the corresponding wavelength range must be exceeded by the fluorescence intensity. But even with visual observation, a certain minimum intensity of the fluorescence radiation is necessary so that the eye can perceive the fluorescence radiation as a detector. However, the main area of application of the present invention is the observation of fluorescence using digital image sensors such as CCD sensors or CMOS sensors.

The following is with reference to the 2 to 4 the optical components of a surgical microscope 2, which can be used as an optical observation device in the optical observation system 100, are explained. However, the invention can also be implemented with other optical observation devices such as cameras or endoscopes.

This in 2 The surgical microscope 2 shown includes, as essential optical components, an objective 5 to be turned toward an observation object 3, which can in particular be designed as an achromatic or apochromatic objective. In the present exemplary embodiment, the objective 5 consists of two partial lenses cemented together, which form an achromatic objective. The observation object 3 is arranged in the focal plane of the objective 5 so that it is imaged to infinity by the objective 5. In other words, a divergent beam of rays 7 emanating from the observation object 3 is converted into a parallel beam of rays 9 as it passes through the objective 5. The image of the object under observation also has a sharpness in a depth range around the focal plane that is sufficient to be perceived as sharp by the image sensor or the eye. This depth range is therefore called the depth of field and is in 1 marked with the reference symbol “TS”.

A magnification changer 11 is arranged on the observer side of the lens 5 and can either be used as a zoom system for continuously changing the magnification, as in the exemplary embodiment shown Magnification factor or as a so-called Galilei changer can be designed to gradually change the magnification factor. In a zoom system that is made up of a lens combination with three lenses, for example, the two lenses on the object side can be moved in order to vary the magnification factor. In fact, the zoom system can also have more than three lenses, for example four or more lenses, whereby the outer lenses can then also be arranged in a fixed manner. In a Galilean changer, on the other hand, there are several fixed lens combinations that represent different magnification factors and can be inserted alternately into the beam path. Both a zoom system and a Galilean changer convert an object-side parallel beam of rays into an observer-side parallel beam of rays with a different beam diameter. In the present exemplary embodiment, the magnification changer 11 is already part of the binocular beam path of the surgical microscope 2, that is, it has its own lens combination for each stereoscopic partial beam path 9A, 9B of the surgical microscope 2. In the present exemplary embodiment, a magnification factor is set by means of the magnification changer 11 via a motor-driven actuator (not shown), which, together with the magnification changer 11, is part of a magnification changing unit for setting the magnification factor.

In the present exemplary embodiment, the magnification changer 11 is connected on the observer side by an interface arrangement 13A, 13B, via which external devices can be connected to the surgical microscope 1 and which, in the present exemplary embodiment, comprises beam splitter prisms 15A, 15B. In principle, other types of beam splitters can also be used, for example partially transparent mirrors. In the present exemplary embodiment, the interfaces 13A, 13B serve to couple out a beam of rays from the beam path of the surgical microscope 2 (beam splitter prism 15B) or to couple a beam of rays into the beam path of the surgical microscope 2 (beam splitter prism 15A). However, they can also both be designed to couple out a beam of rays from the beam path of the surgical microscope 2 or both to couple a beam of rays into the beam path of the surgical microscope 2.

In the present exemplary embodiment, the beam splitter prism 15A in the partial beam path 9A serves to provide information or data to a viewer using a display 37, for example a digital mirror device (DMD) or an LCD display, and associated optics 39 via the beam splitter prism 15A into the partial beam path 9A of the surgical microscope 1. In the other partial beam path 9B, a camera adapter 19 with a camera 21 attached to it is arranged at the interface 13B, which is equipped with an electronic image sensor 23, for example with a CCD sensor or a CMOS sensor. Using the camera 21, an electronic and in particular a digital image of the observation object 3 can be recorded. In particular, a hyper-spectral sensor can also be used as an image sensor, in which not only three spectral channels (e.g. red, green and blue) are present, but a large number of spectral channels. If both interfaces 13A, 13B are designed to decouple a beam of rays from the beam path of the surgical microscope 2, a camera adapter 19 with a camera 21 attached to it can be arranged at both interfaces 13A, 13B. Stereoscopic images can be captured in this way.

In the surgical microscope, at least one further interface arrangement with at least two beam splitters can also be present, for example one interface arrangement can be used to couple in stereoscopic partial images and the other interface arrangement can be used to couple out stereoscopic partial images.

A binocular tube 27 is connected to the interface 13 on the observer side. This has two tube lenses 29A, 29B, which focus the respective parallel beam of rays 9A, 9B onto an intermediate image plane 31, i.e. image the observation object 3 onto the respective intermediate image plane 31A, 31B. The intermediate images located in the intermediate image planes 31A, 31B are finally imaged to infinity by eyepiece lenses 35A, 35B, so that a viewer can view the intermediate image with a relaxed eye. In addition, the distance between the two partial beams 9A, 9B is increased in the binocular tube by means of a mirror system or prisms 33A, 33B in order to adapt this to the distance between the eyes of the observer. The mirror system or prisms 33A, 33B also erect the image.

The surgical microscope 2 is also equipped with an illumination system 40 with which the observation object 3 can be illuminated with illumination light. For this purpose, the lighting system 40 in the present example has a white light source 41, such as a halogen incandescent lamp or a gas discharge lamp such as a xenon lamp. But LEDs are also used as light sources Consideration. The light emanating from the white light source 41 is directed towards the observation object 3 via a deflection mirror 43 or a deflection prism in order to illuminate it. In the lighting system 40 there is also lighting optics 45, which ensures uniform illumination of the entire observed observation object 3. The illumination optics 45 can also include a zoom system (illumination zoom) with which the size of the illumination light spot can be changed, and/or a system with which the focal distance of the illumination optics 45 can be varied. In addition, the lighting system 40 can be equipped with a light source for emitting light of a wavelength that stimulates fluorescence in a fluorescent dye introduced into the observation object 3. Alternatively, as in 2 shown, a spectral filter 47 may be present, which can be introduced into the illumination beam path and which essentially only allows the light from the white light source 41 to pass through the wavelength that stimulates the fluorescence in the fluorescent dye. Filters 38A, 380B are also introduced into the observation beam path, which block the wavelength that stimulates fluorescence in the fluorescent dye.

The lighting system 40 can also include at least one aperture and/or at least one lens, with which the profile of the illuminating light cone emanating from the light source can be influenced, for example in order to generate a center-weighted lighting profile. The at least one aperture and/or the at least one lens 49 can be introduced into the illumination beam path if necessary. Furthermore, the lighting system 40 can include apertures with which a sharp delimitation of the luminous field in the observation object 3 can be brought about.

It should be noted that the in 2 The illumination beam path shown is highly schematized and does not necessarily reflect the actual course of the illumination beam path. In principle, the illumination beam path can be designed as so-called oblique illumination, which corresponds to the schematic representation in 2 comes closest. In such oblique lighting, the beam path runs at a relatively large angle (6° or more) to the optical axis of the main lens 5 and can, as in 2 shown, run completely outside the main lens 5. Alternatively, however, there is also the possibility of allowing the illumination beam path of the oblique illumination to run through an edge region of the main lens 5. Another possibility for arranging the illumination beam path is the so-called 0° illumination, in which the illumination beam path runs through the main lens 5 and between the two partial beam paths 9A, 9B, along the optical axis of the main lens 5 in the direction of the observation object 3 into the main lens 5 is coupled in. Finally, there is also the possibility of designing the illumination beam path as so-called coaxial illumination, in which a first and a second partial illumination beam path are present. The illumination sub-beam paths are coupled into the surgical microscope 2 via one or more beam splitters parallel to the optical axes of the observation sub-beam paths 9A, 9B, so that the illumination runs coaxially to the two observation sub-beam paths.

In the in 2 In the embodiment variant of the surgical microscope 2 shown, the objective 5 consists only of a fixed focal length achromatic lens. However, an objective lens system consisting of several lenses can also be used, in particular a so-called varioscope lens, with which the working distance of the surgical microscope 2, ie the distance of the object-side focal plane from the apex of the first object-side lens surface of the lens 5, also called the object focal length, can be varied. The observation object 3 arranged in the focal plane is also imaged to infinity by the varioscope lens 50, so that a parallel bundle of rays is present on the observer side.

An example of a varioscope lens is shown schematically in 3 shown. The varioscope lens 50 includes a positive element 51, i.e. an optical element with positive refractive power, which is in 3 is shown schematically as a convex lens. In addition, the varioscope lens 50 includes a negative element 52, i.e. an optical element with negative refractive power, which is in 3 is shown schematically as a concave lens. The negative member 52 is located between the positive member 51 and the observation object 3. In the varioscope lens 50 shown, the negative member 52 is arranged in a fixed manner, whereas the positive member 51 is arranged so as to be displaceable along the optical axis OA, as indicated by the double arrow 53. If the positive element 51 is in the in 3 The position shown in dashed lines is shifted, the focal length increases, so that the working distance of the surgical microscope 2 from the observation object 3 changes.

Although in 3 If the positive member 51 is designed to be displaceable, there is also the possibility in principle of arranging the negative member 52 to be displaceable along the optical axis OA instead of the positive member 51. However, the negative member 52 often forms the final lens of the varioscope lens 50. A solid Standing negative member 52 therefore offers the advantage that the interior of the surgical microscope 2 can be more easily sealed against external influences. It should also be noted that although the positive link 51 and the negative link 52 in 3 are only shown as individual lenses, each of these members can also be realized in the form of a lens group or a cement member instead of in the form of an individual lens, for example in order to make the varioscope lens 50 achromatic or apochromatic.

4 shows an example of a purely digital surgical microscope 48 in a schematic representation. In this surgical microscope 48, the main objective 5, the magnification changer 11 and the illumination system 40 do not differ from that in 2 Surgical microscope 2 shown. The difference is that the in 4 Surgical microscope 48 shown does not include an optical binocular tube. Instead of the tube lenses 29A, 29B 2 includes the in 4 shown surgical microscope 48 focusing lenses 49A, 49B with which the binocular observation beam paths 9A, 9B are imaged onto digital image sensors 61A, 61B. The digital image sensors 61A, 61B can be, for example, CCD sensors or CMOS sensors. The images recorded by the image sensors 61A, 61B are digitally sent to a data processing unit 6, as shown in 1 is shown, which prepares it for display on a monitor 8 or on digital displays 63A, 63B and then sends it to the monitor 8 or the digital displays 63A, 63B. The digital displays 63A, 63B can be designed as LED displays, as LCD displays or as displays based on organic light-emitting diodes (OLEDs). As in the present example, they can be assigned to eyepiece lenses 65A, 65B, with which the images shown on the displays 63A, 63B are imaged to infinity so that a viewer can view them with relaxed eyes. The displays 63A, 63B and the eyepiece lenses 65A, 65B can be part of a digital binocular tube, but they can also be part of a head-mounted display (HMD) such as data glasses. The monitor or displays 63A, 63B can be designed in particular to view stereoscopic images. For this purpose, the displays 63A, 63B can be assigned to different eyes of the user and display stereoscopic partial images.

In the case of the monitor 8, the stereoscopic partial images can be displayed sequentially in time. For example, using synchronized shutter glasses, the stereoscopic partial images can then only be presented to the appropriate eye. An alternative is to display the stereoscopic partial images in differently polarized light and to equip the viewer with glasses that only allow the polarized light of one of the stereoscopic partial images to pass through the right and left eyes.

Although in 4 as in 2 only an achromatic lens 5 with a fixed focal length is shown, this can be done in 4 shown surgical microscope 48 like that in 3 Surgical microscope 2 shown includes a varioscope lens instead of the objective lens 5. Furthermore, in 4 a transmission of the images recorded by the image sensors 61A, 61B to the displays 63A, 63B by means of cables 67A, 67B is shown. However, instead of being wired, the images can also be transmitted wirelessly to the displays 63A, 63B, especially if the displays 63A, 63B are part of a head mounted display.

In the in 1 In the exemplary embodiment shown, the robotic tripod 1, the surgical microscope 2 and the lighting system 40 each have a tracking target 10, with the help of which a tracking system 12 can determine the position and orientation of the respective component. A tracking target 12 is also attached directly or indirectly to the observation object 3, so that the position and orientation of the respective component in relation to the observation object 3 can be determined. For indirect attachment to the observation object 3, the tracking target 12 can be fixed, for example, to a skull clamp.

A first exemplary embodiment of a method for preparing the observation of a fluorescence intensity with the in 1 The optical observation system 100 shown is described below with reference to 1 and 5 , which shows a flowchart of the method, is described.

Observing fluorescence is particularly important during tumor resection, as the treating surgeon uses the fluorescence to assess which tissue is tumor and should therefore be removed. However, this requires that the fluorescence intensity of the tumor tissue is sufficient to be able to detect it in the entire tumor tissue. However, some types of tumors, such as low-grade gliomas (LGG), only absorb very small amounts of contrast agent (e.g. PPIX in the case of low-grade gliomas), so that the fluorescence intensity is very low and therefore difficult to measure. Even in sensitive optical observation devices, for example, the fluorescence intensity of PPIX in low-grade gliomas is often close to the detection threshold, so that optimal Observation conditions must be ensured in order to be able to detect a reliable fluorescence signal with the image sensor 23 or the image sensors 61A, 61B. In the prior art, cases can therefore occur in which tissue areas closer to the surgical microscope 2 (e.g. object areas 3A-C) appear to be fluorescent, since enough excitation light comes to the tissue and enough of the emitted fluorescence reaches the image sensor 23 or the image sensors 61A, 6B is directed. However, deeper tissue areas (e.g. object areas 3D-H) in the same scene may no longer be illuminated sufficiently brightly with the excitation wavelength despite the same concentration of fluorescent dye due to the greater distance of the illumination light source 41 and/or the collection efficiency decreases due to the greater distance of the surgical microscope 2 from the corresponding object areas 3D-H, e.g. overall with a (1/distance)^4 dependency. If the fluorescence intensity of the higher-lying object areas 3A-C is just above the sensitivity threshold of the image sensor 23 or the image sensors 61A, 6B, the same fluorescence intensity in the deeper tissue areas 3D-H may no longer be possible from the image sensor 23 or the image sensors 61A, 6B can be detected. This means that it can happen that in the image obtained with the image sensor 23 or the image sensors 61A, 6B, some object areas 3A-C of the observation object 3 shown in the image, which can be illuminated and observed more efficiently, are shown as fluorescent, while other object areas 3D-H can be shown as non-fluorescent with the same concentration of fluorescent dye. This can lead to major challenges for the treating surgeon, which can prolong the treatment unnecessarily. The fluorescence can also suddenly disappear or appear in the image obtained with the image sensor 23 or the image sensors 61A, 6B after an adjustment of the surgical microscope 2 (eg zoom change, movement of the surgical microscope and / or the lighting system 40 relative to the observation object 3), which is a reliable Diagnosis considerably more difficult.

The data processing unit 6 of the optical observation system 100 1 therefore includes a determination device 14 for determining the parameter value of at least one parameter influencing the observation of the fluorescence intensity. It also includes a simulation device 16 for simulating the fluorescence intensity expected for the respective object areas 3A-H based on the determined parameter values of a model for the influence of the at least one parameter on the fluorescence intensity. An evaluation device 18 of the data processing unit 6 then determines the fluorescence intensity to be expected for each object area 3A-H for a predetermined minimum concentration of the fluorescent dye. It can also check whether the determined fluorescence intensity is sufficient to be able to be detected by the image sensor 23 or the image sensors 61A, 61B with the given sensitivity of the image sensor 23 or the image sensors 61A, 61B. If the evaluation device 18 determines that there are object areas 3A-H for which the minimum concentration of fluorescent dye does not lead to a signal that can be detected by the image sensor 23 or the image sensors 61 A, 61 B, the evaluation device 18 in the present exemplary embodiment outputs a warning signal that informs the user that reliable detection of fluorescence is not guaranteed for the minimum concentration.

In an optional further development, the evaluation device 18 can generate a graphical representation from which it can be seen in which object areas 3A-H the expected fluorescence intensity is not sufficient to be detected by the image sensor 23 or the image sensors 61A, 61B. In the simplest case, such a graphical representation can be a contour line 20, which is incorporated into an image of the observation object 3, as shown in 6 is shown, is faded in and surrounds those object areas in which the fluorescence intensity is not sufficient to be detected by the image sensor 23 or the image sensors 61A, 61B (see 7 ). An alternative graphical representation can be in the form of a map. In such a map, for example, object areas in which the minimum concentration of fluorescent dye on the image sensor 23 or on the image sensors 61A, 61B leads to a detectable fluorescence intensity can be colored differently than those object areas for which the minimum concentration of fluorescent dye on the image sensor 23 or on the Image sensors 61A, 61B do not result in a detectable fluorescence intensity. The map can optionally have a color transition that indicates how far the respective object areas are from reliable detection of the fluorescence intensity by the image sensor 23 or the image sensors 61A, 61B. For example, object areas whose fluorescence intensity is only 10% below the fluorescence intensity required for detection by the image sensor 23 or the image sensors 61A, 61B can be displayed in green, while object areas with fluorescence intensity that is 50% below the required value are colored red. Based on the graphical representation, a user can decide whether the object areas in which the fluorescence intensity is not sufficient to be detected by the image sensor 23 or the image sensors 61A, 61B are relevant for the intended observation purpose, such as a tumor resection. If the observed area of the observation object 3 is so large that it includes object areas for which it is known, for example from preliminary examinations, that they do not contain any tumor tissue, it can be acceptable if the min Minimum concentration of fluorescent dye does not lead to a fluorescence intensity detectable by the image sensor 23 or by the image sensors 61A, 61B.

In addition, there is the possibility that the evaluation device 18, using the simulation device 16, determines an improved parameter value for at least one parameter of the surgical microscope 2 and/or the lighting system 40 and/or the image sensor 23 or the image sensors 61A, 61B, which leads to this that the number of object areas 3A-H, in which the fluorescence intensity of the minimum concentration of fluorescent dye on the image sensor 23 or on the image sensors 61A, 61B leads to a detectable signal, becomes larger. The at least one improved parameter value can then either be set automatically or presented to the user as a setting recommendation, for example on the monitor 8.

The determination device 14, the simulation device 16 and the evaluation device 18 are integrated into the data processing unit 6 as software modules in the present exemplary embodiment. However, they can also be integrated into the controller 4 of the optical observation system 1000 or into a PC belonging to the optical observation system 100. There is also the possibility that the determination device 14, the simulation device 16 and the evaluation device 18 are integrated into different components of the optical observation system 100. For example, the determination device 14 can be integrated into the controller 4 and the simulation device 16 and the evaluation device 18 can be integrated into a PC. In principle, it is also possible to design the determination device 14 and/or the simulation device 16 and/or the evaluation device 18 as an independent hardware module.

The determination device 14, the simulation device 16 and the evaluation device 18 essentially carry out three steps in the present exemplary embodiment. These are the detection of parameter values for parameters that influence the fluorescence intensity reaching the image sensor (determination device 14 in step S1), the simulation of the fluorescence intensity that reaches the image sensor 23 or the image sensors 61A, 61B from the respective object area 3A-H, one of The fluorescence observation model dependent on the parameters for which the parameter values are recorded is used (simulation device 16 in step S2) in order to determine the expected fluorescence intensity at the location of the image sensor 23 or the for the individual object areas 3A-H for a predetermined minimum concentration of the fluorescent dye Image sensors 61A, 61B to be determined, and the evaluation of the simulation by the (evaluation device 18 in step S3) in order to determine in which object areas 3A-H the expected fluorescence intensity at the location of the image sensor 23 or the image sensors 61A, 61B for the predetermined minimum concentration of the fluorescent dye is sufficient to be detected by the image sensor 23 or the image sensors 61 A, 61 B. If the evaluation shows that there are object areas 3A-H for which the minimum concentration of the fluorescent dye at the location of the image sensor 23 or the image sensors 61A, 61B does not lead to a fluorescence intensity that can be detected by this or by these, the evaluation device 18 can Issue warning. The evaluation device 18 can also perform other tasks such as creating the described graphical representation or the described determination of at least one improved parameter value.

In step S1, the current parameter values of the parameters used for the simulation in step S2 are recorded. Examples of parameters that can be included in the simulation in the present exemplary embodiment are the distance of the surgical microscope 2 from the object areas 3A-H of the observation object 3, the orientation of the surgical microscope 2 in relation to the object areas 3A-H of the observation object 3, the zoom setting of the Surgical microscope 2, the object focal length of the surgical microscope 2, the aperture setting of the surgical microscope 2, the gain of the image sensor 23 used in the surgical microscope 2 or the image sensors 61A, 61B used in the surgical microscope, the exposure time of the image sensor 23 used in the surgical microscope 2 or the image sensors 61A used in the surgical microscope , 61B, non-linearities of the image sensor 23 used in the surgical microscope 2 or the image sensors 61A, 61B used in the surgical microscope, the distance of the illumination system 40 from the object areas 3A-H of the observation object 3, the orientation of the illumination system 40 with respect to the object areas 3A-H of the Observation object 3, the intensity of an illumination light source 41, the spectral intensity distribution of an illumination light source 41, the zoom position of an illumination zoom, the position of an illumination aperture.

To record the parameter values, the data processing unit 6 in the present exemplary embodiment includes a parameter value determination device 14, which retrieves the parameter values set on the surgical microscope 2, on the stand 1 and on the lighting system 40 from the controller 4. The parameters Alternatively, values can also be recorded by reading out sensor readings. It is also possible to calculate parameter values indirectly from other, directly recorded parameter values. For example, the relative position and the relative orientation of the surgical microscope 2 with respect to the observation object 3 can be derived from the position and orientation of the surgical microscope 2 detected by means of the tracking system 12 in the coordinate system of the tracking system 12, as well as those also detected by means of the tracking system 12 in the coordinate system of the tracking system 12 Position and orientation of the observation object can be calculated. Alternatively, the positions and orientation of the surgical microscope 2 in relation to the object areas 3A-H of the observation object 3 can be determined from stereoscopic images of the observation object using stereographic methods. As in the present exemplary embodiment, variables such as the zoom position of the surgical microscope 2 or the lighting system 40 can be read out by the determination device 14 directly from the control 4 of the surgical microscope 2 and the lighting system or can be detected by sensors in the respective zoom system. The intensity and the spectral intensity distribution of the illumination light source 41 can also be recorded indirectly by calculating them using characteristic curves based on the value of a lifespan counter, which records the previous lifespan of the illumination light source.

Due to a progressive degradation of the illumination light source 41 over time, the intensity of the excitation wavelength currently emitted by the illumination light source 41 deviates over time from the nominally set intensity of the excitation wavelength. The nominally set intensity of the excitation wavelength results from the nominally set intensity of the illumination light source 41 and its spectral intensity distribution. In the present exemplary embodiment, the intensity of the excitation wavelength currently emitted by the illumination light source 41 can be determined by means of an intensity sensor 42 installed in the surgical microscope 2 or in the illumination system 40. The intensity sensor 42 is preferably only or at least mainly sensitive to the excitation wavelength in order to obtain the most accurate possible detection of the intensity of the excitation wavelength for the fluorescence. Optionally, the spectral intensity distribution of the illuminating light can also be determined using an intensity sensor, for example with the help of a multispectral sensor. The intensity of the excitation wavelength (or the spectral intensity distribution of the illumination light) that is currently emitted with the nominally set intensity of the illumination light source 41 can alternatively or additionally also be determined by placing a calibration target (e.g. white sheet of paper) is recorded with the image sensor 23 or the image sensors 61A, 61B and the signal generated in the image sensor 23 or the image sensors 61A, 61B is evaluated. Preferably, only the color channel(s) of the image sensor(s) that comes closest to the excitation wavelength is evaluated. The determined intensity can then be used in the simulation in step S2. From the determined intensity of the excitation wavelength, the deviation of the current intensity of the excitation wavelength from the intensity of the excitation wavelength resulting from the nominally set intensity of the illumination light source 41 can optionally also be determined and a correction value can be determined with which the setting of the illumination light source 41 can be corrected must in order to obtain a current intensity of the excitation wavelength corresponding to the nominal intensity of the excitation wavelength. With the correction value, the intensity of the illumination light source 41 can be adjusted in such a way that the degradation of the illumination light source 41 during fluorescence excitation is just compensated.

Another alternative or additional possibility of determining the intensity of the excitation wavelength currently emitted by the illumination light source 41 with the nominally set intensity is to determine via a lifespan counter of the illumination source 41 (typically a xenon lamp or LED) and that for the illumination source 41 nominally set intensity. As the service life of the illumination light source increases, the intensity of the illumination light emitted by it decreases, so that its actual intensity deviates from the nominally set intensity. Based on the previous service life of the illumination source 41, the current intensity of the illumination light source 41 can be determined from the nominally set intensity and used in the simulation of step S2. Optionally, from the degree by which the current intensity of the illumination light source 41 is reduced compared to the nominally set intensity, a correction value can be determined by which the intensity of the illumination light source 41 must be corrected in order to achieve a current intensity corresponding to the nominally set intensity receive. With the correction value, the intensity of the illumination light source 41 can be adjusted in such a way that the degradation of the illumination light source 41 during fluorescence excitation is just compensated. The degradation model used for the illumination source 41 can optionally occur in addition to the change in the overall intensity of the illumination source 41 over time take into account tending shifts in the spectral intensity distribution of the illumination radiation. The shift in the spectral intensity distribution means that the degree by which the current intensity of the illumination light source 41 is reduced compared to the nominally set intensity depends on the wavelength. By taking into account the shifts in the spectral intensity distribution that occur over time, the current intensity can be determined for each wavelength of the illuminating light. By taking into account the shifts in the spectral intensity distribution that occur over time, a correction value that is precisely tailored to the excitation wavelength of the fluorescence can be determined. Based on the corrected excitation wavelength, a calculation can be made as to how much fluorescent radiation is emitted. In the calculation, the spectral intensity distribution of the illumination light source 41 can be weighted with the effective spectral excitation curve of the fluorescence. Using such a weighting, wavelength-dependent properties of the observation object 3 can be taken into account when calculating the at least one correction value, for example the absorption of the excitation wavelength when penetrating the observation object.

The optional correction of the setting of the illumination light source 41 can be done automatically, so that the actual intensity of the illumination light source always corresponds to the nominally set intensity. However, there is also the possibility of telling the user how he needs to adjust the setting of the illumination light source 41 in order to achieve a desired intensity of the illumination light source.

In step S2, the fluorescence emission is then simulated using a model for observing the fluorescence intensity, a concentration of fluorescent dye which corresponds to a predetermined minimum concentration being assumed for the simulation. The model includes parameters of the lighting system 40, parameters of the surgical microscope 2 and parameters of the image sensor 23 or the image sensors 61A, 61B, which will be discussed in detail below. In the present exemplary embodiment, the simulation is carried out by a simulation device 16 integrated into the data processing unit 6.

The parameters of the illumination system 40 determine, among other things, the emission intensity of the illumination light source 41 and the emission wavelength of the illumination light source 41 and thus the excitation wavelength. In addition, the distance of the illumination system 40 from the object areas 3A-H of the observation object 3 and the orientation of the illumination system 40 in relation to the object areas 3A-H of the observation object 3 determine the zoom position of the illumination system 40 and the position of apertures in the illumination system 40 a lot of excitation radiation per unit area reaches an object area 3A-H. This influences the intensity of the excitation wavelength at the location of fluorescence excitation and thus also the intensity of the fluorescence emission, which in turn influences the fluorescence intensity arriving in the pixels of the image sensor 23 or the image sensors 61A, 61B.

The parameters of the surgical microscope 2 determine how much fluorescent radiation emitted from an object area 3A-H of the observation object 3 reaches the image sensor 23 or the image sensors 61A, 61B (or into the eyepiece). Here, for the fluorescence intensity reaching the image sensor 23 or the image sensors 61A, 61 B and thus for the signal strength caused in the pixels of the image sensor 23 or the image sensors 61A, 61 B, in particular the distance of the surgical microscope 2 from the object areas 3A-H of the observation object 3 and the orientation of the surgical microscope 2 in relation to the object areas 3A-H of the observation object 3, the zoom position in the surgical microscope 2, the transmission behavior of apertures (vignetting) and filters (transmittance) introduced into the observation beam path are important.

In the case of an image sensor, the gain (amplification factor), the exposure time, non-linearities of the image sensor and other variable factors of the image sensor influence the signal strength caused by the incoming fluorescence intensity in the pixels of the image sensor.

As part of the simulation of step S2, the influence of all of the parameters mentioned is preferably taken into account by means of an optical and system model when calculating the signal strength caused when the minimum concentration of fluorescent dye is present in the pixels of the image sensor 23 or the image sensors 61A, 61B.

An example of a possible optics and system model as follows $$\begin{array}{l}SensOrsiGnal=\left(1\text{/}beleucHtunGsterm\right)*\left(1\text{/}Einsammelterm\right)*\\ \left(1\text{/}SensOrtem\right)*ROHsiGnal\end{array}$$

The illumination term results from the parameter values of the illumination system 40, the collection term from the parameter values of the surgical microscope 2 and the sensor term from the parameter values of the image sensor 23 or the image sensors 61A, 61B.

A possible lighting term is: $$\begin{array}{l}beleucHtunGsterm=[(ZOOmfaktOrbeleucHtu\mathrm{<figure-callout\; id="2"\; label="n\; G"\; filenames="DE102022121505A1\_0009.png"\; state="\{\{state\}\}">n</figure-callout>}G2*(linearer\\ AbblendunGsfaktOrbeleucHtunGsst\ddot{a}rke)/(AbstandbeleucHtunG\mathrm{e.g}u\\ ObjektOberfl\ddot{a}cHe)2]*LicHtst\ddot{a}rkenfaktOr\_mit\_spektraler\\ AnreGunGsGewicHtunG*cOs(WinkelbeleucHtunG\mathrm{e.g}urOberfl\ddot{a}cHe\\ desbeObacHtunGsObjekts)\end{array}$$

A possible collection date is: $$\begin{array}{l}Einsammelterm=Einsammeleffi\mathrm{e.g}ien\mathrm{e.g}\left(ZOOmfaktOr,FOkussierunG\right)/\\ \left(AbstandObjektiv\mathrm{e.g}urOberfl\ddot{a}cHedesbeObacHtunGsObjekts\right)*\\ RicHtcHarakteristikfunktiOn\_der\_FluOres\mathrm{e.g}en\mathrm{e.g}emissiOn(Winkel\\ beObacHtunG\mathrm{e.g}urOberfl\ddot{a}cHedesbeObacHtunGsObjekts)*\\ bildOrtabH\ddot{a}nGiGer\_viGnettierunGsfaktOr\end{array}$$

Vorzugsweise werden bei der Berechnung des Beleuchtungsterms und des Einsammelterms Abschattungseffekte durch das Beobachtungsobjekt 3 mit einbezogen. Dies kann bspw. mit Hilfe einer stereografisch gewonnenen 3D-Tiefenkarte erfolgen. Satt stereografisch kann die 3D-Tiefenkarte auch auf andere an sich bekannte Weisen generiert werden, bspw. unter Verwendung eines Tiefensensors, mittels strukturierter Beleuchtung, etc.A possible sensor term is: $$\begin{array}{l}Kameraterm=Kamera\_AntwOrk\_FunktiOn(belicHtunGs\mathrm{e.g}eit,Gain\\ \text{ge}\text{measured raw signal})\end{array}$$

Preferably, shading effects caused by the observation object 3 are included in the calculation of the illumination term and the collection term. This can be done, for example, with the help of a stereographically obtained 3D depth map. The 3D depth map can also be generated stereographically in other ways known per se, for example using a depth sensor, using structured lighting, etc.

The illumination term, the collection term and the raw signal are each calculated for a pixel of the image sensor 23 or the image sensors 61A, 61B and the object area 3A-H of the observation object 3 imaged thereon. The size of an object area 3A-H of the observation object 3 therefore results in the present exemplary embodiment from the imaging factor with which the observation object 3 is imaged onto the image sensor 23 or the image sensors 61A, 61B, the imaging factor depending, among other things, on parameters of the surgical microscope 2, for example .from the zoom setting, the working distance, etc. In addition, the size of the object areas 3A-H can be enlarged to a pixel of the image sensor 23 or the image sensors 61A, 61B by combining several pixels arranged next to one another to form a larger pixel ( so-called binning) to increase the effective pixel area.

The evaluation device 18 then evaluates in step S3 the result of the simulation from step S2 to determine whether there are object areas 3A-H for which the fluorescence intensity determined with the predetermined minimum concentration of the fluorescent dye at the location of the image sensor 23 or the image sensors 61A, 61B is not sufficient in order to be detected by the image sensor 23 or the image sensors 61 A, 61 B. If the evaluation shows that there are object areas 3A-H for which the minimum concentration of the fluorescent dye at the location of the image sensor 23 or the image sensors 61A, 61B does not lead to a fluorescence intensity that can be detected by this or by these, the evaluation device 18 indicates In the present exemplary embodiment, a warning is issued (step S4). The warning can optionally be accompanied by a graphical representation which shows which object areas at the location of the image sensor 23 or the image sensors 61A, 61B do not lead to a fluorescence intensity that can be detected by the image sensor 23 or the image sensors 61A, 61B, so that the user can see the relevance the warning can be estimated. If the evaluation shows that object areas that do not lead to a fluorescence intensity detectable by the image sensor 23 or the image sensors 61A, 61B at the location of the image sensor 23 or the image sensors 61A, 61B are not present, the preparation of the fluorescence observation ends and the fluorescence observation can be carried out (step S5).

• - Ausrichtung des Blickwinkels des Operationsmikroskops 2 und/oder des Beleuchtungssystems 40 über das Stativ 1,
• - Optimieren der Lichtstärke der Optik des Operationsmikroskops 2 (Zoomeinstellung, Arbeitsabstand)
• - Optimieren der Empfindlichkeit des Bildsensors 23 bszw. der Bildsensoren 61A, 61 B
• - Erhöhung der Beleuchtungshelligkeit.

The evaluation device 18 can also optionally perform other tasks. For example, it can carry out the already mentioned determination of at least one improved parameter value. The optional determination of the at least one improved parameter value takes place either automatically or at the request of the user in step S6 if the evaluation in step S3 shows that object areas 3A-H are present for which the minimum concentration of the fluorescent dye at the location of the image sensor 23 or The image sensors 61A, 61B do not lead to a fluorescence intensity to be detected by them. In step S6, the evaluation device 18 then determines an improved parameter value for at least one parameter influencing the fluorescence intensity at the location of the image sensor 23 or the image sensors 61A, 61B, which leads to the number of object areas 3A-H for which the minimum concentration of the Fluorescent dye at the location of the image sensor 23 or the image sensors 61A, 61B does not lead to a fluorescence intensity to be detected by it or by these, is reduced. To improve the parameter value, the evaluation device 18 can, for example, vary the corresponding parameter value and calculate a quality value for each value of the parameter that occurs during the variation. For example, the number of object areas for which the minimum concentration of the fluorescent dye at the location of the image sensor 23 or the image sensors 61A, 61B does not lead to a detectable fluorescence intensity can be used as a quality value. Optionally, weighting can also be carried out to determine where the remaining object areas for which the minimum concentration of the fluorescent dye at the location of the image sensor 23 or the image sensors 61A, 61B does not lead to a detectable fluorescence intensity are located in the image. For example, object areas in the center of the image can receive a higher weighting than object areas at the edge of the image. The improvement of the at least one parameter can then be continued until the quality value has reached a minimum or falls below a predetermined value. Alternatively, the number of those areas for which the minimum concentration of the fluorescent dye at the location of the image sensor 23 or the image sensors 61A, 61B leads to a detectable fluorescence intensity can be used as a quality value. The improvement of the at least one parameter can then be continued until the quality value has reached a maximum or falls below a minimum value. Here too, object areas in the center of the image and object areas at the edge of the image field can be weighted differently. The improved parameter value of the at least one parameter can then either be set automatically (step S7) or displayed to the user as a recommended setting. For example, at least one of the following improvements can be made:

• - Alignment of the viewing angle of the surgical microscope 2 and/or the lighting system 40 via the tripod 1,
• - Optimizing the light intensity of the optics of the surgical microscope 2 (zoom setting, working distance)
• - Optimizing the sensitivity of the image sensor 23 or the image sensors 61A, 61B
• - Increase lighting brightness.

As part of the method, the user can also be offered the opportunity to set, for example in the controller 4 or in the evaluation device 18, which parameters influencing the observation of the fluorescence intensity should be automatically improved by the evaluation device 18 and which he himself would like to improve manually.

If the near-surface concentration of fluorescent dye is to be determined in the fluorescence observation carried out after the preparation of the fluorescence observation, the signal detected by the image sensor 23 or the image sensors 61A, 61B must be corrected using a prefactor. $$\begin{array}{l}KOrriGiertesSiGnal=vOrfaktOr*\left(1\text{/}beleucHtunGsterm\right)*\\ \left(1\text{/}Einsammelterm\right)*\left(1\text{/}SensOrterm\right)*ROHsiGnal\end{array}$$

For this purpose, the prefactor included in the optics and system model can be determined for each fluorescent dye and for an assumed tissue type through calibration. Depending on the type of tissue, a different prefactor is preferably used. If only the fluorescence intensity expected with the minimum concentration is to be determined for each object area, a suitable prefactor can be used can be determined using a model of the emission process and the properties of the optical system used.

Reference objects are used for calibration. A first reference object is, for example, a diffusely reflecting white object that diffusely reflects 50% of the irradiated light intensity. A second reference object is, for example, an aqueous solution in a reference vessel that contains a reference concentration of 100nM of the fluorescent dye. In addition, reference positions and orientations of the surgical microscope 2 and the lighting system 40 are defined in relation to the respective reference object. A reference parameter value is defined for all of the parameters of the surgical microscope 2, the lighting system 40 and the image sensor 23 or the image sensors 61A, 61B that are included in the simulation.

• - Senkrechte Beobachtung der Oberfläche des Referenzobjekts (Referenz-Beobachtungswinkel 90°),
• - Referenzort innerhalb des Bildes auf der dem Bildsensor: Das Objekt befindet sich im Zentrum des Bildes und wird auf den mittigen Pixel abgebildet,
• - Referenz-Arbeitsabstand zwischen Operationsmikroskop 2 und Objekt von 200mm,
• - Referenzstellung des Operationsmikroskops 2 (Referenz-Fokuslage = 200mm, Referenz-Zoomfaktor der Beobachtungsoptik gamma_Beobachtung = 1, Referenzblendenstellung: 100% geöffnete Blende),
• - Referenzhelligkeit der Beleuchtung von 50%, Referenzalter der Lampe: neue Xenon-Lampe,
• - Referenz-Einstellung von Beleuchtungsblende (vollständig offen) und Beleuchtungszoom (mittlere Stellung des Beleuchtungszooms, Zoomfaktor gamma_Beleuchtung = 1),
• - Referenzeinstellung der Bildsensorparameter: Gain=0, Belichtungszeit 20ms, usw.

For example, the reference parameter values can (but do not have to) be selected so that the following reference conditions exist for the calibration:

• - Vertical observation of the surface of the reference object (reference observation angle 90°),
• - Reference location within the image on the image sensor: The object is in the center of the image and is imaged on the central pixel,
• - Reference working distance between surgical microscope 2 and object of 200mm,
• - Reference position of the surgical microscope 2 (reference focus position = 200mm, reference zoom factor of the observation optics gamma_Observation = 1, reference aperture position: 100% open aperture),
• - Reference brightness of the lighting of 50%, reference age of the lamp: new xenon lamp,
• - Reference setting of the illumination aperture (fully open) and illumination zoom (middle position of the illumination zoom, zoom factor gamma_illumination = 1),
• - Reference setting of the image sensor parameters: Gain=0, exposure time 20ms, etc.

The calibration under the reference conditions determines which signal at the image sensor 23 or the image sensors 61A, 61B (i.e. in the present example the central reference pixel of the image sensor) generates the light reflected from the first reference object when the first reference object is illuminated with the reference brightness and the remaining reference settings are available. For example, the first reference object generates a reference raw signal of 100U (brightness units) on the reference pixel of the image sensor 23 or the image sensors 61A, 61B by reflecting the illumination light under the mentioned reference conditions. In the case of the existence of the reference conditions, the data processing unit therefore a reference intensity is stored in the form of a reference signal of also 100U.

In addition, the calibration under the reference conditions determines which signal is present at the image sensor 23 or the image sensors 61A, 61B (i.e. in the present example the central reference pixel of the image sensor) when the fluorescence in the second reference object occurs with the reference intensity of the excitation wavelength. For example, under reference conditions, the second reference object generates a reference raw signal of 100U (brightness units) on the reference pixel of the image sensor 23 or the image sensors 61A, 61B by fluorescence emission excited by the excitation wavelength. In the event that the reference conditions exist, a reference intensity in the form of a reference signal of also 100U is stored in the data processing unit 6.

The aim of the calibration is now that, regardless of whether the actual parameter values deviate from the reference parameter values, the same signal of 100U is always detected when the reference object is observed with the optical observation system 100 according to the invention. Furthermore, the aim of the calibration is that there is a linear relationship between the diffuse reflectivity of an observation object 3 and the detected signal or between the fluorescent dye concentration of the observation object 3 and the detected signal. For example, in a fluorescence observation with a concentration of 10nM, 50nM, 70nM, etc. of fluorescent dye that deviates from the reference concentration of 100nM, a corrected signal of 10HE, 50HE, 70HE, etc. should be detected with an otherwise identical second reference object. In the case of an observation of reflected light (e.g. white light observation), for example, a first reference object with a diffuse reflectivity of 10%, 20%, 30%, etc., which deviates from the reference reflectivity of 50%, should always have a corrected signal of 10 HU , 20 HU, 30 HU, etc. can be detected.

To ensure this, in the present exemplary embodiment in the data processing unit 6, for each parameter influencing the observation of the fluorescence intensity, there is a connection between the deviation of the actual value of the respective parameter from its reference value and the deviation of the signal representing the measured intensity through calibration or precalculation on the image sensor 23 or the image sensors 61A, 61B from the reference signal representing the reference intensity is stored in the memory of the signal processing unit. For example, it can be stored in the data processing unit 6 that an actual zoom factor gamma_Observation = 2 that deviates from the reference zoom factor gamma_Observation = 1 of the observation optics leads to the signal, and thus the measured intensity, being reduced by a factor of 0.25. The connection can, for example, be stored as an equation or system of equations. Alternatively, it can be stored in the form of a value table or several value tables. It is also possible for the relationship to be stored as an equation or system of equations for some parameters and in the form of a table of values or several tables of values for other parameters. In the case of one or more value tables, it is also possible to interpolate or extrapolate between the stored values.

With the help of the formula or formulas stored in the data processing unit 6 or the value table or tables of values stored in the data processing unit 6, a measured intensity, in particular a measured fluorescence intensity, can be converted into a corrected intensity, in particular a corrected fluorescence intensity. If, for example, in actual observation conditions that otherwise correspond to the reference conditions, only the actual zoom factor gamma_Observation = 2 deviates from the reference zoom factor gamma_Observation = 1, a raw material is generated by the image sensor 23 or the image sensors 61A, 61B for the reference concentration of fluorescent dye. Signal of 25U, which records the measured. To correct the raw signal, the raw signal is therefore divided by a deviation factor of 0.25 to obtain a corrected signal of 100HE, which corresponds to the fluorescence intensity of the reference concentration of the fluorescent dye of 100nM.

In another example, the raw signal is reduced by a deviation factor of 0.2, for example, due to vignetting of the optics at the edge of the image sensor 23 or the image sensors 61A, 61B with otherwise constant reference conditions. To correct the raw signal, this is divided by the deviation factor of 0.2 for pixels at the edge of the image sensor 23 or the image sensors 61A, 61B, so that in turn a corrected signal of 100U results.

In yet another example, the raw signal is increased by a factor of 9 to a raw signal of 900U with an illumination zoom factor gamma_illumination = 3 and otherwise unchanged reference conditions. To correct the raw signal, the raw signal of 900HE is divided by a deviation factor of 9 to obtain a corrected signal of 100HE, which corresponds to the fluorescence intensity of the reference concentration of the fluorescent dye of 100nM.

Various parameters influencing the observation of the fluorescence intensity can also be coupled with one another, e.g. the vignetting can depend on the location on the image sensor 23 or the image sensors 61A, 61B as well as on the optical observation parameters zoom, focus and aperture position. In this case, corresponding linked value tables or formulas are stored for the deviation factors.

The correction process generally needs to be carried out separately for each pixel of the image sensor 23 or the image sensors 61A, 61B because some deviation factors from the location on the image sensor 23 or the image sensors 61A, 61B (e.g. vignetting). However, a large part of the deviation factors are independent of the location on the image sensor 23 or the image sensors 61A, 61B (e.g. intensity setting of the light source), so that global deviation factors represent all locations on the image sensor 23 or the image sensors 61A, 61B.

As a result of the procedure described, the reference object with a 100nM concentration of the fluorescent dye always has a corrected measured value of 100HU, and an actual object that only deviates from the reference object in the concentration of the fluorescent dye of 33nM always has a corrected measured value of 33HU, etc. Are all the current parameter values of the The parameters influencing the observation of the fluorescence intensity are known and the resulting deviation factors are correctly stored in the system, then the corrected signal depends solely on the properties of the observation object 3 itself (e.g. optical properties, concentration of the fluorescent dye in the object, ...) and not on the settings of the surgical microscope 2 or the geometric position of the observation object 3 relative to the surgical microscope 2.

In a real system, not all current parameter values of the parameters influencing the observation of the fluorescence intensity can be determined and appropriately corrected. However, at least the influences of as many parameters as possible influencing the observation of the fluorescence intensity can be compensated for, so that the corrected signal is (almost) independent of at least the current parameter values of these parameters. In order to still achieve a reliable corrected signal, the user of the surgical microscope 2 should ensure that all parameters that influence the observation of the fluorescence intensity and that cannot be corrected by the system correspond to the reference conditions. For example, if the influence of the lighting intensity on the signal is not stored in the system in a way that can be corrected, the user should ensure that the reference lighting intensity is set.

Of course, the calibration can also be carried out if the simulation is only intended to determine the fluorescence intensity to be expected with the minimum concentration for each object area.

Fluorescent dyes such as PPIX slowly fade when excited and gradually lose their fluorescence intensity. In a further training of the in 1 The optical observation system 100 shown is therefore preferably registered for each object area 3A-H of the observation object 3 by the data processing unit 6, for which period of time the corresponding object area 3A-H has already been illuminated with the excitation wavelength. The fluorescence signal emerging from the respective object area 3A-H is then corrected by the data processing unit 6 by a bleaching factor determined on the basis of the time period, or a warning is displayed if the duration and intensity of the illumination is at the excitation wavelength than for the fluorescent dye have exceeded the level considered acceptable. In the event of a movement of the surgical microscope 2 relative to the observation object 3, a tracking system and the determination of a depth map are preferably provided in order to be able to assign the respective object areas 3A-H in the new perspective to the previous object areas 3A-H in the correct location. For each object area 3A-H of the observation object 3, a counter is then created, so to speak, which counts how much excitation light has already been irradiated onto this point.

The present invention has been described in detail for explanatory purposes using exemplary embodiments. However, a person skilled in the art will recognize that within the scope of the invention, deviations can be made from the exemplary embodiments described. The invention should therefore not be limited by the exemplary embodiments, but rather only consists of the appended claims.

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• US 2019227288 A1 [0005, 0007]
• US 2016278678 A1 [0006, 0007]

Claims (22)

Method for preparing the observation of a fluorescence intensity of fluorescence radiation of a fluorescent dye in an observation object (3), which comprises object areas (3A-H) that differ from one another in their depth and/or orientation, wherein the observation is to be carried out with an optical observation system (100). , with which fluorescence radiation can be observed, provided that it has a certain minimum intensity, and wherein the method comprises the steps: - determining the parameter value of at least one parameter influencing the observation of the fluorescence intensity, and - simulating the for the respective object areas (3A-H). expected fluorescence intensity based on the determined parameter value of the at least one parameter and a model for the influence of the at least one parameter on the fluorescence intensity, characterized in that - a minimum concentration of the fluorescent dye is specified within the framework of the simulation and - based on the simulation for each object area ( 3A-H) the fluorescence intensity expected with the minimum concentration is determined. Procedure according to Claim 1 , characterized in that a check is carried out for each object area as to whether the expected fluorescence intensity determined is sufficient to be detected by the optical observation system (100) with the given sensitivity. Procedure according to Claim 1 or Claim 2 , characterized in that as part of the simulation, information about the depth distribution of the object areas (3A-H) and/or information about the orientation of the object areas (3A-H) is used. Procedure according to one of the Claims 1 until 3 , characterized in that at least the parameter value of one of the following parameters is determined and taken into account in the simulation: - Distance of an optical observation device (2) of the observation system (100) from the object areas (3A-H), - Orientation of the optical observation device (2) in relation to the object areas (3A-H), - zoom setting of the optical observation device (2), - object focal length of the optical observation device (2), - aperture setting of the optical observation device (2), - gain of an image sensor used in the optical observation device (2) ( 23, 61A, 61B), - exposure duration of an image sensor (23, 61A, 61B) used in the optical observation device (2), - nonlinearities of an image sensor (23, 61A, 61B) used in the optical observation device (2), - distance of an illumination system ( 40) of the object areas (3A-H), - orientation of the lighting system (40) of the observation system (100) in relation to the object areas (3A-H), - intensity of an illuminating light source (41) of the lighting system (40); - spectral intensity distribution of an illumination light source (41), - zoom position of an illumination zoom, - position of an illumination aperture. Procedure according to Claim 4 , characterized in that the spectral intensity distribution of the illumination light source (41) is determined based on the value of a lifespan counter of the illumination source (41) and its nominally set intensity using a degradation model for the illumination source (41). Procedure according to one of the Claims 1 until 5 , characterized in that a warning is issued if the check shows that the expected fluorescence intensity determined for the minimum concentration of the fluorescent dye is not sufficient in every object area (3A-H) to be compatible with the given sensitivity of the optical observation system (100). to be able to be detected. Procedure according to one of the Claims 1 until 6 , characterized in that a graphical representation is generated which shows in which object areas (3A-H) the expected fluorescence intensity determined is sufficient to be detected by the optical observation system (100) with the given sensitivity and in which object areas (3A-H) not. Procedure according to one of the Claims 1 until 7 , characterized in that the simulation is used to determine an improved parameter value for the at least one parameter in such a way that the expected fluorescence intensity simulated with the improved parameter value in as many object areas as possible (3A-H) is sufficient to be compatible with the given sensitivity of the optical observation system ( 100) can be detected by this. Procedure according to Claim 8 , characterized in that the current parameter value of the at least one parameter is automatically set to the improved parameter value. Procedure according to one of the Claims 1 until 9 , characterized in that - with a reference parameter value for the at least one parameter influencing the observation of the fluorescence intensity, at least one reference measurement is carried out with a reference concentration of the fluorescent dye in order to obtain a reference value for the fluorescence intensity at the reference concentration of the fluorescent dye, - simulating the for fluorescence intensity to be expected in each object area (3A-H), in which a change in the fluorescence intensity in comparison to the reference intensity is determined for a deviation of the parameter value of the at least one parameter influencing the observation of the fluorescence intensity from the reference parameter value, - and a compensation factor is determined, with which the change in fluorescence intensity caused by the deviation of the parameter value of the at least one parameter influencing the observation of the fluorescence intensity from the reference parameter value can be compensated for in a digital image. Method for observing a fluorescence intensity of fluorescence radiation of a fluorescent dye in an observation object (3), which comprises object regions (3A-H) that differ from one another in their depth and/or orientation, the observation being carried out with an optical observation system (100), with which fluorescence radiation can be observed, provided that it has a certain minimum intensity, characterized in that it is the method for preparing the observation of a fluorescence intensity of fluorescence radiation of a fluorescent dye in an observation object (3) according to one of Claims 1 until 10 includes. Optical observation system (100) for observing a fluorescence intensity of fluorescence radiation of a fluorescent dye in an observation object (3), which comprises object areas (3A-H) that differ from one another in their depth and/or orientation, the optical observation system (100) being designed in this way that it enables the observation of the fluorescence intensity, provided that it has a certain minimum intensity, and that includes: - a determination device for determining the parameter value of at least one parameter influencing the observation of the fluorescence intensity, and - a simulation device (16) for simulating the for the respective object areas (3A-H) expected fluorescence intensity on the basis of the determined parameter value of the at least one parameter and a model for the influence of the at least one parameter on the fluorescence intensity, characterized by an evaluation device (18), which is designed for one within the scope of Simulation predetermined minimum concentration of the fluorescent dye to determine the fluorescence intensity expected with the minimum concentration for each object area (3A-H) using the simulation Optical observation system (100). Claim 12 , characterized in that the evaluation device (18) is designed to carry out a check for each object area (3A-H) as to whether the determined expected fluorescence intensity is sufficient to be detected by the optical observation system (100) with the given sensitivity can. Optical observation system (100). Claim 12 or Claim 13 , characterized in that the simulation device (16) is designed to use information about the depth distribution of the object areas (3A-H) and / or information about the orientation of the object areas (3A-H) as part of the simulation. Optical observation system (100) according to one of the Claims 12 until 14 , characterized in that the determination device (14) is designed to determine at least the parameter value of one of the following parameters and the simulation device (16) is designed to take this at least one parameter value into account in the simulation: - Distance of an optical observation device (2) the optical observation system (100) from the Object areas (3A-H), - orientation of the optical observation device (2) in relation to the object areas, - zoom setting of the optical observation device (2), - object focal length of the optical observation device (2), - aperture setting of the optical observation device (2), - gain an image sensor (23, 61A, 61 B) used in the optical observation device (2), - exposure duration of an image sensor (23, 61A, 61 B) used in the optical observation device (2), - non-linearities of an image sensor used in the optical observation device (2) ( 23, 61A, 61 B), - Distance of an illumination system (40) of the optical observation system (100) from the object areas (3A-H), - Orientation of the illumination system (40) in relation to the object areas (3A-H), - Intensity an illuminating light source (41); - spectral intensity distribution of an illumination light source (41), - zoom position of an illumination zoom, - position of an illumination aperture. Optical observation system (100) according to one of the Claims 12 until 15 , characterized by the evaluation unit (18) is designed to generate a graphical representation which shows in which object areas (3A-H) the expected fluorescence intensity determined is sufficient to be detected by the optical observation system (100) with the given sensitivity to be able to become. Optical observation system (100) according to one of the Claims 11 until 16 , characterized by an optimization unit (22) which is designed to use the simulation to determine an improved parameter value for the at least one parameter in such a way that the expected fluorescence intensity simulated with the improved parameter value is sufficient in as many object areas (3A-H) as possible, in order to be detected by the optical observation system (100) with the given sensitivity. Optical observation system (100). Claim 17 , characterized by a control unit (4) for controlling the optical observation system (100), which is connected to the optimization unit (22) to receive the improved parameter value and is designed to set the at least one parameter to the improved parameter value. Optical observation system (100) according to one of the Claims 12 until 18 , characterized by a compensation factor determination unit (24), which is designed to determine a compensation factor with which the observation of the fluorescence intensity is influenced by a deviation in the parameter value of the at least one digital image recorded with an image sensor (23, 61A, 61B). Parameter can compensate for the change in the fluorescence intensity caused by a reference parameter value, wherein the compensation factor determination unit (24) is further designed to determine the compensation factor on the basis of at least one reference value for the fluorescence intensity determined for a reference concentration of the fluorescent dye and for a reference parameter value and a simulation of the for the respective Object areas to determine the expected fluorescence intensity, wherein in the simulation for a deviation of the parameter value of the at least one parameter influencing the observation of the fluorescence intensity from the reference parameter value, a change in the fluorescence intensity is determined in comparison to the reference intensity. Computer-implemented method for preparing the observation of a fluorescence intensity of fluorescence radiation of a fluorescent dye in an observation object (3), which comprises object areas (3A-H) that differ from one another in their depth and/or orientation, the observation being carried out with an optical observation system (100). with which fluorescence radiation can be observed, provided that it has a certain minimum intensity, and wherein the method comprises the steps: - receiving or retrieving the parameter value of at least one parameter influencing the observation of the fluorescence intensity, and - simulating the for the respective object areas (3A- H) expected fluorescence intensity based on the determined parameter value of the at least one parameter and a model for the influence of the at least one parameter on the fluorescence intensity, characterized in that - a minimum concentration of the fluorescent dye is specified within the framework of the simulation and - based on the simulation for The fluorescence intensity expected with the minimum concentration is determined for each object area (3A-H). Computer program for preparing the observation of a fluorescence intensity of fluorescence radiation of a fluorescent dye in an observation object (3), which comprises object areas (3A-H) that differ from one another in their depth and/or orientation, wherein the observation is to be carried out with an optical observation system (100). , with which fluorescence radiation can be observed, provided that it has a certain minimum intensity, the computer program comprising instructions which, when executed on a computer, cause it to - receive or retrieve the parameter value of at least one parameter influencing the observation of the fluorescence intensity , and - to simulate the fluorescence intensity expected for the respective object areas (3A-H) on the basis of the received or retrieved parameter value of the at least one parameter and a model for the influence of the at least one parameter on the fluorescence intensity, characterized in that - in As part of the simulation, a minimum concentration of the fluorescent dye is specified and - based on the simulation for each object area (3A-H) the fluorescence intensity to be expected with the minimum concentration is determined. Data processing unit (6) for preparing the observation of a fluorescence intensity of fluorescence radiation of a fluorescent dye in an observation object (3), which comprises object areas (3A-H) that differ from one another in their depth and/or orientation, the observation being carried out with an optical observation system (100 ) is to be carried out with which fluorescence radiation can be observed, provided that it has a certain minimum intensity, the data processing unit (6) comprising a memory and a processor and the processor being designed by means of a computer program stored in the memory to - the parameter value of at least one observation to receive or retrieve the parameter influencing the fluorescence intensity, and - the fluorescence intensity to be expected for the respective object areas (3A-H) on the basis of the received or retrieved parameter value of the at least one parameter and a model for the influence of the at least one parameter on the fluorescence intensity simulate, characterized in that - as part of the simulation, a minimum concentration of the fluorescent dye is specified and - based on the simulation for each object area (3A-H), the fluorescence intensity to be expected with the minimum concentration is determined.

Images