DE102020121453A1 - Assistance device for assisting with an intravitreal injection, computer-implemented method for assisting with an intravitreal injection, computer program and non-volatile computer-readable storage medium - Google Patents

Abstract

An assistance device is provided for assisting with an intravitreal injection. The assistance device comprises: - an optical observation device (5) which is designed to record image data of an eye (1) during an eye treatment; - a controller (7) which is designed to supply the image data from the optical observation device (5) receive and evaluate the received image data in order to derive relevant geometric data from the image data for the intravitreal injection, - the optical observation device (5) being designed to display a real-time image of the eye (1) to which the control derived for the intravitreal Injection relevant geometric data are superimposed.

Description

The present invention relates to an assistance device for assisting with an intravitreal injection. In addition, the invention relates to a computer-implemented method for assisting with an intravitreal injection and a computer program for assisting with an intravitreal injection and a non-volatile computer-readable storage medium with such a computer program.

The standard treatment for age-related macular degeneration (AMD), macular edema, central vein thrombosis and other retinal diseases are intravitreal injections, i.e. injections into the vitreous humor of the eye, with anti-vascular endothelial growth factors (anti-VEGF) such as ranibizumab or bevacizumab. Patients usually receive two to three injections per year, the injections being carried out under sterile conditions in an operating room and a surgical microscope often being used. A number of conditions must be observed during the injection, as described, for example, by Andrzej Grzybowski et al. is described in "2018 Update on Intravitreal Injections: Euretina Expert Consensus Recommendations". For example, injections should be given through a section of the eyeball called the pars plana to avoid damaging the ciliary body or retina. In addition, to prevent infection, each injection should be given in a location that has not previously been used for an injection. It is also important to ensure that the injection needle is only inserted so far into the eyeball that the drug emerges from the injection needle approximately in the center of the vitreous humor and not too close to the retina, in order to avoid damage to the retina. In addition, the injection should preferably be carried out using a so-called double-plane tunnel technique, in which the dermis is pierced at an angle of 15 to 30 ° and the injection needle is then positioned at an angle of 45 ° to 60 °, with the tip the needle is still in the dermis. This technology creates a tunnel on two different levels. This technique is described, for example, in L. Myers et al. "Intravitreal Injection Technique: A Primer for Ophthalmology Residents and Fellows, ophthalmology and visual signs (available at http://medison.uiowa.edu/e).

Since the conditions described are difficult to reproduce, the treating person needs sufficient training, which is why in many countries such injections can only be carried out by a doctor. However, it is estimated that there are more than 6 million intravitreal injections per year in the United States alone. The doctor's time is a cost factor, which is why in some countries, for example in Great Britain, a trial is being made to allow the injections to be carried out by specially trained nursing staff.

Although there are no studies to date on whether there is a correlation between the exact position at which the drug emerges from the injection needle, for example in relation to its distance from the retina, it is plausible to assume that the effectiveness of the treatment with increasing removal of the Leakage of the medicament from the retina decreases, whereby it should be noted, however, that the leakage must not take place too close to the retina, in order to avoid damage to the retina.

For example, in US 2018/0360654 A1 , US 2017/0266045 A1 or in US 2010/0152646 A1 mechanical systems for assisting inravitreal injections have been proposed. Also, for example, are off US 2017/0135768 A1 optical aids for assisting a subretinal injection are known. From the published after the priority date of the present application CN 110384581 A there is also known an intelligent intraocular injection device comprising a computer, an injection control unit and an image analysis unit, the device recognizing the movements of a marker placed on the eyeball and performing the injection when the marker is at rest.

Although the devices described in the cited documents can support the treating person with certain aspects of an intravitreal injection, all the devices known from the cited documents retain aspects of the intravitreal injection for which they cannot offer any support.

It is therefore a first object of the present invention to provide an assistance device for assisting with an intravitreal injection which enables extensive support for a treating person with the intravitreal injection.

A second object of the present invention is to provide a computer-implemented method as well as a computer program and a non-volatile computer-readable storage medium which make it possible to provide a treating person with extensive support when performing an intravitreal injection.

The first object is achieved by an assistance device according to claim 1, the second object by a computer-implemented method according to claim 9, a computer program according to claim 18 and a non-volatile one Computer-readable storage medium according to claim 19. The dependent claims contain advantageous embodiments of the invention.

An assistance device according to the invention for assisting with an intravitreal injection comprises an optical observation device which is designed to record image data of an eye. The optical observation device can in particular be a surgical microscope with at least one image sensor for recording the image data. In addition, the assistance device according to the invention comprises a controller which is designed to receive the image data from the optical observation device and to evaluate the received image data in order to derive relevant geometric data for the intravitreal injection from the image data. The control can be a dedicated hardware module in which a control program is stored in a read-only memory, or a software module which is executed, for example, on a computer of the assistance device. Within the scope of the invention, the optical observation device is designed to display a real-time image of the eye on which the geometrical data relevant for the intravitreal injection, derived from the control, are superimposed. The real-time image overlaid with the geometric data derived from the control can be displayed on an internal display of the optical observation device or on an external display connected to the optical observation device, which can be, for example, a monitor or a head-mounted display. The display can be a stereoscopic display, that is to say a display which shows the surgical field stereoscopically and thus enables the viewer to perceive the surgical field spatially. In this case, the superimposed geometric data can also be displayed stereoscopically so that the viewer can also perceive them spatially.

By superimposing the geometric data in the real-time image of the eye, instructions relevant to the intravitreal injection can be visually displayed to a treating person on the basis of the geometric data.

The geometric data can be represented in a variety of ways and can represent different types of data. For example, the control can be configured to derive from the image data as the geometrical data relevant for the intravitreal injection, geometrical data relevant for at least an intravitreal injection position of an injection needle. The geometric data relevant for the intravitreal injection position can include, for example, a target position at which the injection needle for an intravitreal injection is to be placed. This target position can be determined, for example, in that certain features of the eye are recognized by means of image analysis software and a suitable injection position is determined on the basis of the position of the recognized features. For example, the iris can be recognized by means of image analysis software and the position of the pars plana can be determined from the position of the iris. A position in the area of the Pars Plana can then be specified as the target position.

The assistance device according to the invention can also comprise a memory for storing positions of previous punctures in the eye. In this case, the control is then designed to derive the geometric data relevant for the intravitreal injection from the image data, taking into account the positions of previous punctures. If the geometric data relevant for the intravitreal injection include a target position, the control can in particular be designed to derive the target position taking into account the positions of previous punctures. Additionally or alternatively, the control can be designed to derive those current positions in the eye as the geometric data relevant for the intravitreal injection, which correspond to the positions of previous punctures stored in the memory. This can be used to indicate to the treating person, for example, the positions at which punctures have already taken place in the eye, so that a new injection at one of these positions or in close proximity to one of these positions can be avoided. In particular, it is also possible to display the positions of previous punctures in the eye in addition to the target position. The positions of previous punctures in the eye can represent the positions of previous intravitreal injections. However, they can also represent positions where trocars were placed during previous eye treatments. The positions of previous punctures can in particular also represent positions of previous intravitreal injections as well as positions at which trocars were placed in the context of previous eye treatments.

Additionally or alternatively, the control can be configured to derive at least geometric data for the injection depth and the injection angle of an injection needle with which the intravitreal injection is to be made from the image data as the geometric data relevant for the intravitreal injection. By superimposing the geometric data for the injection angle and the injection depth in the real-time image of the eye, the treating person can be offered valuable support when injecting with the double plane tunnel technique. If the display is a stereoscopic display, the geometric data for the injection angle and the injection depth can be available, for example, in the form of a spatial vector representing a target position and a target orientation of the injection needle, with which the image of the injection needle is to be made to coincide .

The controller preferably includes an eye tracker for determining the position and orientation of the eye, in particular also the rotation of the eye around the optical axis, so that the controller can detect a change in the position and orientation of the eye and the displayed geometric data can adapt to the new position and orientation of the eye. Tracking can take place, for example, on the basis of the scleral vessels or on the basis of the iris. It is particularly advantageous if the control also includes a needle tracking module for determining the position and the orientation of the injection needle. In this way, the controller can not only specify an injection angle and an injection depth, but also monitor whether the treating position complies with the specified injection angle and the specified injection depth. For this purpose, the control unit only needs to check whether the position of the injection needle in relation to the position of the eye corresponds to the specified position and / or whether the orientation of the injection needle in relation to the orientation of the eye corresponds to the specified injection angle. From the length of the injection needle visible in the recorded image, the controller can also determine, taking into account the orientation of the injection needle in relation to the eye, whether the injection depth corresponds to the predefined injection depth.

• - Empfangen von Bilddaten eines Auges;
• - Auswerten der Bilddaten, um aus den Bilddaten für die intravitreale Injektion relevante geometrische Daten abzuleiten;
• - Generieren eines für die intravitreale Injektion relevante geometrischen Daten repräsentierenden Bilddatensatzes und von Informationen, die es ermöglichen, den Bilddatensatz den Bilddaten des Auges zu überlagern; und
• - Bereitstellen des Bilddatensatzes und der Informationen zur Überlagerung über die Bilddaten des Auges.

According to a second aspect of the present invention, a computer-implemented method for assisting with an intravitreal injection is provided. The procedure consists of the following steps:

• - Receiving image data of an eye;
• Evaluation of the image data in order to derive relevant geometric data for the intravitreal injection from the image data;
• Generation of an image data set representing geometric data relevant for the intravitreal injection and of information which makes it possible to superimpose the image data set on the image data of the eye; and
• - Provision of the image data set and the information for superimposition on the image data of the eye.

In an alternative embodiment of the computer program according to the invention, the generation of the image data set representing the geometric data relevant for the intravitreal injection and the provision of the image data set and the information for superimposing the image data of the eye can be achieved by generating an instruction data set comprising the geometric data relevant for the intravitreal injection and the output of the instruction data set to a robotic injection system can be replaced so that the intravitreal injection can also be carried out without active intervention by a person.

The computer-implemented method according to the invention can in particular be executed on a computer or some other unit of the assistance device equipped with a processor in order to implement the control described with reference to the assistance device. As a result, the computer-implemented method makes it possible to implement the assistance system according to the invention and the possible support for a treating person during an intravitreal injection.

As part of the evaluation, geometric data relevant to at least one intravitreal injection position of an injection needle can be derived from the image data as the geometric data relevant for the intravitreal injection. These geometric data relevant for the intravitreal injection position can represent, for example, a target position at which an injection needle for an intravitreal injection is to be placed. In addition, as part of the computer-implemented method, there is the possibility of retrieving stored positions of previous punctures in the eye, for example due to previous intravitreal injections or due to trocars that were placed during previous eye treatments, from a memory, and the geometrical positions relevant for the intravitreal injection Derive data from the image data, taking into account the positions of previous punctures called up from the memory. If the geometric data relevant for the intravitreal injection include a target position, the target position can be derived taking into account the positions of previous punctures. Additionally or alternatively, the geometric data relevant for the intravitreal injection can be derived from those current positions in the eye which correspond to the positions of previous punctures stored in the memory. This embodiment of the method according to the invention makes it possible, for example, to display those positions that are to be avoided during the intravitreal injection that is currently to be performed. This display can in particular also take place in addition to the display of the target position. This can be shown to the treating person how far the target position is from the positions of previous punctures in the eye.

Additionally or alternatively, at least geometric data for the injection depth and the injection angle of an injection needle with which an intravitreal injection is to be made can be derived from the image data in the context of the evaluation. With the help of this geometric data, which can be done in the case of a representation on a stereoscopic display in the form of a vector, a treating person can be supported in inserting the injection needle into the eyeball. For support, the orientation and the position of the injection needle in relation to the eye can be determined from the image data, in particular by means of an image analysis. The geometric data can then indicate the deviation of the injection needle from the injection depth and the injection angle with which the injection is to take place. These deviations can be specified as an alternative to the injection depth and the injection angle or in addition to the injection depth and the injection angle.

• - Bilddaten eines Auges zu empfangen,
• - die Bilddaten auszuwerten, um aus den Bilddaten für die intravitreale Injektion relevante geometrische Daten abzuleiten,
• - einen die für die intravitreale Injektion relevanten geometrischen Daten repräsentierenden Bilddatensatz und Informationen, die es ermöglichen, den Bilddatensatz den Bilddaten des Auges zu überlagern, zu generieren und
• - den Bilddatensatz und die Informationen zur Überlagerung über die Bilddaten des Auges bereit zu stellen.

According to the second aspect of the invention, a computer program for assisting with an intravitreal injection is also provided. The computer program includes instructions which, when executed on a computer, cause the computer to:

• - receive image data of an eye,
• - evaluate the image data in order to derive relevant geometric data for the intravitreal injection from the image data,
• an image data set representing the geometric data relevant for the intravitreal injection and information which make it possible to superimpose the image data set on the image data of the eye, to generate and
• - to provide the image data set and the information for overlaying the image data of the eye.

With the aid of the computer program according to the invention, a computer can be configured to carry out the computer-implemented method according to the invention. Developments of the computer program according to the invention can include instructions which, when executed on a computer, configure the computer to carry out the developments of the computer-implemented method according to the invention.

• - Bilddaten eines Auges zu empfangen,
• - die Bilddaten auszuwerten, um aus den Bilddaten für die intravitreale Injektion relevante geometrischen Daten abzuleiten,
• - einen die für die intravitreale Injektion relevanten geometrischen Daten repräsentierenden Bilddatensatz und Informationen, die es ermöglichen, den Bilddatensatz den Bilddaten des Auges zu überlagern, zu generieren, und
• - den Bilddatensatz und die Informationen zur Überlagerung über die Bilddaten des Auges bereitzustellen.

According to the invention, a non-volatile computer-readable storage medium with instructions stored thereon for assisting with an intravitreal injection is also provided. The instructions cause a computer, when executed on it, to:

• - receive image data of an eye,
• - evaluate the image data in order to derive relevant geometric data for the intravitreal injection from the image data,
• an image data set representing the geometric data relevant for the intravitreal injection and information which make it possible to superimpose the image data set on the image data of the eye, and
• to provide the image data set and the information for superimposing on the image data of the eye.

With the aid of the non-volatile computer-readable storage medium, the computer program according to the invention can be loaded into a computer. Developments of the non-volatile computer-readable storage medium make it possible to load the developments of the computer program according to the invention into a computer, and thus to have the developments of the computer-implemented method according to the invention carried out by a computer.

• 1 zeigt ein exemplarisches Ausführungsbeispiel für eine Assistenzvorrichtung zum Assistieren bei einer intravitrealen Injektion.
• 2 die Steuerung für die Assistenzvorrichtung aus 1.
• 3 zeigt den Aufbau eines Operationsmikroskops in einer schematischen Darstellung.
• 4 zeigt einen alternativen Aufbau eines Operationsmikroskops in einer schematischen Darstellung.
• 5 zeigt ein exemplarisches Ausführungsbeispiel für das computerimplementierte Verfahren zum Assistieren bei einer intravitrealen Injektion in Form eines Ablaufdiagramms.

Further features, properties and advantages of the present invention emerge from the following description of exemplary embodiments with reference to the accompanying figures.

• 1 shows an exemplary embodiment of an assistance device for assisting with an intravitreal injection.
• 2 the control for the assistance device 1 .
• 3 shows the structure of a surgical microscope in a schematic representation.
• 4th shows an alternative construction of a surgical microscope in a schematic representation.
• 5 shows an exemplary embodiment for the computer-implemented method for assisting with an intravitreal injection in the form of a flowchart.

The present invention is described in detail below on the basis of exemplary embodiments. A person skilled in the art will recognize that he can deviate from the exemplary embodiments within the scope of the present invention. The invention is therefore not to be restricted by the exemplary embodiments, but rather only by the appended claims.

1 shows, in a highly schematic manner, an exemplary embodiment of an assistance device for assisting with an intravitreal injection, ie an injection into the vitreous humor of the eye 1 . The figure shows an eye to be treated 1 along with a syringe 3 , with which the intravitreal injection is to be carried out, as well as an assistance device that supports the treating person with the injection. In the present exemplary embodiment, the assistance device comprises a surgical microscope 5 that has one or more digital image sensors with which a real-time image of the eye 1 into which the injection is to be made is recorded. Possible configurations of the surgical microscope 5 will be later with reference to the 3 and 4th explained. The assistance system is also with a surgical microscope 5 associated monitor 9 equipped on which with the image sensor or the image sensors of the surgical microscope 5 recorded real-time images are displayed. A surgical microscope is usually capable of stereoscopic partial images of the eye 1 to record. Using shutter glasses and a time-sequential display of the stereoscopic partial images on the monitor 9 can then be a three-dimensional representation of the eye 1 respectively.

In the present exemplary embodiment, the assistance system also includes a controller 7th between the surgical microscope 5 and the monitor 9 is switched. The control 7th receives from the image sensor or the image sensors of the surgical microscope 5 the digital image data and is designed to evaluate the received image data in order to derive therefrom relevant geometric data for the intravitreal injection. These geometric data are used by the controller 7th superimposed on the image data of the digital real-time image in the form of image data. On the monitor 9 the image resulting from the overlay is then displayed, in which, in addition to the real-time image, information is shown which is provided by the controller 7th represent derived geometric data. When on the monitor 9 If time-sequential stereoscopic partial images are displayed, the geometric data relevant for the intravitreal injection can be displayed spatially, for example in the form of a spatial vector.

In the present exemplary embodiment, a target position is used as the relevant geometric data 11 where to inject, along with locations 13th that have previously had intravitreal injections is displayed. In the present exemplary embodiment, the target position is determined taking the positions into account 13th , on which intravitreal injections have already taken place, are determined in such a way that they have a predetermined minimum distance or the greatest possible distance from the positions 13th who have previously had intravitreal injections. In the present exemplary embodiment, information fields are the real-time image as further relevant geometric information 15th , 17th superimposed, one of which (field 15th ) indicates the angle that the injection needle 19th the syringe 3 with respect to the surface normal of the eyeball at the target position 11 occupies. In the present exemplary embodiment, the numbers are shown in red as long as the deviation of the actual angle from a target angle suitable for the injection exceeds a predetermined tolerance range. As soon as the tolerance range is no longer exceeded, the display changes from red to green, indicating to the treating person that the injection angle is now suitable for performing the injection.

In the information field 17th in the present exemplary embodiment, however, the depth with which the injection needle is displayed 19th has penetrated the eyeball so far. Color coding can also be used here. For example, the displayed depth can be displayed in black until a specified target depth is reached. The display then changes to green. If, on the other hand, a specified maximum depth is exceeded, the display changes to red to indicate that injections must not be deeper and that the syringe should be withdrawn as far as possible so that the display changes back to green.

The real-time image of the eye can be used as additional geometric data 1 Data are superimposed showing the location of the Pars Plana 21 view in eye. The display can be, for example, by a Pars Plana 21 surrounding contour 22nd or by coloring the Pars Plana 21 be realized.

The color coding can differ from the coding described and in particular also depend on the injection technique. If, for example, the double plane tunnel technology is used, the green color coding of the injection angle can be changed to yellow after a certain injection depth to indicate that the injection angle must now be changed. The display then changes back to green when the new target angle is reached. For example, the first target angle can be in the range between 60 ° and 75 ° based on the surface normal of the eyeball at the target position and after a certain penetration depth of the injection needle 19th change to a new target angle in the range between 30 ° and 45 ° based on the surface normal of the eyeball.

An alternative way, as the geometric data, the correct injection angle is that which is the target position 11 The marking changes color as soon as the correct injection angle is reached within the tolerance limits. Another way of showing the correct injection angle is to find the target position 11 indicated by an open circle, which is completely or partially filled when the specified injection angle is reached within the tolerance limits.

In addition, there is also the possibility of displaying the target injection depth and the target injection angle in the form of a target vector, the tip of which represents the position of the injection needle tip when the target injection depth is reached and its orientation represents the target injection angle, with which the injection needle 19th should be brought to congruence, so that the injection needle tip is at the location of the tip of the target vector and the injection needle 19th extends along the target vector. If, in the context of the double plane tunnel technique, different target depths and different target angles of the injection needle are used one after the other 19th are to be taken, the representation of the previous target vector can be replaced by a new target vector when the injection needle 19th has been brought into congruence with the first nominal vector.

The structure of the control 7th the assistance device is in 2 shown schematically. The figure shows the structure of the controller 7th in the form of function blocks. These functional blocks include an eye tracker 23 , a needle tracking module 25th , a recording module 27 , a memory 29 , an injection parameter calculation module 31 and a processing module 33 , which processes the information obtained from the other modules in order to generate an overlay image therefrom, which represents the geometrical data relevant for the intravitreal injection.

The control 7th can be a specially designed hardware component, which in particular is also used in the surgical microscope 5 can be integrated. That from the controller 7th The executed control program can then be stored, for example, in a read-only memory of the hardware component. The control 7th but can alternatively also be a computer with control software running on it. In this case, the control is available as a computer program that runs on one with the monitor 9 and the surgical microscope 5 connected computer is running. The instructions of the computer program representing the control program are then generally not stored in a read-only memory, but are loaded into the computer from a non-volatile computer-readable storage medium or are called up from a local network or the Internet.

The control 7th is with the surgical microscope 5 connected to receive image data that is a real-time digital image of the eye 1 and, provided this is already in the field of view of the surgical microscope 5 located, the syringe 3 with the hypodermic needle 19th represent. It also includes the control 7th an input device 35 , via the inputs in the control 7th can be made, or is associated with such. The input device 35 can for example be a keyboard, a touchscreen, a voice input unit, a data interface or any combination of two or more of these elements.

The controller receives in order to derive the relevant geometric data 7th which is the real-time image of the eye 1 possibly with the syringe 3 representing image data and sends them to the eye tracker 23 , to the needle tracking module 25th as well as to the recording module 27 further. From the recording module 27 the recorded image data representing the real-time image can be transferred to the memory 29 be passed on in order to save them for a long time, for example for documentation purposes. In the recording module 27 on the other hand, they are only cached.

The eye tracker 23 recognizes characteristic features of the eye based on the image data representing the real-time image 1 , roughly the position of the pupil 18th and characteristics of the iris and / or the scleral vessels, and in the present exemplary embodiment, on the basis of the recognized features, the viewing direction of the eye 1 and the rotation of the eye 1 determine about its optical axis. From the line of sight and the rotation of the eye 1 the eye-tracker determines around its optical axis 23 then the orientation of the eye 1 which it finally sends to the processing module in the form of data representing the determined orientation 33 passes on. The eye tracker can also 23 In the present exemplary embodiment, include image analysis software that enables it, for example. The edge 22nd the iris 20th to be recognized and based on the recognized edge 22nd the location of the Pars Plana 21 to determine. Also the determined location of the Pars Plana 21 gives the eye tracker 23 finally in the form of data representing the determined position to the processing module 33 further.

The needle tracking module 25th likewise receives the image data representing the real-time image from the surgical microscope 5 . The needle tracking module 25th includes software that makes it possible to use a model of the syringe used 3 retrieved from memory or via the input device 35 can be entered the syringe 3 and especially the hypodermic needle 19th the syringe 3 recognizable in the real-time image. After the needle tracking module 25th the syringe 3 in the real-time images, it determines the Position and orientation of the injection needle 19th in the reference system in which the eye tracker also records the position and orientation of the eye 1 has determined. Information about the determined position and the determined orientation of the injection needle 19th gives the needle tracking module 25th finally in the form of data representing the determined position and the determined orientation to the processing module 33 further.

In the injection parameter calculation module 31 at least one suitable injection angle and a suitable injection depth together with suitable tolerances are determined for the selected injection type, which finally in the form of at least the determined injection angle and the determined injection depth together with the associated tolerances to the processing module 33 be passed on. The injection technique used can be used, for example, via the input device 35 can be entered.

The one from the eye tracker 23 , from the needle tracking module 25th and from the recording module 27 to the processing module 33 Data passed on is continuously updated, so that the processing module 33 receives current data in real time. The processing module also receives 33 also the image data representing the real-time image from the recording module 27 .

The processing module determines from the received data 33 Image data representing an overlay that corresponds to the real-time image of the eye 1 is superimposed and shows the relevant geometric data for the intravitreal injection. In the present example, these image data represent the position of the pars plana as geometric data 21 , a target position 11 where to inject, locations 13th previous injections, the current angle of the injection needle 19th with respect to the surface normal of the eyeball at the target position 11 and the current penetration depth of the injection needle 19th in the eyeball. The processing module also compares 33 the one from the needle tracking module 25th determined current angle of the injection needle 19th with that of the injection parameter calculation module 31 determined suitable injection angle and the associated tolerance to change the overlay image if the current angle of the injection needle 19th is within the tolerance for the appropriate injection angle. The change in the overlay image can take place, for example, in such a way that the in the information field 15th indicated angle is displayed in a different color than before, as described above. Additionally or alternatively, the overlay image can also be changed in such a way that the display of the syringe in the real-time image is colored green, for example, if the current angle of the injection needle 19th is within the tolerance for the appropriate injection angle. A person skilled in the art can also find further display options that are suitable for indicating that the current angle of the injection needle 19th is within the tolerance for the appropriate injection angle. In the same way, the overlay image can also be changed if the injection needle is the appropriate injection depth 19th is reached within the tolerance limits, as has already been described above.

In the present exemplary embodiment, the processing module is 33 the determined target position together with a patient ID to the memory 29 from where the current target position 11 is saved. When the processing module 33 then a new target position for the same patient 11 is determined, the target positions previously determined for him are based on his patient ID from the memory 29 retrieved and compared with the newly determined target position. The saved positions 13th provide those positions 13th where previously intravitreal injections, i.e. punctures in the eye 1 have taken place and should be avoided with the new injection. Optionally, the positions saved in the memory 13th Also include positions where there are punctures in the eye 1 have been made to insert trocars in previous eye treatments. If the comparison shows that the new target position 11 too close to one of the saved positions 13th the processing module determines 33 a new target position until it is a sufficient distance from the positions previously used for injections 13th Has.

In order to be able to determine the positions, angles and depths described, the corresponding modules of the control system 7th The data provided contain spatial information. This becomes possible when the surgical microscope 5 for example provides stereoscopic image data from which the spatial information can be obtained. However, there is also the possibility of obtaining the spatial information from the image data from more than two cameras that record the eye from different angles. In addition, there are a number of other options for obtaining the spatial image data. The following methods are listed here as examples: 3D scanning, line scan photogrammetry, photogrammetry by pattern projection, confocal tomographic imaging, chromatic confocal imaging, multi-wavelength interferometry, optical coherence tomography, depth of focus scanning, moiré deflectometry, phase measurement deflectometry, differential interference contrast mapping. In the present exemplary embodiment, the spatial data are obtained on the basis stereoscopic images or based on pattern projection.

Possible variants for the construction of a surgical microscope 5 How it can be used in the described exemplary embodiment of the assistance device are described below with reference to FIG 3 and 4th described.

This in 3 Operating microscope shown 5 comprises, as essential components, a lens to be turned towards an object field 105 , which can be designed in particular as an achromatic or apochromatic objective. In the present case, the object field is the surface of an eye 1 . This in 3 lens shown 105 consists of two partial lenses cemented together to form an achromatic objective.

The object field is in the focal plane of the lens 105 arranged so that it is away from the lens 105 is mapped to infinity. In other words, a divergent bundle of rays 107A, 107B emanating from the object field is generated as it passes through the objective 105 into a parallel bundle of rays 109A , 109B transformed.

Observer side of the lens 105 is a magnification changer 111 arranged, which can be designed either as a zoom system for stepless change of the magnification factor or as a so-called Galilean changer for step-by-step change of the magnification factor, as in the exemplary embodiment shown. In a zoom system, which is composed, for example, of a lens combination with three lenses, the two lenses on the object side can be shifted in order to vary the magnification factor. In fact, however, the zoom system can also have more than three lenses, for example four or more lenses, the outer lenses then also being able to be arranged in a fixed manner. In a Galileo changer, on the other hand, there are several fixed lens combinations that represent different magnification factors and can be alternately introduced into the beam path. Both a zoom system and a Galilean changer convert a parallel beam on the object side into a parallel beam on the observer side with a different beam diameter. The magnification changer 111 is already part of the binocular beam path of the surgical microscope in the present exemplary embodiment 5 , ie it has its own combination of lenses for each stereoscopic partial beam path 109A , 109B of the surgical microscope 5 on. Setting a magnification factor using the magnification changer 111 takes place in the present exemplary embodiment via a motor-driven actuator, which together with the magnification changer 111 Part of a magnification change unit for setting the magnification factor.

To the magnification changer 111 an interface arrangement closes on the observer side 113A , 113B via the external devices to the surgical microscope 11 can be connected and the beam splitter prisms in the present exemplary embodiment 115A , 115B includes. In principle, however, other types of beam splitters can also be used, for example partially transparent mirrors. The interface arrangement 113A , 113B serve in the present exemplary embodiment for decoupling a beam from the beam path of the surgical microscope 11 (Beam splitter prism 115B ) or for coupling a beam into the beam path of the surgical microscope 11 (Beam splitter prism 115A ).

The beam splitter prism 115A in the partial beam path 109A serves in the present exemplary embodiment with the aid of a display 37, for example a digital mirror device (DMD) or an LCD display, and associated optics 139 via the beam splitter prism 115A Information or data for a viewer in the partial beam path 109A of the surgical microscope 5 to reflect. In particular, superimposed images can be mirrored in by the control 7th a real-time image of the eye 1 be derived. In the other partial beam path 109B is at the interface 113B a camera adapter 119 arranged with an attached digital color camera 103, which with at least one digital image sensor 123 , for example. Is equipped with at least one CCD sensor or at least one CMOS sensor. A digital image of the object field can be obtained by means of the digital color camera 103 1 be included. Spatial information that is required to derive the geometric data relevant for the eye treatment can be obtained from the image recorded with the color camera 103 if, for example, a stripe pattern is projected onto the eye during the recording. The three-dimensional image information can then be obtained from distortions of the stripe pattern in the recorded image. For example, a striped pattern can be projected for every second image. For the generation of the real-time image, only those images are used that were not projected onto the eye when the stripe pattern was recorded.

To the interface arrangement 113A , 113B a binocular tube closes on the observer side 127 at. This has two tube lenses 129A , 129B on which the respective parallel bundle of rays 109A , 109B on intermediate image levels 131A , 131 B focus, i.e. the object field 1 on the respective intermediate image level 131A , 131 B depict. The ones in the intermediate image levels 131A , 131B Any intermediate images located are finally provided by eyepiece lenses 135A , 135B again mapped to infinity, so that a viewer can look at the intermediate image with a relaxed eye. In addition, it is done in the binocular tube by means of a mirror system or prisms 133A , 133B an increase in the distance between the two partial beams 109A , 109B to adjust it to the eye relief of the observer. With the mirror system or the prisms 133A , 133B The image is also straightened.

The surgical microscope 5 is also equipped with a lighting device with which the object field 1 can be illuminated with broadband illumination light. For this purpose, the lighting device in the present exemplary embodiment has a white light source 141 , such as a halogen lamp or a gas discharge lamp. That from the white light source 141 outgoing light is via a deflecting mirror 143 or a deflecting prism in the direction of the object field 1 steered to illuminate this. In the lighting device there is also an optical lighting system 145 available, for uniform illumination of the entire observed object field 1 cares. In addition, the lighting device can comprise a projector for projecting a stripe pattern onto the eye.

It should be noted that the in 3 Illumination beam path shown is highly schematic and does not necessarily reflect the actual course of the illumination beam path. In principle, the illumination beam path can be designed as so-called oblique illumination, which is shown in the schematic illustration in FIG 3 comes closest. In such an oblique illumination, the beam path runs at a relatively large angle (6 ° or more) to the optical axis of the objective 105 and can, as in 3 shown completely outside the lens 105 run away. Alternatively, however, there is also the possibility of the illumination beam path of the oblique illumination through an edge region of the objective 105 to run through. Another possibility for arranging the illumination beam path is the so-called 0 ° illumination, in which the illumination beam path passes through the lens 105 runs through and between the two partial beam paths 109A , 109B , along the optical axis of the lens 105 towards the object field 1 into the lens 105 is coupled. Finally, there is also the possibility of designing the illumination beam path as what is known as coaxial illumination, in which a first and a second partial illumination beam path are present. The partial beam paths are parallel to the optical axes of the observation partial beam paths via one or more beam splitters 109A , 109B into the surgical microscope 11 coupled in so that the illumination runs coaxially to the two observation beam paths.

In the in 3 shown variant of the surgical microscope 5 is the lens 105 only from an achromatic lens. However, an objective lens system made up of several lenses can also be used, in particular a so-called varifocal objective, with which the working distance of the surgical microscope can be adjusted 5 , ie the distance between the object-side focal plane and the vertex of the first object-side lens surface of the objective 105 , also called the object focal length, can be varied. The object field arranged in the focal plane is also produced by a varifocal lens 1 Imaged to infinity, so that there is a parallel bundle of rays on the observer side.

4th shows an example of a purely digital surgical microscope 5 ' in a schematic representation. The main objective of this surgical microscope is different 105 , the magnification changer 111 as well as the lighting system 141 , 143 , 145 not from the in 3 operating microscope shown 5 with optical insight. The difference is that in 4th Operating microscope shown 5 ' does not include an optical binocular tube. Instead of tube lenses 129A , 129B out 3 includes the surgical microscope 5 ' out 4th Focusing lenses 149A, 149B with which the binocular observation beam paths 109A , 109B on digital image sensors 161A , 161B can be mapped. The digital image sensors 161A , 161B can be, for example, CCD sensors or CMOS sensors. The ones from the image sensors 161A , 161B Captured images are sent digitally to digital displays 163A , 163B sent, which can be designed as LED displays, as LCD displays or as displays based on organic light emitting diodes (OLEDs). The displays 163A , 163B can, as in the present example, ocular lenses 165A , 165B associated with those on the displays 163A , 163B displayed images are mapped to infinity, so that a viewer can look at them with relaxed eyes. The displays 163A , 163B and the eyepiece lenses 165A , 165B can be part of a digital binocular tube, but they can also be part of a head-mounted display (HMD) such as data glasses. Additionally or alternatively, those with the digital image sensors 161A , 161B recorded images also on a monitor 9 as shown in 1 is shown. From the ones with the digital image sensors 161A , 161B The spatial information required to determine the geometric data relevant for eye treatment can be derived from recorded stereoscopic partial images.

An exemplary embodiment for the computer-implemented method according to the invention for assisting in an eye treatment is described below with reference to the in 5 The flowchart shown is described. In the present exemplary embodiment, the method is carried out by the controller 7th the in the 1 and 2 and the eye treatment is an intravitreal injection.

At the beginning of the procedure in step S1 receives the control 7th from the surgical microscope 5 electronic image data, which is a real-time image of the eye 1 and when it is in the field of view of the surgical microscope 5 located, also from the syringe 3 contains. The received image data are then used in step S2 evaluated in order to derive relevant geometric data for the intravitreal injection from the image data. These geometric data can, for example, be the position of the Pars Plana 21 , a current target position 11 where the injection is to be made, an angle of the injection needle 19th relative to the surface normal of the eyeball at the target position 13th and a penetration depth of the injection needle 19th the syringe 3 include. If the geometric data also includes the positions 13th previous punctures in the eye 1 are to be included in one step S1 ' the corresponding positions from memory 29 the control 7th retrieved. In addition, the geometric data can also contain information as to whether the angle of the injection needle 19th corresponds to an angle suitable for intravitreal injection and / or whether the injection depth corresponds to a depth suitable for intravitreal injection.

In step S3 an overlay image is then generated in the form of image data that is derived from the real-time image of the eye 1 Represent derived geometric data, and of information that the correct position and orientation superimposition of the geometric data on the real-time image of the eye 1 enable. This image data is eventually stepped into S4 superimposed on the real-time image and in step S5 for display on the monitor 9 or any other than the display of the surgical microscope 5 Serving display provided.

The present invention has been described in detail on the basis of exemplary embodiments for explanatory purposes. However, a person skilled in the art recognizes that it is possible to deviate from the exemplary embodiments. For example, in the exemplary embodiment, the real-time images with the superimposed geometric information were created on one with the surgical microscope 5 connected monitor 9 displayed. However, you can also use the internal displays of the surgical microscope 5 or head-mounted displays worn by the treating staff. In addition, a numerical indication of the injection angle and / or the injection depth can be dispensed with, provided that the representation in the real-time image enables the person performing the injection to recognize otherwise if the injection angle corresponds to the appropriate injection angle within the tolerance limits and / or the injection depth within corresponds to the tolerance limits of the suitable injection depth, for example if a target vector is displayed with which the injection needle is to be brought into congruence. There is also the possibility of generating an instruction data set that includes the geometric data relevant for the intravitreal injection and of outputting this to a robotic injection system so that the intravitreal injection can be carried out in an automated manner. It is then not necessary to generate an image data set representing the geometric data relevant for the intravitreal injection and to generate information that enables the image data set to be superimposed on the image data of the eye and to provide the image data set and the information for superimposing the image data of the eye. The generation and provision of the image data set and the information on the superimposition can, however, take place in order to make it easier for a person to monitor the automated injection.

List of reference symbols

1

eye

3

syringe

5, 5 '

Surgical microscope

7th

control

9

monitor

11

Target position

13th

previous position

15th

Information field

17th

Information field

18th

pupil

19th

needle

20th

iris

21

Pars Plana

22nd

contour

23

Eye tracker

25th

Needle tracking module

27

Recording module

29

Storage

31

Injection parameter calculation module

33

Processing module

35

Input device

105

lens

107 A, B

divergent bundle of rays

109 A, B

parallel bundle of rays

111

Magnification changer

113A, B

Interface arrangement

115A, B

Beam splitter prism

119

Camera adapter

123

Image sensor

127

Binocular tube

129 A, B

Tube lens

131 A, B

Intermediate image plane

133 A, B

prism

135 A, B

Eyepiece lens

137

Display

139

optics

141

White light source

143

Deflection mirror

145

Lighting optics

149 A, B

Focusing lens

161 A, B

Image sensor

163 A, B

Display

165 A, B

Eyepiece lens

167 A, B

electric wire

S1

Receipt of image data

S1 '

Calling up previous positions

S2

Evaluate image data

S3

Generate image data and information for overlay

S4

Overlaying the real-time image with the image data

S5

Making available for viewing on a display

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• US 2018/0360654 A1 [0005]
• US 2017/0266045 A1 [0005]
• US 2010/0152646 A1 [0005]
• US 2017/0135768 A1 [0005]
• CN 110384581 A [0005]

Claims (19)

Assistance device for assisting with an intravitreal injection, comprising: - An optical observation device (5) which is designed to record image data of an eye (1); - A controller (7) which is designed to receive the image data from the optical observation device (5) and to evaluate the received image data in order to derive relevant geometric data for the intravitreal injection from the image data, - The optical observation device (5) being designed to display a real-time image of the eye (1) on which the geometrical data relevant to the intravitreal injection derived from the control are superimposed. Assistance device according to Claim 1 , in which the relevant geometric data for the intravitreal injection include a target position (11) at which the injection needle (19) for an intravitreal injection is to be placed. Assistance device according to one of the preceding claims, which also comprises a memory (29) for storing positions of previous punctures in the eye (1) and in which the controller (7) is configured to read the geometric data relevant for the intravitreal injection from the image data taking into account the positions of previous punctures. Assistance device according to Claim 2 and Claim 3 , in which the control (7) is designed to derive the target position (11) taking into account the positions of previous punctures. Assistance device according to Claim 3 , in which the controller (7) is designed to derive those current positions in the eye (1) that correspond to the positions of previous punctures stored in the memory (29) as the geometric data relevant for the intravitreal injection. Assistance device according to one of the preceding claims, in which the controller (7) is designed to use the image data as the geometrical data relevant for the intravitreal injection at least geometrical data for the injection depth and the injection angle of an injection needle (19) with which an intravitreal injection should take place to derive. Assistance device according to one of the preceding claims, in which the control comprises an eye tracker (23) for determining the position and the orientation of the eye (1). Assistance device according to Claim 6 , in which the controller (7) comprises a needle tracking module (25) for determining the position and the orientation of the injection needle (19). Computer-implemented method for assisting with an intravitreal injection, with the following steps: - Receiving image data of an eye (1); Evaluation of the image data in order to derive relevant geometric data for the intravitreal injection from the image data. Computer-implemented method according to Claim 9 , with the further steps: - Generating an image data set representing the geometric data relevant for the intravitreal injection and information that enables the image data set to be superimposed on the image data of the eye (1) and - Providing the image data set and the information for superimposing the image data of the eye (1). Computer-implemented method according to Claim 9 or Claim 10 , with the further steps: generating an instruction data set comprising the geometric data relevant for the intravitreal injection and outputting the instruction data set to a robotic injection system. Computer-implemented method according to one of the Claims 9 to 11 , in which the geometric data relevant to the intravitreal injection include a target position (11) at which the injection needle (19) for an intravitreal injection is to be placed. Computer-implemented method according to one of the Claims 9 to 12th , in which the stored positions of previous punctures in the eye (1) are retrieved from a memory (29) and the geometric data relevant for the intravitreal injection are derived from the image data, taking into account the positions of previous punctures. Computer-implemented method according to Claim 12 and Claim 13 , in which the target position (11) is derived taking into account the positions of previous grooves. Computer-implemented method according to Claim 13 , in which those current positions in the eye (1) which correspond to the positions of previous punctures stored in the memory (29) are derived as the geometric data relevant for the intravitreal injection. Computer-implemented method according to one of the Claims 9 to 15th , in which at least geometric data for the injection depth and the injection angle of an injection needle (19) with which an intravitreal injection is to be made are derived from the image data as the geometric data relevant for the intravitreal injection. Computer-implemented method according to Claim 16 , in which the orientation and position of the injection needle (19) in relation to the eye (1) are determined from the image data by means of an image analysis and the geometric data indicate the deviation of the injection needle (19) from the injection depth and the injection angle. Computer program for assisting with an intravitreal injection, comprising instructions which, when executed on a computer, cause the computer to: - to receive image data of an eye (1), - evaluate the image data in order to derive relevant geometric data for the intravitreal injection from the image data, - an image data set representing the geometric data relevant for the intravitreal injection and information which make it possible to superimpose the image data set on the image data of the eye (1), or to generate an instruction data set comprising the geometric data relevant for the intravitreal injection, and - to provide the image data set and the information for superimposing the image data of the eye (1) or the instruction data set. Non-transitory computer-readable storage medium having stored thereon instructions for assisting with an intravitreal injection, the instructions, when executed on a computer, causing the computer to - to receive image data of an eye (1), - evaluate the image data in order to derive relevant geometric data for the intravitreal injection from the image data, - an image data set representing the geometric data relevant for the intravitreal injection and information which make it possible to superimpose the image data set on the image data of the eye (1), or to generate an instruction data set comprising the geometric data relevant for the intravitreal injection, and - to provide the image data set and the information for superimposing the image data of the eye (1) or the instruction data set.

Images