DE102016217628A1 - Movement of a surgical microscopy system - Google Patents

Abstract

The invention relates to a method for operating a surgical microscopy system (1), comprising: a microscope optics (21) which images an object area for consideration during an operation, a movement device (18) which transmits the microscope optics (21) moved such that a position of the object area is changed, - at least one image recording device (23, 24), (a) which is either connected to the microscope optics (21) and moves with movement of the microscope optics (21), wherein the image recording device (23, 24) detects a detection structure (28, 29) arranged separately from the microscope optics (21), which is not moved when the microscope optics (21) move, by at least one recording image, (b) or is arranged separately from the microscope optics (21), is not moved during movement of the microscope optics (21) and a detection structure (28, 29) through the microscope optics (21) and / or by at least one fe St formed with the microscope optics (21) part is formed and / or on the microscope optics (21) and / or on which at least one fixed to the microscope optics (21) connected part, detected by at least one recording image wherein, using information about an actual appearance of the detection structure (28, 29) from differences in the at least one recording image to the actual appearance of the detection structure (28, 29), a speed of relative movement of the image recording device (23, 24) and the detection structure (28 , 29) is determined.

Description

The invention relates to a method of operating a surgical microscopy system, a motion sensing system for a surgical microscopy system, and a surgical microscopy system having the motion sensing system. The surgical microscopy system has a microscope optic which is designed to image an object area for consideration during an operation. Specifically, the object area includes the area a surgeon observes during a patient's operation. By means of a movement device, the microscope optics can be moved in such a way that a position of the object area is changed. For example, the object area shifts thereby transversely to the observation direction of the microscope optics.

In particular, microsurgical procedures on the human body are often performed by a surgeon using a surgical microscopy system. Such a surgical microscopy system typically has a microscope optics to detect the object area to be imaged. In some surgical microscopy systems, the surgeon can use oculars to perceive the imaged subject area with the eye. Alternatively or additionally, an image detector (for example with at least one matrix of photocells) can be present, which records an image of the object region. This captured image may then in turn be displayed to the surgeon perceptibly, for example on a screen, via a display device worn on the surgeon's head, or on a display device carried on a tripod or other carrier, which in turn may have eyepieces.

In many cases, it is desired by the surgeon that the object area to be imaged does not change its position during surgery once it has been adjusted by the surgeon. However, environmental influences, such as Vibrations occur due to the occurrence of the floor of the operating room. In order to keep the imaged object area constant with respect to its position, a position control can be used, which, however, relies on actual information about the actual position of the object area or the microscope optics.

In particular, surgical microscopy systems may cooperate with a navigation system of an operating room designed to detect positions of various objects in the operating room. Such a navigation system may, for example, comprise a plurality of cameras, which are arranged at a distance from one another in the operating theater, in order to detect markings which are attached to the objects and to calculate their positions and orientations in a coordinate system of the navigation system. Such a marking has, for example, three retroreflectors spaced apart from one another and not arranged on a straight line. Such markers may be attached to a patient, a tool used in the procedure and / or the microscope optics to detect their positions in the coordinate system of the operating room. These sensed positions may be used to superimpose anatomical structures of the patient imaged by the surgical microscopy system with previously acquired images of those structures, such as images obtained by computed tomography. Thereby, e.g. attempt to ensure that any manipulation performed on the patient during the procedure is at a desired location determined in pre-surgery examinations.

In practice, it may occur in such navigation systems that people involved in the surgical procedure obscure the line of sight between the cameras of the navigation system and a mark attached to the object whose position is to be determined so that it is temporarily impossible to achieve the desired ones To gain positional information. Furthermore, the position determination depends on the operating conditions, such. B. temperature and humidity. This requires a calibration.

The calibration of the position determination device can at least be simplified if an optical position determination takes place by taking camera images through at least one camera. By evaluating the camera images, the position of the movable part (at least the microscope optics) can be determined with high accuracy with a suitable configuration and arrangement of the camera or cameras.

Another way to determine the position of the microscope optics is to detect the state of motion of the movement device by integrated into the wearer of the microscope optics measuring devices. For example, corresponding angular encoders can be integrated in joints of the carrier. However, high-precision motion measuring devices are relatively expensive to manufacture and must be calibrated.

All known positioning methods assume that several camera images are successively recorded to determine the time profile of the movement, the relative position of moving part (here: microscope optics) and base is determined at different times from the camera images and taking into account the time interval between the times at which the various camera images were recorded, the speed or acceleration is determined. In the case of motion control or regulation of the movement, however, this leads to delays between the recording of the camera images and the time at which the information about the time course of the movement for further processing by the controller or control is available.

In addition to the movement of the microscope optics by a e.g. multi-drive automatic movement means, the surgeon may have the ability to adjust or adjust the position and / or orientation of the microscope optics, if the microscope optics should be an obstacle during the operation. In particular, to prevent the microscope optics from being moved to other obstructive positions, drive control or drive control may resist or prevent movement in predefined obstructive location areas. There are z. B. corresponding constraints deposited, which affect the movement means for moving the microscope optics and in particular affect the control of the drive or the drives of the moving device. In particular, this can ensure that the microscope optics continue to image the object region to be imaged, albeit from a different angle.

Surgical microscopy systems, as already briefly mentioned, often not only offer the possibility of enlarging the object area for the surgeon, but can also provide the surgeon with additional information, in particular by superimposing the additional information with the image or images imaged by the microscope optics of the object area. This is also called augmentation. The additional information may provide the surgeon with e.g. displayed which tissue areas of the patient are to be removed. The correct position and orientation of the microscope optics, which is as completely independent as possible of environmental influences, is therefore in many cases a necessary prerequisite for the success of an operation.

It is an object of the present invention to provide a surgical microscopy system, a motion measurement system for a surgical microscopy system, and a surgical microscopy system in which information about the timing of the movement of the microscope optics with little time delay for further processing by a motion control and / or regulation of movement available stand.

The object is achieved by a motion measuring system with at least one image recording device, which is referred to as a camera in the description, even if the image recording device is not combined with an optical lens device. For example, so-called LSS (Lensless Smart Sensors) are offered for sale, which are suitable as an image recording device. However, there are also embodiments of the motion measuring system, in which the microscope optics is used for imaging.

• a) entweder mit der Mikroskop-Optik verbunden ist und bei Bewegung der Mikroskop-Optik mitbewegt wird, wobei die Bildaufzeichnungseinrichtung zumindest eine separat von der Mikroskop-Optik angeordnete Erfassungsstruktur, die bei Bewegung der Mikroskop-Optik nicht mitbewegt wird, durch zumindest ein Aufzeichnungsbild erfasst,
• b) oder separat von der Mikroskop-Optik angeordnet ist, bei Bewegung der Mikroskop-Optik nicht mitbewegt wird und eine Erfassungsstruktur, die durch die Mikroskop-Optik und/oder durch zumindest ein fest mit der Mikroskop-Optik verbundenes Teil gebildet ist und/oder an der Mikroskop-Optik und/oder an dem zumindest einen fest mit der Mikroskop-Optik verbundenen Teil angeordnet ist, durch zumindest ein Aufzeichnungsbild erfasst.

In addition to the at least one image recording device, there is at least one detection structure which is detected by the image recording device or a plurality of the image recording devices. For the image recording device or, in the case of a plurality of the image recording devices, it is the case that the image recording device during operation

• a) is either connected to the microscope optics and is moved with movement of the microscope optics, wherein the image recording device detects at least one separately arranged from the microscope optics detection structure, which is not moved during movement of the microscope optics, by at least one recording image .
• b) or is arranged separately from the microscope optics, is not moved during movement of the microscope optics and a detection structure which is formed by the microscope optics and / or at least one fixedly connected to the microscope optics part and / or is arranged on the microscope optics and / or on which at least one part fixedly connected to the microscope optics is detected by at least one recording image.

In the case of a plurality of image recording devices, it is possible that at least one image recording device according to case a) and at least one other image recording device according to case b) is used or provided. However, it is also possible to use or provide all the image recording devices of the motion measuring system according to case a) or according to case b).

The at least one image recording device takes at least one recording image (hereinafter also referred to as camera image without limiting the generality) of at least one detection structure or is designed accordingly.

A recording image is therefore understood to be information about a mapped scene generated by the image recording device, wherein image values (for example binary values, gray values or color values) are defined with respect to an image coordinate system. Although preferred, the recording image need not be the image of an optical lens or lens array. Rather, other types of images come into question, for example by means of grids or masks. In particular, however, the type of mapping is taken into account in the evaluation of the recording image.

In a relative movement of the image recording devices and the scene occurs a well-known in the photographing of moving objects problem, namely the smearing of structures in the recorded camera image. For example, a sharp transition (edge) from a bright, white, or highly reflective texture region to a dark, black, or weakly reflective texture region is blurred to a gradual transition from light, white, or highly reflective to dark, black, or low reflective in the camera image. The same applies to sharp transitions (edges) between structural areas of different color or color depth. In the present invention, the detection structure (not only in the case of the presence of at least one edge) may reflect radiation and / or even generate radiation (e.g., a field of light emitting diodes).

The structure detected in the camera image therefore already differs in the case of the relative movement from the actual appearance of the structure. This is due to the fact that during the time interval of the structure detection (often referred to as exposure time) by the camera, the structure has moved relative to the camera and therefore a shift of the image of the structure has taken place. In other words, the smudges are caused by a radiation distribution of the radiation detected by the camera during the recording time interval. Therefore, the recording image can be taken during a time interval of detection of the detection structure, and in case of relative movement of the image recording device and the detection structure, an actual appearance of the detection structure due to the relative movement in the recording image changes.

The invention takes advantage of these otherwise undesirable differences. Using information about an actual appearance of the detection structure, a speed of relative movement of the image recording devices and the detection structure is determined from differences of the at least one recording image to the actual appearance of the detection structure. An essential advantage of the invention is that the speed of the relative movement can already be determined from a single recording image. However, this does not exclude the use of a plurality of recording images, which were taken simultaneously, temporally overlapping and / or successively, in order to determine the speed. In contrast to a determination of the speed from position determinations from successively recorded recorded images, however, the invention makes it possible in each case to determine the speed with a lesser time delay after recording the image (s).

Of advantage is the small time delay in the detection of the movement, especially when the surgical microscope system is combined with a movement device for moving the microscope optics with at least one drive motor and a motion control or movement control. The scope of the invention therefore also includes such a control or regulation for a surgical microscopy system, which is connected via a signal input to the motion measuring system which determines the speed of movement from at least one recording image, in particular repeatedly from at least one recording image of a temporal sequence of Recorded images. Especially in the compensation of environmental influences such as mechanical vibrations, which affect the wearer of the microscope optics, a small time delay of determining the speed after recording the recording image of great advantage. Corresponding regulations can be carried out all the more stably, the shorter the time delay is.

The microscope slide carriers of known surgical microscopy systems can have projecting parts (eg arms) with a length in the range of, for example, 1 to 2 m. Alternatively or additionally, the carrier may have a plurality of degrees of freedom of movement for the positioning and / or alignment of the microscope optics carried by it. For example, so-called 6-axis kinematics are known. For example, a corresponding support arm of the carrier is fixed at one end to the base of the surgical microscopy system (eg the Base of the operating room) and carries at the opposite end of the microscope optics, which for example has a mass of a few kilograms. In terms of design and also in terms of motion control, this places great demands, since, on the one hand, a stable, vibration-resistant design is desirable and, on the other hand, the carrier should be made as narrow as possible in the area near the patient so as not to disturb the surgeons and assistants during the operation. A fast-acting control makes it possible to make the carrier lighter and thus narrower, since the greater susceptibility to vibration due to the lower inertial mass can be compensated by the control of the movement device.

A control requires a very accurate (computer-implemented) model of the moving device and the moving parts and a complex calibration in coupling of set values of the movement with expected actual values (so-called feed-forward control), for example, in order to obtain e.g. From movement sensors and / or position sensors on the moving parts of the carrier to calculate the necessary corrections for a specific position (for example, due to the specific mass distribution or due to environmental influences). In contrast, regulation with feedback of measured actual movement values (feed-back control) can in principle also compensate for environmental influences on the patient pad or even movements of the patient himself by corresponding tracking of the microscope optics.

The above-mentioned movement control is therefore preferably carried out as a feed-back control and takes into account as movement actual value the speed of the relative movement determined from the at least one recording image. If e.g. the detection structure is arranged on the patient, the at least one camera connected to the microscope optics can detect a relative movement due to a movement of the patient as information in the respective recording image and the detection device of the motion measuring system therefrom determine the speed of movement, which is then transmitted to the motion control , But even with the arrangement of the at least one camera on a patient pad and arrangement and / or configuration of the detection structure on the microscope optics, a relative movement due to the movement of the patient documents can be detected and corrected.

In general, a refinement of the invention is thus proposed in which the determined speed of the relative movement is used to minimize and / or compensate for the relative movement.

The detection structure may be a radiation-reflecting and / or radiation-emitting structure, for example by a structure formed by a luminous image content of a display (for example with an LED matrix). The display can be arranged on the moving part (to which the microscope optics belongs) or, in another case, on a fixed part (base) of the surgical microscope system. In the latter case, the camera is moved with the moving part. The base may be any part of the surgical microscopy system, such as the part of the microscope optics support not moving with the microscope optics, a pad on which the patient is positioned during the operation, and / or the patient himself ,

A radiation emitting structure may optionally be e.g. a non-self-generating radiation substructure in front of a radiation emitting rear substructure. The front substructure thus generates a locally non-constant radiation distribution, since it allows radiation to pass in a location-dependent manner. Such a radiation-emitting structure, but also displays without a non-self-generating radiation component structure in the direction of the camera are particularly well suited for the determination of the orientation of the relative movement of image recording devices and base described in more detail below.

Regardless of the origin of the radiation emanating from the detection structure and / or being modulated, reflected and / or deflected by the detection structure, the detection structure may in particular be a composite structure, ie different areas of the imaged scene together form the sensing structure. For example, several markers together may form the detection structure. It is also not necessary for the same composite structure to always be recorded in different successively recorded recording images.

It is possible to reduce the degree of smearing by shortening the time interval of the image pickup and / or illuminating the structure to be detected with very short radiation pulses. However, it is important to ensure sufficient illumination to the signal-to-noise ratio of the captured by the camera image information is not too low. Short radiation pulses represent an additional expense for the lighting and lead in some operating situations to disruptions of the operation or Irritation of the persons who are in the area of the surgical microscopy system during operation.

In mathematical generalization, the solution is based on the following principle. For physical reasons, no measuring system with infinite bandwidths can be realized, but every information processing system has a certain finite transfer function. This describes how information present at the input is changed or output at the output of the system because it can not be transmitted or processed in its full bandwidth. For optical systems, there are several ways in which the transfer function of the system is described. The choice of description is determined largely by the question to be answered. For corresponding considerations, which will be explained in more detail with reference to the figures, the argumentation of the optical transfer function of an optical system is used. In particular, it can be understood as a convolution kernel of a mathematical convolution and describes, for example, how contrast in the object is converted into contrast in the image. The application of the inverse optical transfer function (as an example, the mathematical folding is described in more detail) thus allows the reconstruction of the object content from the image content. However, this applies only to those components of a frequency analysis (for example Fourier analysis) of the object which have been transmitted by the system and / or for which an inversion of the optical transfer function is possible. In other words, the mapping has to be bijective at least with respect to a sufficient information content for the desired purposes and / or allow a sufficiently low-loss reconstruction.

It is proposed to evaluate the effects of the relative movement of the image recording device and the detected system part, namely in particular the smearing of the structures in the recording image, and to determine the speed of the relative movement therefrom. It is therefore not necessary to shorten the exposure time as far as possible and / or to illuminate with radiation pulses.

• – eine Mikroskop-Optik, welche dazu ausgestaltet ist, einen Objektbereich zur Berücksichtigung bei einer Operation abzubilden,
• – eine Bewegungseinrichtung, welche dazu ausgestaltet ist, die Mikroskop-Optik derart zu bewegen, dass eine Position des Objektbereichs verändert wird,
• – zumindest eine Bildaufzeichnungseinrichtung, (a) die entweder mit der Mikroskop-Optik verbunden ist und bei Bewegung der Mikroskop-Optik mitbewegt wird, wobei die Bildaufzeichnungseinrichtung eine separat von der Mikroskop-Optik angeordnete Erfassungsstruktur, die bei Bewegung der Mikroskop-Optik nicht mitbewegt wird, durch zumindest ein Aufzeichnungsbild erfasst, (b) oder die separat von der Mikroskop-Optik angeordnet ist, bei Bewegung der Mikroskop-Optik nicht mitbewegt wird und eine Erfassungsstruktur, die durch die Mikroskop-Optik und/oder durch zumindest ein fest mit der Mikroskop-Optik verbundenes Teil gebildet ist und/oder an der Mikroskop-Optik und/oder an dem zumindest einen fest mit der Mikroskop-Optik verbundenen Teil angeordnet ist, durch zumindest ein Aufzeichnungsbild erfasst,

wobei unter Verwendung von Informationen über eine tatsächliche Erscheinung der Erfassungsstruktur aus Unterschieden des zumindest einen Aufzeichnungsbildes zu der tatsächlichen Erscheinung der Erfassungsstruktur eine Geschwindigkeit einer Relativbewegung der Bildaufzeichnungseinrichtung und der Erfassungsstruktur ermittelt wird. In particular, a method for operating a surgical microscopy system is proposed, wherein the surgical microscopy system has:

• A microscope optics, which is designed to image an object area for consideration in an operation,
• A movement device which is designed to move the microscope optics such that a position of the object region is changed,
• - At least one image recording device, (a) which is either connected to the microscope optics and is moved with movement of the microscope optics, wherein the image recording device arranged separately from the microscope optics detection structure, which is not moved when moving the microscope optics , detected by at least one recording image, (b) or which is arranged separately from the microscope optics, is not moved during movement of the microscope optics, and a detection structure that passes through the microscope optics and / or through at least one fixed with the microscope Optically connected part is formed and / or on the microscope optics and / or on the at least one part fixedly connected to the microscope optics part is detected by at least one recording image,

wherein, using information about an actual appearance of the detection structure from differences in the at least one recording image to the actual appearance of the detection structure, a speed of relative movement of the image recording device and the detection structure is determined.

• – zumindest eine Bildaufzeichnungseinrichtung, (a) die während eines Betriebes entweder mit der Mikroskop-Optik verbunden ist und bei Bewegung der Mikroskop-Optik mitbewegt wird, wobei die Bildaufzeichnungseinrichtung eine separat von der Mikroskop-Optik angeordnete Erfassungsstruktur, die bei Bewegung der Mikroskop-Optik nicht mitbewegt wird, durch zumindest ein Aufzeichnungsbild erfasst, (b) oder die während eines Betriebes separat von der Mikroskop-Optik angeordnet ist, bei Bewegung der Mikroskop-Optik nicht mitbewegt wird und eine Erfassungsstruktur, die durch die Mikroskop-Optik und/oder durch zumindest ein fest mit der Mikroskop-Optik verbundenes Teil gebildet ist und/oder an der Mikroskop-Optik und/oder an dem zumindest einen fest mit der Mikroskop-Optik verbundenen Teil angeordnet ist, durch zumindest ein Aufzeichnungsbild erfasst,
• – eine Ermittlungseinrichtung, die ausgestaltet ist, unter Verwendung von Informationen über eine tatsächliche Erscheinung der Erfassungsstruktur aus Unterschieden des zumindest einen Aufzeichnungsbildes zu der tatsächlichen Erscheinung der Erfassungsstruktur eine Geschwindigkeit einer Relativbewegung der Bildaufzeichnungseinrichtung und der Erfassungsstruktur zu ermitteln.

It is further proposed: A motion measuring system for a surgical microscope system with a microscope optical system, which is designed to image an object area for consideration during an operation, wherein the motion measuring system comprises:

• - At least one image recording device, (a) which is connected during operation either with the microscope optics and is moved with movement of the microscope optics, wherein the image recording device arranged separately from the microscope optics detection structure, the movement of the microscope optics is not moved, detected by at least one recording image, (b) or which is arranged separately from the microscope optics during operation, is not moved when moving the microscope optics and a detection structure, by the microscope optics and / or at least one part fixedly connected to the microscope optics is formed and / or is arranged on the microscope optics and / or on which at least one part fixedly connected to the microscope optics is detected by at least one recording image,
• A detection means configured to determine a speed of relative movement of the image recording means and the detection structure by using information about an actual appearance of the detection structure from differences of the at least one recorded image to the actual appearance of the detection structure.

Further, within the scope of the invention is a surgical microscopy system comprising:

• The motion measuring system in one of the embodiments described in this specification,
• The microscope optics, which is designed to image an object area for consideration in an operation,
• - A movement device which is adapted to move the microscope optics such that a position of the object area is changed.

As mentioned elsewhere in this description, the detection structure may in particular reflect and / or emit radiation. Therefore, the production of a detection structure also includes the setting of a variably adjustable detection structure, such as a display. By adjusting the emanating from the detection structure, is transmitted to the at least one camera transmitting radiation. Alternatively or additionally, the dimensions of the detection structure may be changed over time, for example by irradiation with a radiation distribution which changes over time, for example whose wavelengths (colors) change. For example, depending on the impinging radiation distribution (eg, color distribution), the detection structure may have different dimensions of structural features of the detection structure that can be detected by the camera. An easy way to change the detection structure is the use of at least one display whose displayed content is changed. In particular, additional information can be obtained by the camera image if the camera detects detectable structural features and in particular dimensions of the detection structure during the time interval over which image information for the camera image is recorded.

In particular, a detection structure may be arranged on a patient on whom surgery is to be performed, e.g. as an arrangement of a plurality of markers. In particular, the detection structure can be arranged such that it is detectable not only for at least one image recording device connected to the microscope optics, but also for the surgeon in a predetermined relative position to the patient. Further preferably, the detection structure may already be arranged on the patient during an image acquisition of the patient preceding the operation. Both the preceding image acquisition, which is optionally followed by a tomographic reconstruction to form a three-dimensional image data set of the patient, and the image acquisition by the at least one image recording device during the operation can therefore refer to the same arrangement of the detection structure on the patient. For example, Therefore, a coordinate system can be defined / defined with respect to the patient or with respect to the detection structure on the patient.

Optionally, the detection structure on the patient can be configured such that it can also be detected by a cover of the patient transparent to the at least one image recording device for the radiation which uses the at least one image recording device for recording the recording image. Typically, covers in the form of wipes or foils are used in operations to cover areas of the patient that are outside of the area being operated on. The transparent cover thus allows the detection structure to be located outside the operating area and to cover the area as usual.

When the surgeon can recognize the sensing structure on the patient, the following procedure is preferred: The surgeon moves the microscope optics such that the patient's area to be operated on is imaged as an object area by the microscope optics and, in addition, the at least one image recorder detects the detection structure. Optionally, the surgeon may adjust the at least one imaging device in addition to the movement of the microscope optics.

In the case of several image recording devices, a plurality of detection structures may also be provided, e.g. each one sensing structure associated with an imaging device, or each imaging device may be aligned with any of a plurality of sensing structures.

The surgeon can manually move the microscope optics and optionally additionally the at least one image recording device and / or by actuating the movement device (eg via a joystick or a touch screen as an interaction device). Once both the microscope optics and the at least one image recording device are positioned and / or aligned, the operation of an automatic movement control can be started, which regulates a movement of the microscope optics in the respective desired manner. In this case, the regulation can also be, at least temporarily, that relative movements of the microscope optics and the patient or patient pad or, more generally, the object area to be imaged are prevented and / or compensated. The movement control uses the result of the detection of the at least one detection structure by the at least one image recording device, namely the speed for the motion control determined from the at least one recording image.

The automatic control mode may be started by a signal generated by the surgeon, e.g. by pressing a control, by the surgeon's acoustic command through use of his voice, by a visually detectable gesture of the surgeon or otherwise. The automatic operation can also be started automatically or at a predetermined time. Optionally, the beginning of the automatic control operation may be signaled to the surgeon, i. a signal is issued when the operation starts or starts. After the start of automatic operation, the surgeon may be visualized e.g. in the form of the above-mentioned additional information that a relative movement of the patient and the microscope optics still takes place and optionally which movement takes place. In particular, the relative movement can be characterized by the determined speed and a value corresponding to the height of the speed can be displayed to the surgeon. For example, a representation is a visual representation. However, the representation can also take place in other ways, e.g. by an acoustic signal or the output of speech information.

As already mentioned, at least one image recording device can use the microscope optics as imaging optics in order to capture the object area with the detection structure by at least one recording image. However, it is preferred that at least one image recording device detects at least one recording image without using the microscope optics as imaging optics, from which the speed of the relative movement is determined. This has e.g. the advantage that the image recording device can be oriented to a region other than the object region to be detected by the microscope optics and the detection structure can be located in this other region. Also, with separate imaging optics, e.g. an optics adapted to the purpose of imaging the detection structure may be used.

It is also proposed that the surgical microscopy system, on the one hand, uses radiation of different wavelength ranges for its image-enlarging system function, on the one hand, and the detection of the at least one detection structure, on the other hand. For example, For example, visible light can be used for the system function and radiation in the near infrared range can be used for the acquisition of the detection structure by the at least one recording image. This makes it possible to additionally illuminate in the non-visible infrared range for the purpose of detecting the detection structure, without disturbing the surgeon during the operation.

If for the operation, i. the treatment of the patient, radiation in the wavelength range is used, in which the detection of the detection structure for the purpose of recording the at least one recorded image takes place, the detection of the detection structure can be limited to time intervals in which the radiation used for the operation is not generated. For example, For example, the radiation used for the operation can be generated in each case over short time intervals and in each case at least one recording image can be recorded in intermediate time intervals for the purpose of determining the speed of the relative movement. The motion measuring system may therefore be connected to a control of a radiation source for generating the radiation used in the operation and receive signals from this radiation source or its control, from which a control of the motion measuring system can take the information when undisturbed recordings record images are possible.

As an alternative or in addition to the arrangement of at least one image recording device on the microscope optics, at least one image recording device can be provided separately from the microscope optics, for example on the patient pad or on the patient. Preferred is an arrangement and attachment to the patient pad (eg an operating table) or to an object resting relative to the patient pad. This at least one image recording device can be designed such that it can simultaneously detect both a detection structure arranged on the patient and a detection structure arranged on the microscope optics and / or formed by the microscope optics. The order at the microscope optics includes the case that the detection structure is at least partially arranged on a part connected to the microscope optics and / or formed by this. This makes it possible, in particular, to use a detection structure which has a radiation source. The radiation generated by the radiation source can be used, for example, to illuminate the operating room and in particular to illuminate the area of the patient to be operated and also be recorded for recording the at least one recording image. In particular, therefore, the detection structure on the microscope optics does not have to be illuminated by a radiation source arranged elsewhere, which possibly disturbs the surgeon and the auxiliary personnel. In the case of the different wavelength ranges, in particular also when the wavelength ranges overlap, in this case it can be prevented that the radiation from the radiation source of the detection structure on the microscope optics impairs the system function of the surgical microscopy system. For example, the radiation of the radiation source can be aligned and / or masked on the microscope optics such that it does not enter the object region to be imaged by the microscope optics.

It has already been mentioned that mechanical oscillations of components of the surgical microscopy system can disturb the desired system function since they can lead to a relative movement of the microscope optics and of the object region to be imaged. In particular, the natural frequencies of the mechanical oscillations of the microscope optics can range from a few hertz to a few tenths hertz. Vibrations with these natural frequencies lead to disturbances in the imaging by the microscope optics, which are perceptible as de-focusing and / or lateral image offset. For the surgeon, such disturbances can be tiring. In the case of augmentation, such disorders may also affect the outcome of the surgery. It would be conceivable to use acceleration sensors, in particular on the microscope optics, in order to be able to detect these low-frequency oscillations and compensate them by means of a movement control. However, such sensors are expensive to purchase and due to the low frequencies and relatively large volume. The corresponding movement control also tends to the microscope optics driven by the moving means is gradually moved away from its desired position. In order to maintain the desired position, the information about the absolute position of the microscope optics in the operating room is needed.

By contrast, the detection according to the invention of the detection structure and determination of the speed from the at least one recording image makes it possible to directly detect the relative movement of the microscope optics and the detection structure and thus also with a corresponding configuration of the object region. The compensation of low-frequency mechanical vibrations is thus possible directly without determining the absolute position in space.

Not only in the embodiment described above, the determination of the speed from the at least one recording image does not preclude that the relative position of microscope optics and detection structure is additionally determined, e.g. also from at least one recording image and in particular from the same image or the same images as in the speed determination.

In particular, in the low-frequency oscillations mentioned, cameras available today with a resolution of a few megapixels and image readout rates in the range of a few tens to a few hundred hertz are sufficient to provide the information for a motion control to compensate for the oscillations. In addition to determining the speed, it is also possible in particular to determine the position vector which describes the relative movement including its direction of movement.

In practice, in operations, e.g. a positional accuracy of image acquisition and image representation of the system of a few tens of microns sought. At the aforementioned local resolutions of cameras, the matrix of the camera's photocells is sufficient as a two-dimensional reference, i. the matrix represents a two-dimensional coordinate system, to which the recorded recording images and the evaluation of these recording images can be referred.

However, if the microscope optics has a zoom function, ie if the focal length of the optics can be changed and in particular if additional information for the surgeon to be provided, at least one detection structure in the detection range of the microscope optics not only by the at least one image recording device for Purpose of the speed determination are detected, but also an image of the detection structure imaged by means of microscope optics for determining the scale and / or size ratio of the image by the microscope optics are used. In this case, the reference coordinate system of the detection structure is used. For augmentation, the additional information can therefore be related to the coordinate system of the detection structure and Particularly in the case of a detection structure arranged near or in the operating area of the patient, it is ensured that the additional information is provided on the correct scale of the object area.

If the scale of the detection structure is used in the manner described, the at least one recording image which is captured by the detection structure by an image recording device, but the microscope optics are not used for imaging, can also be used in combination with at least one microscope image. Optics are evaluated at a given zoom setting generated image. Taking into account the known information about the scale and / or the coordinate system of the detection structure, it is therefore additionally possible to make use of the image information from the at least one recording image. There is thus redundant information available, provided that the at least one image recording device and the microscope optics are firmly connected to each other. In particular, a calibration of the image scales of the zoom function is reliably possible in this way.

It is preferable that a plurality of cameras are used for recording each of at least one recording image. Furthermore, it is preferred that a plurality of detection structures from the viewpoint of the surgical microscope with the microscope optics (if the cameras are arranged on the microscope optics) or from the perspective of the patient pad or the area around the patient pad (if the cameras are arranged there) in different directions are arranged on relative to the cameras moving part. As a result, the problem can be solved that the resolution of the optical devices of known cameras in the tangential direction (transverse to the beam path when recording the recording image) is greater than in the direction of the beam path.

In particular, for the simultaneous detection of a plurality of the detection structures or of a distributed detection structure having a plurality of elements, at least one camera can have fisheye optics. Furthermore, the use of a so-called pericentric optics is also possible with which not only the end surfaces but also the lateral surfaces of an object (here the patient or the patient pad or the respective detection structure) can be detected.

In particular, due to the comparatively low accuracy required in the operating room for determining the position and determining the speed as well as the local resolution, it is also possible to use further optical imaging principles which allow a three-dimensional image or the acquisition of a three-dimensional image data set. These include optics according to the principle of plenoptics, the line triangulation and / or the fringe projection. The detection structures can be adapted to the optical detection methods.

Parts of the following description refer to the case where the relative movement is perpendicular to the optical axis of the image. However, the invention is not limited to this case. For example, by pre-processing the image data and / or by prior knowledge of the geometry of the image (in particular via the position and / or orientation of the detection structure relative to the camera), it can be taken into account in the determination of the velocity that the relative movement does not run perpendicular to the optical axis.

In the most common use case of a two-dimensional recording image (also referred to as a camera image without limitation to lens optics), there is no depth information, that is, information about the third dimension. However, it can be taken into account in particular by the aforementioned preprocessing and / or prior knowledge that the detection structure can not extend only in a plane which runs perpendicular to the optical axis of the image. For example, the plane of the detection structure may be angled against that plane. It is also possible for the detection structure not to run only in one plane, but for example to have a three-dimensional structure and / or to run along a curved surface. Depending on the depth position of the respective point or area of the detection structure depicted in the camera image, different degrees of lubrication in the camera image may result.

The invention is also not limited to the case of movements of a movable part along a straight-line coordinate axis or within a plane. Nor is the invention limited to linear movements along any direction in space. Rather, it is also possible to measure rotational movements or superimpositions of rotary movements and rectilinear movements with the movement measuring system or by the method.

In particular, the detection structure may have at least one, and preferably more, of the edges mentioned above. In these cases, it is possible in particular to evaluate the local course of the smearing (ie, for example, the transition from white to black or light to dark) at the edge or the edges in the camera image. In this case, for example, knowledge of the position and / or the course of the edge in the detection structure is sufficient as information about the actual appearance of the detection structure. Optionally, it can also be taken into account how sharp the edge is in the detection structure and to what extent and / or in which way the edge is already changed and, in particular, smeared due to the image of the camera image (eg due to aberrations and local resolution of the camera). However, change due to mapping alone can be taken into account when determining the speed even if no edge is present or no smearing is evaluated on one edge.

For example, a detection structure may have one or more line-like surfaces, at the edges of which are the edges, that is, the sudden transitions. In particular, a detection structure can have line-like areas with different widths, that is to say distances of the edges and / or directions of the edge courses. In particular, their staggering can be selected depending on the exposure time interval of the camera and the speed range in which the speed of the relative movement is expected and is to be determined. For example, the detection structure may have a continuous gray scale distribution and / or (in the frequency domain) a continuous frequency spectrum of the structural elements. The frequencies of the frequency spectrum are superimposed to a common gray value distribution. The frequency superposition can be done with statistical phase, so that there is a statistical gray scale distribution in the picture. Such a configuration of the detection structure is well suited to average out noise components in the speed measurement. However, the computational effort for the evaluation of an image of a detection structure with statistical gray value distribution is relatively high. If, as is preferred, each camera image of a sequence of consecutively recorded camera images (for example at a frequency of greater than 1 kHz) is to be evaluated to determine the speed, the computational effort is very high. Therefore, a continuous phase relationship defined by the detection structure for the Fourier frequencies of the structural elements is preferred. In this case, the camera image may be referred to as a contiguous data set, so that no reference or comparison with external information is necessary.

With the described approach of the frequency superimposition of continuous gray-scale distributions it can be achieved that every frequency occurs at every location in the detection structure. Grayscale imaging methods for generating such continuous gray level distributions are already known. Alternatively, however, a binary black-and-white (or light-dark) spatial distributions can be realized by the detection structure, for example. In order to determine the speed from relatively large distances, the point densities (pixel densities) of the detection structure are not resolvable in this case, ie they lie in a spatial frequency interval outside the transmission bandwidth of the optical system used. An example of such a grayscale method is the analysis of Bernoulli noise patterns, eg as published in Mauricio Delbracio et al., International Journal of Computer Vision (2012) 96, pp. 175-195, Springer, "The Non-parametric Sub-pixel Local Point Spread Function Estimation Is a Well-Posed Problem" ,

The continuous gray value distribution described above is merely an exemplary embodiment of detection structures. Basically, according to Fourier theory, each signal can be represented as a superimposition of frequencies. That Even binary structures (which thus form edges at the transitions of the structures) contain many frequencies from the possible frequency spectrum. However, since an edge, i. a single contrast jump in the image that produces a non-periodic signal is for its description a Fourier integral, i. no Fourier sum, required. The frequency spectrum is infinitely extended and contains information at high frequencies that can no longer be resolved by the optical transmission system. This leads to a low signal-to-noise ratio. However, the binary structures in the Fourier plane can be designed to contain low frequencies. In particular, structural elements can be designed regularly or have appreciable periodic proportions. This periodicity has the consequence that the frequency spectrum contains peaks (ie narrow frequency ranges with a sudden increase and decrease in amplitude) whose frequency spacing corresponds to the inverse structure period.

For example, For example, the smearing of the edge in the camera image can be determined, characterized and / or evaluated by methods already known per se in the field of image processing and image enhancement. For example, For example, the first and / or second local derivative of the image value (e.g., the gray value) may be formed along a line transverse to the edge of the edge, and more particularly perpendicular to the edge. For example, By determining a measure of the course of the first local derivative, the degree of smearing can be characterized.

Another possibility is to determine the smearing at each of a single edge by mathematical convolution of a mathematical function that describes the smearing of the edge in the camera image with a second mathematical function that describes the edge of the detection structure.

The immediate evaluation of smearing on a single edge, however, is relatively expensive and therefore costs relatively much computing time if the evaluation, as generally (not only in the case of an edge) preferred, is performed by a data processing device. Also, the information obtained by evaluating a single Kantenverschmierung information content is relatively low. The accuracy of the velocity determination can be increased if not just a single edge is evaluated. It is therefore preferred to evaluate not only a single edge with respect to the smearing in the camera image, but to take into account a plurality of edges and in particular also the spacing of the edges from one another. In particular, it is advantageous if the detection structure has a plurality of edges (i.e., at least three edges) whose distances from each other are not equal. In particular, this concerns the placement of the edges along a virtual or actually existing (e.g., straight) line in the detection structure that intersects the edges transversely to their course, and more particularly perpendicular to their course. In particular, the edge distance is defined between two adjacent edges so that pairs of nearest adjacent edges are at different distances from one another in the detection structure. This does not exclude that edge distances repeat, i. Different pairs of edges that are equally spaced are located in different areas of the structure.

In particular, the local edge distribution of the detection structure may be virtual or actual, e.g. be transformed by a Fourier transform in the frequency domain and in particular the frequency distribution are analyzed and determined in this way the speed. Different edge distances thus correspond to different frequencies in the frequency domain, which have larger amplitudes than other frequencies. It is a possibility to evaluate the differences between the Fourier transform of a reference image, which represents the non-smeared detection structure, and the Fourier transform of the camera image or the region of the camera image assigned to the detection structure, and to determine therefrom the relative velocity. For example, can be determined by calibration of the motion measuring system at different relative velocities and each determination of the change in the frequency distribution based on the Fourier transform of the reference image, which effects the respective value of the speed on the camera image and the frequency distribution has. In particular, by smearing edges, high amplitudes of the Fourier transform of the reference image are reduced and the corresponding peaks are broadened. Both the amplitude reduction and the peak broadening can be evaluated and thus the speed can be determined.

Not only the above-mentioned detection structures can be designed so that properties of the detection structures can be evaluated with respect to a plurality of in particular straight-line evaluation directions. For example, the evaluation can be made at least in two evaluation directions extending perpendicular to one another relative to the detection structure. In this way, not only the velocity component in an evaluation direction, but in several evaluation directions is determined. In particular, it is therefore also possible to determine the direction of the speed. For this purpose, the evaluation directions do not have to be predefined. Rather, it can be concluded from the effects of the smearing on the direction of the speed by determining in the camera image the direction in which the relative movement leads to the highest degree of smearing.

If it is known in advance in which direction the moving part moves, the directional evaluation described above is not required. This is z. As in a so-called linear axis of the case along which the microscope optics is moved in a straight line. However, if the direction of the movement or the speed is also to be determined, a detection structure which has similar structural features with regard to a plurality of possible evaluation directions is advantageous. Similar structural features are not just the distances of edges described above. Other examples will be discussed. The structure preferably has similar structural features, in particular similar geometric structural features, with respect to any theoretically possible direction of the relative movement. For example, therefore, edges of the structure may have a curved and, in particular, circularly curved (eg concentric) profile. The evaluation directions are z. B. perpendicular to the course of the edge (s). The determination of the direction of movement is to be distinguished from the determination of the orientation of the movement, ie the orientation of the movement (forward or backward) along a given or possible movement path. In some of the cases described here, the orientation can not be determined from a single camera image. One way to determine the orientation is to compare camera images of the collection structure taken at different times. With a variant of the method, the orientation can already be determined by evaluating a single camera image. This presupposes that the camera image contains image values which were obtained by evaluation (with regard to the duration and / or the beginning or end) of different time intervals. This will be discussed in more detail.

As an alternative or in addition to at least one edge, the detection structure can have at least one region in which the brightness, color and / or reflectivity for radiation changes continuously and thus not abruptly as with an edge, but continuously increases or decreases. The change is particularly related to a straight virtual or actual line of the structure, which may be an evaluation direction. Preferably, the detection structure has such a continuous change not only in a straight direction, but in at least one further straight direction which is transverse to the first straight direction. For example, in a circular area along different radius lines of the circle, the brightness, color and / or reflectance for radiation may change. In particular, the movement of such a detection structure can be evaluated with at least one continuous transition by means of mathematical convolution of the image value distribution of the camera image.

Furthermore, a detection structure is proposed that has circular structures, in particular concentric structures, which has periodic structural elements with different periods, that is to say in the frequency domain with different high-amplitude frequencies. In particular, such a structure, but also other circular structures, allow the determination of the speed with respect to different directions, with respect to which the speed or components of the speed can and should be determined. If one or more directions of movement are / are fixed and therefore known in advance, the detection structure may be specially designed and manufactured for that direction (s). For example, the edge courses extend perpendicular to the fixed direction of movement. By this restriction to one or more directions of movement, the local resolution in the determination of the speed and / or the accuracy of the determination can be increased.

The evaluation of the edge smearing offers (as well as other embodiments of the method described elsewhere in this description) not only the ability to determine the mean value of the amount of speed over the exposure interval, but under certain conditions, the time course of the speed , eg the average acceleration or even the time course of the acceleration over the exposure interval. The prerequisite is that during the exposure time no reversal of the movement has taken place. This is based on the knowledge that movements with such a reversal and courses of motion without such a reversal can cause at least largely the same smearing. The course of the movement over the exposure interval can be e.g. from the course of the image values and e.g. Considering the local derivative or derivatives along a line of analysis across the course of the edge are determined. But also e.g. the determination and evaluation of a convolution kernel described in more detail enables the determination of the course of motion.

The mapping of the detection structure by the camera onto the camera image can be described as mathematical folding. In particular, an equation can be formulated by which the image value distribution of the camera image over the image area is equated to the result of an operation in which the so-called convolution kernel is processed with the intensity value distribution of the recorded detection structure. In particular, only that part of the image value distribution of the camera image which corresponds to the acquired acquisition structure is considered. In particular, a reference image of the detection structure which is suitable, for example, as the intensity distribution of the detection structure (which is not affected by smearing) is suitable. was recorded by the camera when there is no relative movement between the camera and the detection structure. The reference image may alternatively be e.g. from planning data of the detection structure and / or by computer simulation of the detection structure. As will be described later, the convolution kernel can be determined. It contains the information about the mapping and thus change of the intensity distribution of the detection structure and thus also the information about the speed of the relative movement.

The at least one camera image and the previously and / or subsequently mentioned reference image of the detection structure are, in particular, digital images, ie have a plurality of pixels. Accordingly, the camera is in particular a digital camera. However, the reference image does not need to be in the same data format and / or processed as the camera image. In particular, the reference image and the region of the camera image in which the detection structure is imaged can be referred to as mathematical functions are understood, so that in particular a mathematical convolution of the area of the camera image with the reference image or derived from the reference image function is executed and preferably also executed.

In the preferred case, the camera image and / or the reference image are two-dimensional images. Therefore, in particular, a camera known per se with a two-dimensional matrix of photosensitive sensor elements arranged in rows and columns can be used. The pixels of the camera image are accordingly also arranged in rows and columns in this case.

In particular, the information about the actual appearance of the detection structure may comprise a reference image of the detection structure, wherein the speed of the relative movement is determined by evaluating differences of the reference image and the at least one camera image recorded by the camera. By taking into account the reference image, the speed can be determined easier, more accurate and in less time. The evaluation of the differences can be limited in particular to a partial area of the camera image, for example to the partial area in which the detection structure is detected. If several acquisition structures are detected with the camera image, the evaluation of the differences can be limited to the subregions in which one of the acquisition structures is detected in each case.

A preferred possibility of evaluating the differences between the camera image and the reference image and determining the speed of the relative movement provides the abovementioned mathematical description of the convolution of the intensity distribution of the detection structure with a convolution kernel which yields the image value distribution in the camera image corresponding to the detection structure. In particular, when using certain classes of detection structures whose properties are discussed in more detail, the convolution kernel and thus the information about the speed of the relative movement can be determined in a simple manner from the camera image using the reference image.

In particular, by mathematical convolution of the reference image with the region of the camera image in which the detection structure is imaged, a convolution kernel of the convolution can be determined, and the speed of the relative movement can be determined from the convolution kernel.

In a simple and intuitive way, the determination of the speed of relative movement succeeds when the convolution kernel is interpreted as a geometric structure whose outer dimensions correspond to the outer dimensions of the reference image and the outer dimensions of that portion of the camera image in which the detection structure is imaged. In this case, the speed of the relative movement can be determined from at least one geometric property of a partial structure of the convolution kernel. In particular, in a rectilinear motion, the amount of velocity may be determined from a length of a substructure of the convolution kernel. In the case of a two-dimensional camera image and reference image, the magnitude and / or direction of the velocity can be determined from the geometry of a partial structure of the convolution kernel. This applies in particular to the simple case that the detection structure moves through the camera transversely to the optical axis of the image of the detection structure. Otherwise, however, such an evaluation is also possible, for example, if a corresponding geometric correction is made and / or the geometric properties of the arrangement of camera and detection structure are taken into account.

As a collection structure, certain classes of patterns are particularly well suited. Patterns with edges have already been discussed. If the convolution kernel is determined by mathematical convolution of a mathematical function which describes the detection structure as a function of the location (spatial function), in particular a corresponding operation takes place in the frequency domain. This means that properties of the detection structure in the frequency domain are important. The detection structure described as a function of the location can in particular be transformed into the frequency domain by a Fourier transformation.

Preferably, therefore, a frequency space detection structure has the property of having no zero within a frequency range beginning at zero frequency but not including frequency zero and ending at a given maximum frequency. That is, the function value of the function in frequency space, which may also be referred to as amplitude, is greater than zero within the frequency range. The maximum frequency is / is in particular set so that it is not smaller than the Nyquist frequency of the camera. In particular, the maximum frequency is equal to the Nyquist frequency of the camera or greater than the Nyquist frequency of the camera, but is not more than twice the Nyquist frequency of the camera. A detection structure without zeros in the frequency domain has the advantage that the convolution of the detection structure by an operation related to the frequency space succeeds and therefore can be carried out in a simple manner. For zeros, on the other hand, the operation is not defined in the frequency domain.

In a further development of this class of detection structures, with which the convolution can be reliably and accurately performed, the function value of the spatial function of the detection structure transformed into the frequency domain is greater than a predefined minimum value in the entire frequency range from zero to the maximum frequency. In particular, the predetermined minimum value is greater than a statistical fluctuation (noise) of the image values caused by the acquisition of the camera image and by the evaluation of the camera image by the determination device. More generally, the predetermined minimum value is greater than a statistical fluctuation amplitude of image values of the recorded image caused by the recording image recording and speed determination. The fluctuation amplitude therefore corresponds in particular to the statistical fluctuation amplitude of that with an additional noise-affected detection structure which, under the theoretical assumption of noise-free camera image recording and processing, leads to the same noisy evaluation result as the actual noisy recording and processing of the detection structure. By observing the minimum value, it is therefore possible to achieve an evaluation of the camera image that is not or not significantly influenced by statistical fluctuations. Such a detection structure can be generated, for example, using Barker codes.

Preferably, the function value of the spatial function of the detection structure transformed into the frequency domain is constant in the entire frequency range from zero to the maximum frequency. Such a detection structure is particularly easy to evaluate and with little computational effort.

The constant function value in the frequency domain corresponds to the property that the convolution of the detection structure with itself or its inverse function results in a delta distribution in the spatial domain. In other words, a preferred class of sensing structures is therefore defined by the property of having a perfect autocorrelation. This means that the result of the convolution of the structure with its inverse structure gives a structure in place space which at a single position (in particular in a central local area of the structure) has a value different from the values at all other positions in the central area of the structure Has. In particular, this one value is a high value so that it can be called a peak. The values at all other positions in the central area of the result structure are the same, e.g. zero or can be normalized to zero, i. the value distribution is constant except for the peak. As will be explained in more detail, the convolution kernel of the convolution can be determined with such a detection structure in a particularly simple and time-saving manner, which results in the corresponding image by the camera when the convolution kernel is applied to the intensity distribution of the detection structure. Also in a particularly simple way, the speed can be determined from this convolution kernel.

If e.g. Positions of the relatively movable relative to the camera part or be determined, the determined speed can be used for correcting the position determination. Apart from an improvement of the position determination result by immediate evaluation of the recorded camera image, the determined speed can also be used for the correction of delay effects independently of the question of whether a position determination takes place. Delays occur in particular due to the evaluation of the camera image and the duration of further processes of information processing, in particular in the motion control of the microscope optics. The knowledge of the speed of the movement allows at least an estimate of how far the relatively moving parts have moved during a delay period. When considering a relative movement, it does not matter which or which of the parts has actually moved in a fixed coordinate system of the surgical microscope system. Also, in the calibration of an optical positioning system having, in particular, the same camera or cameras and in particular the same sensing structure (s), the determined velocity may be used, for example the effects of motion during capture of a camera image for the purpose of calibration to correct.

The movement measurement according to the invention has the advantage that not only the position of a movable part, but also the speed of the movement can be determined from a single camera image and in particular also determined. For the purpose of determining the position, it is therefore possible in particular to correct a position-determining error which arises as a result of the movement by evaluating the current (that is to say of the last-determined) camera image with determination of the speed. There is no need to wait until the next camera picture is taken.

As already briefly mentioned, the determination of at least one kinematic variable which describes the relative movement (for example the speed) can be used to control and / or regulate the movement of at least one movable part of the surgical microscope system, in particular the microscope optics. In this case, the orientation of the movement determined by evaluating at least one camera image can also be taken into account.

In the following, the problem of determining the time course and in particular also the orientation of the relative movement of the relatively movable parts will be discussed. In particular, when using the detection structures described above with at least one edge, but also in other described in this description detection structures, it is already possible by evaluating a single recording image of the camera to determine the speed of movement, but not readily the temporal course of movement and orientation. In many cases, the movement causes the same change in the actual appearance of the detection structure as recorded by the recording image, regardless of whether the movement takes place in one direction or in the opposite direction. This also complicates the determination of the temporal course of movement.

At least one solution should be specified, which makes it possible to determine the orientation of the movement and also the temporal course of movement, in particular from a single recording image.

Before considering the solution or solutions it should be pointed out that the determination of the orientation of the movement can also be carried out without a determination of the speed of the movement. This concerns both the motion measurement system and the surgical microscopy system with a motion measurement system and the method for operating the surgical microscopy system.

For example, Determining the orientation is also advantageous if further information about the movement is obtained in a different way than by evaluating at least one recording image. However, this does not exclude that the evaluation of at least one recording image of a detection structure also contains additional information about the movement and this information is obtained. A case in which an evaluation of a camera image for determining the speed or position of the microscope optics is not absolutely necessary, concerns conventional scales with scale sensors, as are possible in stands for microscope optics with relatively movable parts. In particular, the scale sensors may gain pulses from the recognition of marks (e.g., line marks) on the scales. Since the distances of these markings are known, information about the position and the speed of the movement can be obtained from the pulses and the time course of the pulses. However, it is not possible to deduce the orientation of the movement solely from the temporal course of the impulses. In particular, in this case, the orientation of the relative movement of the microscope optics and the base of at least one recording image of the detection structure can be determined.

The solution, which enables the determination of the orientation of the movement and also of the temporal course of movement already from a single recording image, is based on a temporal variation in the transmission and / or detection of the radiation emanating from the detection structure. As a result, additional information about the relative movement of the microscope optics and the base is obtained. The temporal variation can in principle take place before, during and / or after the transmission of the radiation from the detection structure to the at least one camera and also during the detection of the radiation by the at least one camera. To use the additional information, the knowledge about the temporal variation in the evaluation of the at least one recorded image is used.

The determination of the orientation or the knowledge about the orientation of the movement makes it possible to eliminate an ambiguity in the evaluation. This makes it possible to determine the temporal course of movement. Although this is preferred, the invention is not limited to determining both the orientation and the time course of the movement from the respective recording image or the respective combination of recording images. For example, the knowledge about the orientation of the movement can be obtained in another way and, as already explained above, alone the determination of the orientation is advantageous.

According to a first proposal for determining the orientation and / or the time course of the movement, the radiation transmitted from the detection structure to the camera during the exposure time interval is varied during the exposure time interval, ie the local radiation distribution emanating from the detection structure changes over time during the exposure time interval. This therefore leads to a temporal variation in the transmission of the radiation. All sensor elements of the camera Sensor matrix can have the same exposure time interval, ie the exposure time interval begins for all sensor elements at the same time and ends at the same time. This is the case with commercially available cameras. Alternatively, the individual sensor elements of the camera, which each detect a local area of the radiation distribution, but have different exposure time intervals.

The temporally varying local radiation distribution has different effects on different sensor elements of the camera and therefore the amount of radiation detected by the various sensor elements during the exposure time interval (the received radiation power integrated over the exposure time interval) differs even at a constant movement speed. More generally, a first one of the sensor elements of the camera detects a particular local area of the detection structure during the exposure time interval due to the time-varying local radiation distribution other than a second sensor element of the camera, during the exposure time interval the same local area of the detection structure, but to a different sub-period of the exposure time interval. Knowledge of the local radiation distribution as a function of time makes it possible to determine the temporal course of movement and / or the orientation of the relative movement of the microscope optics and the base.

The temporal course of the movement can be described in different ways. For example, it can be described by the path, the speed, or by the acceleration of the motion, each as a function of time.

Already from a single recording image, the time course of the movement can be determined during the exposure time interval corresponding to the acquisition of the recording image. Optionally, the progressions of the movement can be determined from a plurality of successive recording images and from this a temporal course of the movement can be determined over a longer period of time.

As will be explained in more detail in the description of the figures, also with reference to mathematical equations, there are two ambiguities in the evaluation of a single recording image. One of the ambiguities can be eliminated by the known temporal variation described herein in the transmission and detection of the radiation emanating from the detection structure, for example the local radiation distribution (see above) and / or the camera characteristics (see below). The other of the ambiguities is the orientation of the movement. In the evaluation of the recorded image, it is possible in particular first to use the knowledge of the temporal variation and then to check which of the two possible orientations of the movement is more plausible and / or leads to a plausible result for the time course of the movement.

In particular, the distribution of the image values in the recording image can be mathematically described as convolution and a convolution kernel of the convolution can be determined, wherein the convolution kernel is determined by the known temporal variation and by the temporal course of the movement. With the exception of orientation, the convolution kernel is uniquely determined by these two functions of time. Taking into account the knowledge of the temporal variation and by deciding which orientation of the movement applies, the temporal course of the movement can be determined.

Another ambiguity arises when the speed becomes zero during the exposure time interval, and especially when the direction of movement reverses during the exposure time interval. This can be countered with a correspondingly short duration of the exposure time interval. Alternatively or additionally, the information content in the at least one recording image can be increased.

There are various possibilities for changing the radiation distribution which originates during the exposure time interval from the detection structure and can be detected by the sensor matrix of the camera during the exposure time interval. A first possibility of the temporal change of the local radiation distribution is the change of the radiation intensity, ie the radiation flux density. A second possibility is the change of the spectral distribution of the radiation. Yet another possibility is the change in the polarization of the radiation. The possibilities can be combined in any way. However, it is also possible, for example, that the total intensity of the radiation emanating from each local area of the detection structure remains constant during the entire exposure time interval, but the spectral distribution and / or the polarization is changed over time, at least for partial areas of the detection structure. For example, the spectral components lying in the green, red and blue spectral range can change during the exposure time interval, for example in opposite directions. For example, therefore, the radiation intensity in a first spectral range increase, while the radiation intensity in a second spectral range decreases and vice versa. Optionally, in the meantime, the radiation intensity in a third spectral range can remain constant. This makes it possible to use the constant spectral range as a reference, ie all sensor elements that detect this spectral range over equally long part-time intervals of the exposure time interval also receive the same amount of radiation in this spectral range. A sensor element usually consists of several sub-sensor elements which are sensitive to radiation in different spectral ranges (eg red, green and blue). If, for example, two different sensor elements in the green spectral range have received the same amount of radiation during the exposure time interval, but not in the blue and red spectral range, this allows, with knowledge of the temporal change in the radiation distribution, to conclude the orientation of the movement.

Both with respect to the variation of the intensity of the spectral components and with respect to the total intensity over the entire spectral range of the radiation, the intensity can not only be monotonically increasing or alternatively monotonically decreasing during the entire exposure time interval, but alternatively the intensity can be at least maximum and / or at least go through a minimum. In particular, the intensity can be periodically varied, e.g. with sinusoidal intensity over time.

Preferably, a sinusoidal variation of the intensity of different spectral components at at least three different frequencies of radiation, e.g. one frequency each in the red, green and blue spectral range. In contrast to the embodiment already mentioned above, in this case also a variation of the intensity takes place in the green spectral range. In practice, e.g. red, green and blue light emitting diodes may be used to generate the radiation emanating from the sensing structure with varying intensity over time. Light-emitting diodes, not only in this combination of three colors, can be easily controlled in order to produce the desired time course of the intensity.

The at least three spectral components with a sinusoidal temporal course of the intensity are in particular a narrow spectral range. A majority or all of the temporal courses of the intensity may have the same frequency of the intensity change. The spectral components are not limited to the visible spectral range. Rather, in particular, radiation components in the infrared range and in compliance with health and safety regulations are also in the ultraviolet range in question.

The at least three sinusoidal waveforms of the radiation intensity in the at least three frequency ranges each have a phase shift in pairs, i. the maximum of the intensity is reached at different times. Although this is preferred, the frequency of the intensity change over time need not be the same for all at least three sinusoidal waveforms. With exactly three sinusoidal curves, it is preferred that the frequency of the intensity change is the same and the phase shift between each pair of sinusoidal curves is one third of the period.

The use of at least three sinusoidal intensity curves in different spectral ranges has the advantage that the orientation of the relative movement of the two parts can be clearly determined at any rate in all cases in which no reversal of the movement occurs during the exposure time interval.

As an alternative to a sinusoidal intensity curve, the intensity can be varied in the manner of a repeated random jump-like change (in the following: pseudo-random) over time.

In a pseudo-random variation, the different intensity levels of the radiation are similar as in a random distribution, but the variation is predetermined and known. It would be conceivable to actually make the change in intensity random. However, this increases the effort to determine the actually occurring variation of the intensity and to take into account in the evaluation. Also, in a truly random variation, there is a certain probability of variations that contain little information for determining the orientation.

The pseudo-random variation of the intensity can be compared with a code corresponding to the temporal sequence of intensity levels. Unlike an actual random variation of the intensity levels, it may be specified that each intensity jump from one level to a following level or at least one of several consecutive intensity jumps must have a minimum jump height (down or up).

Regardless of the nature of the variation in intensity, it is preferred that the amount of radiation incident maximally on any sensor element of the camera sensor array during the exposure time interval be matched to a saturation amount of the sensor elements. Saturation means that the sensor signal from reaching a saturation amount of radiation is no longer or no longer in a suitable manner, the above the saturation amount of received radiation amount reproduces.

The consideration of the saturation quantity makes it possible to emit (emit and / or reflect) a high proportion of the saturation quantity from the detection structure, in particular at the beginning or at the end of the exposure time interval during a short subperiod, in relation to the length of the subperiod. For example, For example, during the first percent of the integration period, a radiation amount in the range of 10% to 20% of the saturation amount, e.g. 15% of the saturation amount to be broadcast. In the course of the exposure time interval, therefore, a correspondingly rapid drop in intensity takes place, so that the saturation amount can not be reached or exceeded. Sensor elements that have received a certain amount of radiation during the exposure time interval must therefore have detected the area of the detection structure from which the initially high amount of radiation was emitted, at the beginning of the exposure period. Already this information gives clear indications of the orientation of the movement. Not only with respect to the exemplary embodiment described in this paragraph, this clarifies that at least in the case of a movement that does not reverse during the exposure period, the orientation of the movement can be clearly determined in a simple manner.

Previously, an example of a widely varying intensity has been described, i. the first time derivative of the intensity of the radiation is large. As mentioned above, this results in a relatively small proportion of the sensor elements receiving a large amount of radiation during the exposure time interval and therefore it is easy to deduce the orientation of the movement and the time course of the movement. This is not limited to that the radiation amount emanating from the detection structure is particularly large in a certain sub-period (e.g., at the beginning or end) of the exposure time interval. As long as the information about the period in which the amount of radiation is particularly large is taken into account in the evaluation of the image, the orientation and also the time course of the movement can be determined in a simple manner. However, in a preferred embodiment, it should be noted that the change in the radiation intensity in any case does not run in all spectral regions with time symmetry to the center of the exposure time interval. Without further measures that can still be taken, the orientation can not be clearly determined with such a symmetrical variation of the radiation intensity.

It has already been mentioned above that the radiation intensity can be periodically varied. In a further development of this approach with a plurality of temporally successive intensity maxima and intensity minima during each exposure time interval, the frequency of the intensity maxima or intensity minima is varied. However, the frequency matched to the expected movement speed is not so high that the radiation intensity has the same effect on all sensor elements along the movement direction of the movement. Rather, it is preferred that the frequency has a maximum value which is not exceeded and allows one to distinguish the effects on the reception of radiation quantities on the various sensor elements due to the variation of the frequency and thereby conclusions on the orientation and / or to trace the time course of the movement.

Another way to determine the orientation and / or the time course of the movement by temporal variation is a temporal variation on the receiving side of the radiation, i. the camera. Individual embodiments have already been indicated. So it is e.g. possible that the exposure time intervals of all sensor elements of the sensor matrix of the camera do not start at the same time and end at the same time. This is achieved, in particular, by the values of the integrated radiation quantities of the individual sensor elements being read out in succession and being reset to zero with the readout. In this case, it is optionally possible to carry out the readout of the radiation quantities simultaneously for a plurality of the sensor elements, so that the sensor elements are assigned to a plurality of read-out sequences. In accordance with the read-out sequences, the sensor elements assigned to the read-out sequences are read out one after the other. Alternatively, several read-out sequences of adjacent sensor elements may be present in the sensor matrix, but only one sensor element of the entire sensor matrix is read out at a time. For example, in the case of two read-out sequences, one of the sensor elements of the two read-out sequences can therefore always be read alternately.

In any case, the knowledge about the different exposure time intervals of the individual sensor elements in the evaluation and the determination of the orientation and / or the time course of the movement is taken into account. A complete recording image of the camera, which contains values from all the sensor elements, is therefore not, as usual, a snapshot limited to a single exposure time interval, but contains information about a longer time course of the detection of the Sensing structure. If, for example, a sensor element, past which a certain part of the detection structure has passed, does not detect this part of the detection structure or with less radiation than its neighboring sensor elements, then information about the orientation and / or the temporal course of the movement can be obtained therefrom win. In general terms, taking into account the knowledge of the exposure time intervals of the individual sensor elements, it can be determined in a simple manner whether the movement with the one orientation or with the opposite orientation corresponds to the recording image of the camera and / or the temporal course is determined.

Alternatively or in addition to using different exposure time intervals, other camera characteristics may be varied over time. This particularly relates to the influence of the radiation transmission of the radiation optics of the camera, e.g. using apertures and / or filters with temporally varying properties (eg, by movement or control of the aperture and / or filter), as well as the sensitivity of the incident radiation sensor elements. At lower sensitivity, the same received radiation quantity results in a smaller value of the sensor signal of the sensor element and vice versa. If e.g. the sensitivity decreases rapidly from a high value at the beginning of the exposure time interval to a low sensitivity value later in the exposure time interval, then this corresponds to the embodiment described above with high radiation intensity at the beginning and low radiation intensity at the end of the exposure time interval.

The temporal variation of properties of the camera, including the use of different exposure time intervals of the individual sensor elements, can be compared with the above-described temporal variation of the local intensity distribution of radiation emanating from the detection structure and leads in part to corresponding procedures for determining the orientation and / or the time course of the movement. All of the above-mentioned methods of temporal variation make it possible, in particular, individually or in combination with each other, to determine the orientation and / or the temporal course of the movement from a single recording image.

Furthermore, it is possible to use not just one camera but multiple cameras, e.g. Cameras with sensor elements that are sensitive in comparison to cameras in different spectral ranges. It is also possible to use a camera, either as a single camera or in combination with other cameras, the sensor matrix of the camera having sensor elements whose spectral sensitivity differs from the spectral sensitivity of other sensor elements. For example, For example, the various measures of temporal variation may be combined by varying the intensity level of the radiation emanating from the detection structure pseudo-randomly while at least one characteristic of at least one camera producing a recording image of the detection structure is varied other than the intensity level.

Another way to change the local radiation distribution over time during the exposure time interval is to change the polarization of the radiation transmitted from the detection structure to the camera over time. For example, the detection structure may comprise a light source and / or a display, wherein in the direction of the camera, a controllable polarization filter of the detection structure is arranged, which allows to control the polarization, the radiation emanating from it in the direction of the camera locally different and time-dependent.

The camera or at least one of the cameras can have sensor elements which are sensitive to radiation of a specific polarization direction selectively. In an analogous manner as in conventional sensor matrices, the individual sensor element has a plurality of sub-sensor elements sensitive to different spectral ranges, the sensor elements can each comprise a plurality of (at least two) sub-sensor elements that are sensitive to radiation of different polarization directions. In this way, the polarization direction can be varied in an analogous manner, as above for the different temporal variation of the intensities in different spectral ranges of the radiation incident on the camera, and the same information regarding the motion detection and in particular with respect to the determination of the orientation and / or the temporal Course of the movement win.

In the description of the figures, it will be explained in more detail that the detection structure can have at least one marker. The description of the figures concentrates on concrete exemplary embodiments. However, it is generally possible that the detection structure has at least one marker or consists of at least one marker. In particular, a marker can be understood as meaning a surface having a circumferentially closed outer circumferential line (ie outer edge, eg circle or rectangular line) be, with the radiation properties (emissivity and / or reflectance) change abruptly at the perimeter. In particular, the radiation properties of the surface bounded by the circumferential line can be constant. The peripheral line of the marker thus represents an edge in the sense of the above description.

In particular, the detection structure may comprise at least one circular marker. This has the advantage that the relative movement of the detection structure and the camera in each direction along the marker surface results in that the circular line on the outer circumference of the marker is perpendicular, i. in the direction of a radius of the circular line, is crossed by a projection of the movement path of the relative movement to the marker surface. Thus, in each of these cases, there are adjacent sensor elements of the camera sensor array exposed to a sudden increase or decrease in received radiation intensity during the course of the relative movement, as described above with respect to edges. In particular, if only certain directions of movement can occur, the use of differently shaped than circular markers of advantage.

In the following, a development of the motion detection and in particular the determination of the orientation and / or the time course of the movement will be described. This development is based on the problem that different local intensity distributions and / or different temporal variations of the local intensity distribution and / or the camera properties lead to different good results. This also depends on the nature of the relative movement of the relatively movable parts. In particular, the movement may be faster or slower and have greater or lesser dynamics, i. the acceleration and / or the jerk of the movement can be different.

To solve this problem, it is proposed to record and evaluate a chronological sequence of recording images by the camera, by a plurality of cameras or in each case by one of a plurality of cameras. In this case, at least a first recording image of the temporal sequence is evaluated and, depending on the result of the evaluation of at least one first recording image, the temporal variation of the local radiation distribution and / or the temporal variation of the camera properties are changed. Accordingly, a second recorded after the evaluation of the at least one first recorded image recording image radiation in an exposure time interval in which the temporal variation of the local radiation distribution and / or the temporal variation of the camera characteristics is already changed from the exposure time interval of the first recording image.

In this way it is possible to adapt the temporal variation to the relative movement. In particular, with present day means of controlling radiation sources, e.g. Displays and / or arrangements of light-emitting diodes, an adjustment of the temporal variation of the local radiation distribution at a frequency of recording of recorded images in the range of up to 1 kHz or even over 1 MHz possible. While the radiation emanating from displays e.g. can be varied in the range of a few 100 Hz, for example in the range of 200-600 Hz, the emanating from light emitting diodes radiation can be varied at much higher frequencies up to the MHz range. Due to the computing power of computers available today, the evaluation of individual recording images and the control of changed temporal variation of the local radiation distribution at these frequencies is possible. The same applies to the temporal variation of the camera properties. For example, For example, the order in which the integration values of the sensor elements are read out can be easily changed even though it may result in a meaningful recording image not being obtained during a period between the recording of the first recording image of the sequence and the second recording image of the sequence the readout is changed.

Alternatively or additionally, after evaluating the at least one first recording image and before recording the second recording image, the duration of the exposure time interval can be changed, eg if it is determined during the evaluation of the at least one first recording image that the saturation value has been reached, exceeded or at least one of the sensor elements is approximately reached, or if it is determined that the saturation value is not nearly reached in all sensor elements, for example, only 10% or 20% of the saturation value can be achieved. Alternatively or in addition to changing the duration of the exposure time interval, the intensity of the radiation may be adjusted. In general terms, it is preferable to specify in which range of the radiation quantity between zero and the saturation value the radiation quantity of the sensor elements should lie when recording images are recorded and / or which conditions should follow the distribution of the radiation quantity values supplied by the sensor elements. If the default is not met, for example, the predetermined range, for example, for a given number or a predetermined percentage of the sensor elements not is maintained, the radiation intensity and / or the duration of the exposure time interval is changed until the criterion for a following recording image is met.

Returning to the temporal variation of the local radiation distribution, two embodiments will now be described. According to the first embodiment, the intensity of the radiation in at least one local area of the radiation distribution (e.g., at least one marker associated with the local area) in different spectral regions of the radiation is temporally sinusoidally varied, i. the intensity is a sine function of time. The sinusoids of the intensity can be phase-shifted in at least two, preferably at least three different spectral ranges.

If, during the evaluation of the first recording image of the sequence, it is determined that the result of the evaluation in relation to the determination of the orientation and / or the time course of the movement is unsatisfactory (because, for example, it is not possible to clearly determine which orientation the movement has) , then the amplitude of at least one sinusoidal waveform, the frequency of at least one sinusoidal waveform and / or the phase shift between at least two sinusoidal waveforms of the different spectral ranges is changed. In at least one further recording image, which is recorded with changed sinusoids, it is then determined whether the result of the evaluation is satisfactory or whether a change in the temporal variation of the radiation distribution is made. Optionally, the sinusoids are changed again after evaluation of one or more of the further recording images until a satisfactory evaluation result is obtained. Typically, the sinusoidal waveforms are set to a higher sinusoidal frequency when the dynamics of the motion are greater and vice versa. The amplitude of the sinusoidal curves is preferably set so that the signal / noise ratio with respect to the evaluation of the various spectral components is high, whereby it should also be ensured that the radiation quantities determined by the sensor elements of the camera sensor field are in the exposure time interval over a substantial portion distribute between zero and the saturation amount.

According to a second embodiment relating to the above-described pseudo-random variation of the intensity level, the kind of the pseudo-random variation is changed when it is determined after evaluation of the first recording image that there is a need. The type of pseudo-random variation of the intensity level of the radiation is determined, in particular, by the length of the time intervals during which the intensity is constant, by the difference between the maximum allowed and the minimum allowed intensity and / or by the maximum permitted intensity, i. one or more of these parameters are changed when the type of pseudo-random variation of the intensity level of the radiation is changed. In particular, the length of the time intervals of constant intensity can be chosen to be shorter if the dynamics of the movement are high and vice versa. The difference between the maximum allowed intensity and the minimum allowed intensity can be selected in an analogous manner as in the first embodiment for the amplitudes of the sinusoidal waveform. The same applies to the maximum value of the intensity.

Fast image processing algorithms that allow a decision as to whether the temporal variation of the radiation intensity and / or the camera properties should be changed, for example, use a fast Fourier transform (FFT). Even methods that evaluate radiation quantities in different spectral ranges can be designed quickly in this sense. In addition, even when evaluating a recording image with respect to radiation in a single spectral range, for example, a frequency shift due to the relative movement of the relatively movable parts can be quickly detected. In this case, the radiation intensity in particular periodically (for example, cosine-shaped) can be changed over time. At a constant speed of the relative movement, the frequency of the periodic change in the recording image is unchanged from the locally periodic course of the image values. In contrast, a non-constant speed leads to a frequency shift in the spatially periodic course of the image values and the temporal velocity profile can therefore be determined from the recorded image. The frequency shift can be detected, for example, by means of a locally resolved frequency analysis at a plurality of locations of a recorded image. In this case, the local distribution of the frequency shift can be determined and thus information about the time course of the relative movement can be obtained.

Furthermore, information from the history of the movement, in particular from the past evaluation of a sequence of camera images, can be used to adjust the temporal variation of the transmission and / or detection of the radiation. Particularly suitable for this purpose are Kalman filters.

The variation of the intensity in different spectral regions and the alternatively or additionally performed variation of the polarization of the radiation are perceived by persons who are in the area of the surgical microscopy system as less disturbing than the variation of the total radiation intensity.

In particular, at least one camera can be used with sensor elements which are each sensitive in one of more than three or four spectral ranges. In particular, so-called hyperspectral cameras are known which have sensitive sensor elements in more than ten, for example, sixteen different spectral ranges. In this way, the information content in the recording image can be further increased.

Particularly in the case of such cameras, but also in the case of cameras with sensor elements which are sensitive in less spectral ranges, the radiation intensity of the radiation emanating from the detection structure can be varied in the different spectral ranges with different frequency of the intensity change. For example, in a first spectral range, the radiation intensity may be varied at a frequency of 5 kHz and in another, second spectral range, the radiation intensity may be varied at a frequency of 10 kHz. Different frequencies increase the information content and simplify the evaluation of the recorded images.

In particular, the range of movement of the two relatively movable parts (microscope optics and base) can be small. It is therefore possible to make the spatial detection range of the at least one camera also small, without the detection structure being able to leave the detection range. In particular, in this case, the method described below of the irradiation of a pattern on the surface of the detection structure is advantageous because both the relatively small spatial detection area and the evaluation of the result of the irradiation of the pattern requires little effort or an evaluation of the recording images with particularly high Accuracy allows.

One possibility of temporal variation is irradiation (in particular projection) of an adjustable local radiation distribution onto the detection structure. The appearance of the detection structure, which already exists without the projection, determines, together with the incident radiation distribution, the effective appearance which is detected by the at least one camera.

In particular, the appearance already present without projection can form a first pattern (that is, a structure) and the irradiated local radiation distribution on the surface of the detection structure can produce a second pattern. In particular, binary patterns, for example checkerboard-like patterns, parallel-striped patterns, concentric-ring patterns, and / or patterns having special properties such as those based on 8th MURA further questioned. In particular, the first pattern and the second pattern may be similar, so that the superimposition of the first pattern with the second pattern, in particular the same appearance as the first pattern alone and / or at least in local portions of the surface of the detection structure deletion of the pattern (ie homogeneous appearance) can result. This makes it possible, in particular when evaluating at least one recording image, to determine a simple variable which contains information about the relative movement. For example, the degree of cancellation of the first binary pattern by the second binary pattern can be easily determined by the area ratio that does not correspond to the binary value of the cancellation (eg, "1" for "bright"). Alternatively or additionally, a moiré effect arising from the superimposition of the two patterns and / or other pattern-specific overlay effect can be determined and evaluated in a simple manner.

In a further development, a predetermined state of the superposition of the first and the second pattern may be present and may be attempted by the irradiation of the second pattern on the detection structure to reach this predetermined state despite the relative movement of the relatively movable parts or to come as close as possible , In order to achieve this, for example, as described above, at least one first recorded image is evaluated and the irradiation of the second pattern is adapted as a function of the result of the evaluation. This makes it possible in particular to keep the range of values for the size to be evaluated small. This in turn makes it possible to determine the size to be evaluated with little effort and good signal-to-noise ratio.

Embodiments of the invention will now be described with reference to the accompanying drawings. The individual figures of the drawing show:

1 in a simplified schematic illustration of a surgical microscope system and a patient lying on a patient pad in an operating room,

2 a schematic representation of a part of in 1 illustrated surgical microscopy system with two cameras, in their detection areas each have a detection structure, which is arranged on the patient,

3 schematically in a simplified representation of an arrangement as in 2 however, the cameras are attached to the patient pad and detect sensing structures attached to the microscope optics,

4 schematically the lower part of a housing of the microscope optics, wherein four cameras detect detection structures on different sides of the housing,

5 a schematic representation of different subregions of a detection structure with different dimensions and an associated image value distribution in a camera image, wherein there is no relative movement between the detection structure and the camera,

6 a schematic representation of the sub-areas and the associated image value distribution as in 5 but where there is a relative movement between the detection structure and the camera,

7 temporal curves of the radiation flux density recorded by two pixels of the camera image for the top left in 6 illustrated portion of the detection structure during the relative movement,

8th a MURA (Modified Uniformly Redundant Array) as an example of a part of a detection structure whose camera image can be evaluated with little effort to determine the speed of the relative movement,

9 schematically in a single representation a convolution kernel as a two-dimensional location-dependent function, wherein in a central region of the convolution kernel for two different cases of a substructure is shown, based on the geometric properties of the speed of relative movement can be determined

10 Examples of the temporal course of a radiation flux density with three different movements, whereby integration of the radiation quantity over the exposure time interval alone can not differentiate between the three different movements,

11 integrated radiation quantities of a pixel of a camera image over time for the three movements, for the in 10 the temporal courses of the radiation flux density are shown,

12 integrated amounts of radiation of the pixel similar to in 11 However, where the integration periods in time against the integration periods of the representation in 11 are offset,

13 an example of a simple position controller for controlling the position of a moving part,

14 a cascaded control for controlling the position and speed and optionally also the acceleration of a moving part (eg a microscope optics or a patient pad) as an example of a control that takes into account the actual value of the speed of the moving part determined from a single camera image,

15 schematically positions of a circular marker in a movement with constant moving speed in a straight line, a course of image values of a sensor field along the direction of movement and a temporal intensity curve during the exposure time interval, wherein the transmitted from the marker to the sensor array of a camera radiation with respect to the intensity does not vary over time becomes,

16 a schematic representation of a case as in 15 However, the marker performs a movement in a straight line direction with decreasing speed,

17 a schematic representation of a case as in 15 and 16 however, the marker performs a rectilinear motion that reverses during the exposure time interval, ie the orientation of the movement changes during the exposure time interval,

18 a schematic representation of a case as in 17 however, the intensity of radiation transmitted from the marker to the sensor array of the camera decreases during the constant pitch exposure time interval,

19 the sinusoidal time course of the intensities of radiation components in three different spectral ranges and

20 the intensity profile of radiation in a pseudo-random variation of the intensity level.

1 shows a surgical microscope system 1 and one on a patient pad 15 lying patients 16 in a schematic side view. This may be, for example, a surgical microscopy system marketed by Carl Zeiss Meditec GmbH under the name OPMI PENTERO 900. The surgical microscopy system has a substructure 2 on, which can stand on the floor of the operating room and can optionally be provided with rollers or wheels, not shown, to the substructure 2 to move on the ground. In the substructure 2 are preferably all electronic components for operating the surgical microscope system 1 arranged, such as a controller 4 for controlling drives, not shown. The control can be realized in particular by a computer with computer programs. In addition, the computer can also control the in 1 not shown microscope optics of the surgical microscope system 1 Perform the microscope optics relative to the patient 16 or to the patient pad 15 to keep in constant relative position.

At the substructure 2 Optionally, a display and / or display may be connected via which the surgeon or operator of the surgical microscopy system 1 Information about the operating state, the operating options and / or can perceive the images displayed by the microscope optics. Furthermore, at least one interaction device may be connected to the substructure to allow the surgeon or the operator to interact with the surgical microscopy system. For example, in this way the position and / or orientation of the microscope optics can be controlled manually. Examples of interaction devices include a keyboard, a joystick, a footswitch, and a touchscreen.

The surgical microscope system 1 has a carrier 3 in the form of a tripod. To the carrier 3 include a height-adjustable telescopic arm 6 , a first arm 7 and a second arm 8th on which in turn a suspension 9 is arranged. The suspension 9 in turn has a connection 10 , a swivel arm 11 and a holding piece 12 on, on the holding piece the surgical microscope 5 is arranged.

The first arm 7 is at one end about one in 1 not shown in the vertical direction by the telescopic arm 6 extending first axis rotatably mounted. At its other end is again an end of a second arm 8th rotatably mounted about a second axis parallel to the first axis. Thus, the first arm and the second arm form a hinge mechanism that allows the surgical microscope 5 in a plane parallel to the floor of the operating room to a desired position. Furthermore, the telescopic arm allows 6 a movement of the carrier and the operating microscope arranged thereon 5 in the vertical direction. Furthermore, the suspension allows 9 a rotary movement of the surgical microscope 5 around three axes of rotation. The first axis of rotation extends vertically in the longitudinal direction of the connection 10 , The second axis of rotation runs horizontally in the longitudinal direction of the swivel arm 11 , The third axis of rotation is perpendicular to the longitudinal axis of the pivot arm (ie in the in 1 shown position perpendicular to the plane of the figure) in the transition region between the swivel arm 11 and the holding piece 12 ,

Movements according to the above-described degrees of freedom of movement are driven by associated drives with motors (eg linear motors or rotary motors). They are used in particular by the controller 4 controlled by control cables (not shown) laid in the stand. In addition, position and / or rotary encoder can be provided on the joints. However, it is also possible to dispense with such signal transmitters if position control or position regulation or other motion control or motion control is used with consideration of the speed determination method according to the invention with optional additional position determination.

In the 2 and 3 shown partial representations of the arrangement in 1 represent possible embodiments of the invention. They can also be combined with each other, ie detection structures can both with the surgical microscope 5 as well as with the patient 16 and / or the patient's pad 15 be connected. The same applies to cameras that capture the detection structures in at least one recording image.

In 2 are only the connection of the tripod 10 , the swivel arm 11 and the holding piece 12 shown, wherein schematically also a drive 18 in the area of the transition between the connection 10 and the swivel arm 11 is shown, which the rotational movement of the pivot arm 11 to drive the first axis of rotation. The drive 18 is via a dotted control line with a control 19 connected, for example, the controller 4 out 1 may have, which in the substructure 2 is arranged. The regulation 19 is also via dotted signal lines with two cameras 23 . 24 connected, which has a holder 22 with the surgical microscope 5 are connected. The control receives over the signal lines 19 during operation of the scheme repeated camera images, the cameras 23 . 24 each of a collection structure 28 . 29 in their coverage. Every camera can do this 28 . 29 repeatedly, at short time intervals or on immediately without a distance successive time intervals recording images (ie camera images) of the respective detection structure 28 . 29 record and via the signal line to the scheme 19 to transfer.

Schematically is in 2 and accordingly also in 3 the microscope optics 21 of the surgical microscope 5 shown. It detects an object region to be detected, in which at least one body region of the patient 16 located. The edges of the detection areas of the microscope optics 21 as well as the cameras 23 . 24 are indicated by dashed lines in 2 and accordingly also in 3 shown schematically.

In the embodiment above the microscope optics 21 there is a sensor matrix 20 for taking pictures of the object area. By a signal line, not shown, the images are for example to the substructure 2 of the surgical microscope system 1 transmitted to be visually perceptible to the surgeon or operator. In addition, an augmentation of the displayed image information can be found.

In the in 2 illustrated embodiment are two sensing structures on the patient 28 . 29 arranged. They are shown schematically as crosses. This can be any suitable detection structure. For example, as will be described later, line patterns or arrays of circular markers may be used as the detection structure. Numerous other possible embodiments have already been described, including luminescent markers or structures.

During operation of the in 2 and also in accordance with 3 arrangements shown capture the cameras 23 . 24 in each case at least one recording image of the detection structure that it detects 28 . 29 and, from differences of the at least one recorded image to the actual appearance of the detecting structure, becomes a speed of relative movement of the camera 23 . 24 and the surgical microscope 5 on the one hand, and the collection structure, on the other 28 . 29 on the other hand.

In 3 an embodiment is shown in which the cameras 23 . 24 about additional supports 35 . 36 with the patient pad 15 are connected. The collection structures they cover 28 . 29 are on the bracket 22 attached. In particular, it may be in the collection structures 28 . 29 in 3 to act as self-luminous collection structures. This does not disturb the surgeon during the operation, as the illumination is directed from the top towards the cameras. Optionally, it can also be directed in the direction of the surgical area and thus additionally illuminate the surgical area.

The embodiments of the 2 and 3 serve to illustrate the two different basic principles of the sensing structure and camera arrangements on the relatively movable parts or patient parts or arrangements. As already mentioned, combinations are possible. Also, many other modifications are possible in terms of alignment, type of mounting, number of cameras and number of detection structures. For example, there may be a different number than two detection structures. The same applies to the number of cameras. Furthermore, at least one camera, in the case of attachment to the surgical microscope, can use the microscope optics for the imaging and recording of the detection area covered by it. There may be more than one detection structure in the detection area of the camera or cameras. The collection structures must in the basic case of 3 can not be arranged on the patient pad or on the patient, but can also be attached to other resting relative to the patient pad parts of the operating room.

In 4 is a variant of an arrangement with four cameras 43a . 43b . 43c . 43d represented, which are connected via one or more support structure, not shown, stationary in the operating room with the surgical microscope and thus with the microscope optics. 4 shows schematically and simplified that on the housing of the surgical microscope 5 an arrangement with a multiplicity of markers 41 can be located. From the simplified representation are in 4 only in the foreground and to the right of the case marker 41 recognizable. However, there are more markers left and in the background on the housing. Depending on their point of view, each camera observes and captures 43 at least the markers 41 at one or two of the surface areas of the housing which are planar in the exemplary embodiment.

At the in 2 . 3 and 4 The arrangements shown, the speed, the time course and / or the orientation of the movement and optionally also the position and / or orientation of the movable part (the microscope optics or the patient or the patient pad), for example, as follows. The cameras repeatedly record images of their detection range, eg with a clock frequency of 100 Hz. Each image is given a time stamp, ie the time of the recording of the respective camera image is assigned to the camera image and stored, for example, in a common data set with the image data.

In particular from camera images of several of the cameras recorded at the same time, the speed, the time course and / or the orientation of the movement and optionally also the position and / or orientation of the movable part are detected within its movement range. Each individual camera image to be evaluated is optionally subjected to pre-processing, in which the image region of the camera image in which there is at least one detection structure of the movable part (for example a characteristic feature or a marker) is determined. In this case, only the image data of the determined region of the camera image are further evaluated for the purpose of determining the speed, the time course and / or the orientation of the movement and optionally also the position and / or orientation of the movable part. This pre-processing is e.g. within the respective camera which has taken the camera image, and is e.g. performed by a computing device of the camera. Also, the next processing step, namely the determination of the speed, the time course and / or the orientation of the movement and optionally also the position and / or orientation of the at least one detection structure, which is detected in the camera image, with respect to a coordinate system of the camera ( which may in particular be a two-dimensional coordinate system lying in the image plane of the camera image), can be executed decentrally, eg by the mentioned computing device of the camera. Alternatively, however, it is also possible for the camera images of a plurality of cameras recorded at the same time and / or the camera images of a plurality of cameras not captured at the same time but having the same or approximately the same position and / or orientation of the movable part to be jointly from a central evaluation device of the surgical microscopy system. This is particularly useful when camera images of different cameras capture the same detection structure or the same detection structures. The computing device can be, for example, one or more microprocessors and / or FPGA (Field Programmable Gate Array).

When ascertaining the speed, the time course and / or the orientation of the movement and optionally also the position and / or orientation of the movable part from the camera image or images, the knowledge of the geometric properties of the detection structure is used in particular. For example, the detection structure may be a detection structure having a circular or rectangular outer edge, the surface of the detection structure not being optically homogeneous within its boundary, i. over the course of its surface, the detection structure has an optical structure. It can therefore be concluded from the image of the structured surface on the speed, the time course and / or the orientation of the movement and optionally also on the position and / or orientation of the detection structure and thus on the associated movable part.

The knowledge of the at least one detected detection structure makes it possible, on the basis of geometric considerations, to determine from the at least one camera image how the detection structure is positioned and / or aligned in the movement region of the movable part. For example, a circular edge detection structure in the camera image is generally depicted as a structure having a circumferential edge of an ellipse. For example, by detecting the position and length of the major axes of the ellipse in the camera image, the angle of view of the camera with respect to the detection structure and the distance of the camera to the detection structure can be determined. Preferably, the at least one detection structure detected in the camera image or camera images contains redundant information, such that the position and / or orientation of the detection structure is based not only on a structural feature but on a Plurality of structural features can be performed. This increases the safety in determining the position and / or orientation of the movable part. This also applies with regard to the evaluation of a plurality of camera images which have detected at least one detection structure from different viewing directions in the same movement state of the movable part.

The determination of the imaging geometry of camera and movable part, e.g. however, calculation of the viewing angle and distance from the geometry of the imaged registration structure is not the only possible approach. Other methods are known from image processing. For example, By comparing the detection structure imaged in the camera image with simulated and / or previously acquired images, the position and / or orientation of the detection structure in the movement region can be determined. For example, Each of the simulated or previously recorded images may be assigned the appropriate position and / or orientation. By determining the correct comparison image, therefore, the position and / or orientation is determined.

In particular, if markers of the detection structure (s) each have an individual structure or are otherwise individually designed for the camera, information about the camera can already be obtained from the identification structure captured in the camera image and from the knowledge of the alignment of the camera's coverage area Approximate position and / or orientation of the movable part can be obtained.

Unlike in 2 a plurality of detection structures can be arranged in the operating room and the cameras can detect different of the detection structures with their recording images, depending on their position and also on their orientation. The detection range of the cameras in this case is so small that not all detection structures can be detected simultaneously. Therefore, as compared to detection areas extending over larger solid angles, the respective detected detection structure occupies a larger portion of the captured camera image, and therefore, in principle, a finer resolution and more accurate detection of the speed of the movable portion can be achieved while detecting the position and / or orientation of the movable part can be achieved on the basis of the identified marker and / or the number of pixels of the camera images can be reduced so that camera images can be recorded, in particular with a higher repetition frequency.

Based on 5 to 7 is explained in simplified examples, which effects a relative movement of a first part (for example, the microscope optics or in another case of the patient or the patient pad) and a second part (of the patient or the patient pad or in the other case the microscope optics) and in general also have relative movements of any parts when a camera is coupled to one of the parts and takes a camera image of the other part. According to 5 For example, partial areas (shown at the top in the figure by four rectangles with different widths) of a binary structure with different dimensions, that is to say partial areas of a movable part and in particular the detection structure, can be considered. In the simplified example, one-dimensional portions of the binary structure are considered whose dimensions in the width direction (from left to right in the figure) are one, two, three, and four (generally n) image pixels. In the case that the subregions are radiation-reflecting or emitting regions of an otherwise non-reflecting or emitting structure 5 including the local history of image values of corresponding pixels of the camera image, which captures the subregions. Taking into account the optical properties of the detected subregions and the image of a maximum possible saturation S, the image values correspond to the maximum possible integration of the radiation incident on the respective pixel over the exposure time if the exposure time (integration time) of the camera is selected accordingly. Prerequisite for the appearance of the sketched image value course is that the object is not moved during the image acquisition.

6 illustrates the relationships when the object (in this case the binary structure) suddenly starts to move at time zero, shifts to the right during the constant-speed integration time by one pixel in the image space (the position is symbolized by dashed rectangles), and then abruptly comes to rest. Among the subregions, the local profiles of the saturation S or the image value profiles of the corresponding n camera pixels are again shown.

In 7 For example, the radiation flux density Φ for two of the camera pixels p1 and p2 is shown over time. The exposure time interval begins at time t1 and ends at time t2. 7 sets the case of only the one-pixel portion to the left in 6 The radiation flux density which the pixel p1 receives is constant until the beginning of the exposure time interval at time t1 Level and decreases linearly to the value 0 during the exposure time interval. At the same time, the radiation flux density Φ of the second pixel p2 increases linearly from time t1 to time t2 to the same level that previously had the radiation flux density on the pixel p1.

From the course of the radiation flux density Φ for the pixels p1 and p2, the executed movement can be determined. However, the respective image value of pixels p1 and p2 provides like 6 shows the same value because it is integrated over the integration time interval and the timing over this time interval is unknown. However, this is different for the local image gradients in 6 for the broader sections. Since there is a radiation-reflecting and / or emitting partial area in the middle local area during the entire exposure time interval, the saturation S there is maximal. At the edges of the image value curves or saturation curves, however, the value is lower. There has been a smearing of the edges, which are located in the direction of movement at the ends of the sections. This information can now be evaluated to determine the speed of the movement. Already the length of the regions of pixels n which received radiation and integrated during the exposure time interval gives the magnitude of the constant velocity or, in general, the mean amount of velocity during the exposure time interval. In the exemplary embodiment illustrated, the partial area has moved on by one pixel and therefore the length of the area of the exposed pixels or of the non-disappearing portions of the saturation curve shown is greater than in the case of FIG 5 , by exactly one pixel. The speed therefore results by dividing the increase in length between the cases of 5 and the 6 , divided by the exposure time interval. The information in 5 corresponds to knowledge about the acquisition structure recorded with the camera image. In particular, the width of the four top in 5 shown rectangular sections known in advance. These subareas can be identified by per se known methods of image processing despite the smearing of edges in the camera image. In this case, with repeated recording and evaluation of camera images of the camera, in particular from frame to frame, an object tracking take place, so that the effort for the identification of the known subregions or structures is small.

To return to the above consideration of the transfer function of the map, the transfer function changes with the movement. Therefore, the transfer function contains the information about the movement. In particular, the change in the transfer function can be viewed with the moving and non-moving object (detection structure) or the motion can be evaluated directly from the moving object transfer function. In particular, a frequency analysis, in particular after transformation into the frequency domain (in particular Fourier transformation) and / or mathematical convolution can be used. In addition to an evaluation of image contrasts, e.g. At structural edges (this has already been discussed), therefore, a mathematical convolution of the image value distribution of the camera image for evaluating the movement in question and is also preferred. In particular, depending on the selected method of evaluation and / or depending on the expected speed of movement, the detection structure can be produced, which is detected by the camera. Examples of suitable collection structures have already been mentioned. For example, Siemens stars are suitable as detection structures with edges.

Embodiments for determining the velocity from a convolution kernel will now be described. For this purpose, an underlying mathematical description of the mapping of the detection structure to the camera image as a convolution of the intensity distribution of the detection structure with a convolution kernel will first be described.

In the following formulas and equations, the gray-scale distribution GW of the image of the detection structure, possibly smeared by the camera image, occurs. However, this is just one example of the distribution of image values. For example, alternatively or additionally, the color distribution, eg color values and color intensities, can be taken into account. In the one-dimensional case in which the gray value distribution GW depends only on a spatial coordinate x, the gray value distribution GW (x) can be described by the following integral equation (1):

As an example, equation (1) simplifies the assumption that the relative velocity v _x of the detection structure and the camera in the x-axis direction is constant during the exposure interval and in particular runs perpendicular to the optical axis of the image. The velocity v _x is related to the time variable t and the distance s _x traveled by the movement s _x in the direction of the x-axis via the relationship s _x = x + v _× t:

T is the length of the exposure time interval, ie the time interval over which the imaging structure has been imaged onto the camera image. I (s _x ) is the intensity distribution of the non-smeared, imaged acquisition structure. The last line of equation (1) contains the replacement of the integral by a simplified notation of the convolution of the convolution kernel k with the intensity distribution I. The convolution is dependent on the spatial variable x. For the convolution kernel k the following applies:

Where ℓ is a spatial variable in the direction of the x-axis.

The following describes the corresponding two-dimensional case. The relative movement takes place with a velocity constant in direction and magnitude in an xy plane and runs perpendicular to the optical axis of the image. The velocity is therefore clearly described by its components v _x in the direction of the x-axis and v _y in the direction of the y-axis extending perpendicular to the x-axis. Accordingly, s _y is the distance in the y-direction, which is dependent on the time variable t and has been traveled since the beginning of the exposure interval. The function δ is the Dirac function also called delta distribution. Equation (4) reads as follows:

Since this is the two-dimensional case, two variables are defined, namely ℓ _x in the x direction and ℓ _y in the y direction. The convolution kernel k is accordingly described by the following equation (5):

Again, the gray value distribution can therefore be expressed as a convolution of a convolution kernel with the intensity distribution of the non-blurred detection structure: GW (x, y) = (k * I) (x, y) (6)

As expressed in equation (1), last line, equation (6) for simplified cases of constant velocity perpendicular to the optical axis, the image value distribution of the image of the detection structure detected by the camera can be expressed by the convolution of a convolution kernel k with the intensity distribution. Since the intensity distribution describes the detection structure in the original non-motion blurred state, and since the blurred state by the movement in the camera image results from the image distribution, the convolution kernel k contains the information about the speed of the relative movement that led to the blurred camera image Has.

For the convolution kernel k, a length can now be defined which is related to the speed over the exposure time or integration time and the imaging geometry. By determining the length, therefore, the speed can be determined. Of course, this first requires the determination of the convolution kernel. This will be discussed in more detail. In particular, the detection structure can be selected such that in the central local area of the convolution kernel, which does not form the geometric boundary of the convolution kernel, a geometric substructure of the convolution kernel is included and the length of this substructure is that length, which is the size and / or direction corresponds to the speed.

In the following, examples of detection structures are described in which this substructure only corresponds to the size of a single pixel of the camera image when the speed is zero (and otherwise there are no aberrations or other influences that lead to smearing of the detection structure in the camera image). If, on the other hand, the amount of speed is not zero, the substructure increases so that its size can be described by said length corresponding to the magnitude and / or direction of the velocity. For example, the substructure may correspond to the size of two adjacent pixels of the camera image. This means that from the beginning of the exposure interval to the end of the exposure interval, a movement has taken place which led to a camera pixel for the displacement of the detection structure as seen by the camera. This is especially true in the case of constant speed. The substructure results in the central area from the difference of the image values (eg gray values or color values) to the environment of the substructure in the central area. For example, the substructure is defined by constant image values in a contiguous local subregion of the central region of the convolution kernel, while outside the substructure in the central region, constant image values lie at a different image value level than for the substructure. In particular, it can be achieved by suitable normalization of the convolution that the image values of the substructure have the value "1" in a possible image value interval of [0, 1], while the central region outside the substructure is described by constant, lower image values, for example by Image values "0". However, since the convolution kernel can be interpreted in particular as a spatial function whose scaling corresponds to the matrix of the camera image, the central region of the convolution kernel is in particular corresponding to the subdivision of the camera image into pixels digitally structured. There may therefore be blurring at the edges of the substructure of the convolution kernel, ie image values between the constant level of the image values of the substructure and the constant level of image values outside the substructure. In this case, nevertheless, the size of the substructure and thus in particular the length mentioned can be determined. Methods for determining the size of the substructure are already known from image processing for the identification of objects that have blurred edges in the camera image. Therefore, this description will not be discussed in more detail.

As already mentioned, not only the magnitude of the velocity but also its directional course and / or temporal change can be determined from the geometry of the substructure in the central region of the convolution kernel. In particular, a relative movement with non-constant amount of velocity leads to image values in the substructure of the convolution kernel which are not constant. The image values of the local regions corresponding to the pixels in the camera image (also referred to below as pixels for the sake of simplicity) can therefore have constant high or constant low values, for example, only in a central region of the substructure. However, it is also possible that due to the nature of the increase or decrease in the speed amount, even in a central region of the substructure, no constant values of pixels can be found, ie there are no two adjacent pixels whose image values are the same. Nevertheless, the determination of the size of the substructure is possible, for example, with the mentioned already known methods. From the course of the image values along the extension of the substructure, it is also possible to obtain information about the temporal course of the speed during the exposure time interval. This is based on the knowledge that the pixels of the substructure of the convolution kernel behave like the pixels of the camera image: with a slower movement, the respective pixel of the camera image receives more radiation intensity than with slower motion, if a corresponding location area with high radiation emission and / or high radiation reflectivity is imaged onto this pixel and a corresponding subsequent movement of the local area of the detection structure has a lower radiation emission and / or radiation reflectivity. In the simplified example of the above equations (1) and (5), the extension of the partial structure in the moving direction corresponds to the running variable ℓ, although the examples of the simplified case of the constant speed have been described. However, it is also possible for a corresponding convolution kernel to define a corresponding run variable at non-constant speed, which has non-vanishing values only in the region of the extent of the substructure. In particular, in the event that the direction of the movement changes during the exposure time interval, it should again be made clear that the length of the substructure of the convolution kernel is generally not a length to be determined in a straight direction but a path traveled by the movement corresponding length is. An example is still based on 9 received.

Certain classes of detection structures can be easily manufactured and allow a particularly simple and time-saving determination of the convolution kernel. Also, the resulting substructures of the convolution cores can be particularly easily evaluated in particular in the manner described above. One such class of collection structures is the so-called Modified Uniformly Redundant Arrays (MURA), which have the property of giving a single peak of intensity with their inverse structure folded. This corresponds to a mathematical delta function. In particular, the size of the structures of the particular binary MURA used is matched to the geometry of the image and resolution of the camera image such that the intensity peak corresponds to a single pixel of the camera image. The corresponding local area of said substructure in the central region of the convolution kernel therefore has the dimensions of a single pixel if no smearing and no aberrations occur. Not only for MURA, this coordination of the geometry of the image and the image resolution of the camera is preferred.

However, this condition can not always be maintained depending on the type of motion tracking during operation of the surgical microscopy system. For example, The geometry of the image changes with the distance of the detection structure from the camera. In this case, appropriate corrections of the distance can be made and / or the evaluation of the substructure of the convolution kernel can be made taking into account the changed geometry of the mapping. Preferably, therefore, the geometry of the image and in particular the removal of the detection structure are determined by the camera and taken into account.

Alternatively, the class of URA (Uniformly Redundant Arrays) comes into question for a simple determination and evaluation of the convolution kernel. URA results in a delta-function peak folded with itself as a result.

Both MURA and URA are binary structures in particular. For example, they are chosen for the determination of the convolution kernel so that only the image values normalized to the interval [0, 1] occur. An example of a MURA 41 is in 8th shown. One recognizes the point-symmetric structure to the center of the represented surface of the MURA.

In the case of a detection structure which has substructures created with Barker code, the convolution kernel has the following properties at the relative velocity 0:

In generalization of the above equations (1) and (6), the convolution kernel can be determined by applying to the convolution (to the right of said equations) the operation that eliminates the intensity distribution of the detection structure. The type of operation is therefore dependent in particular on the type or class of the collection structure. For example, in MURA, the term describing the convolution of the convolution kernel with the intensity distribution of the detection structure is convoluted with the inverse detection structure. Therefore, it is only necessary to apply the corresponding operation to the gray value distribution obtained and the convolution kernel k results. In the case of the URA, this is the convolution of the image value distribution with the intensity distribution of the detection structure. Thus, only a single convolution operation is required to determine the convolution kernel. In particular, as already mentioned, an aberration of the image can also be taken into account in the evaluation of the convolution kernel. In particular, a convolution kernel can be determined in each case in a previously executed calibration of the movement measuring system for at least one relative position and preferably for different relative positions of the detection structure and the camera, if no relative movement takes place. By evaluating the convolution kernel, the result of the aberration can be determined and taken into account in the later actual operation of the motion measuring system.

Preferably, a detection structure is composed of a plurality of adjoining subareas which, in the manner of tiles, give a total area without gaps. An example is the total area of nine tiles composed of three lines with three subregions each 9 is shown. Three of the tiles, namely the tiles in the first row, are identified by the reference numeral 51 designated. The sub-area or the tile in the center of the arrangement is denoted by the reference numeral 51e designated. Each of the partial areas is preferably formed by the same pattern, so that the patterns repeat periodically due to the tiling. It is also possible to generate such periodic structures as a detection structure by patterns and in adjacent subregions inverted patterns. The periodic structures have the advantage that edge effects due to the finite dimensions of the detection structure in the determination of the convolution kernel in the central region have little or no effect. Depending on the number of central patterns that do not result in the edge of the entire detection structure, there are one or more central areas in each of which the above-mentioned partial structure of the convolution kernel exists, which can be evaluated to determine the relative speed. Several such central areas can optionally be evaluated simultaneously or one after the other. This makes it possible, in particular, to increase the accuracy in determining the speed if edge effects are still to be detected in the central area and / or if other effects, such as statistical fluctuations of the image signals of the camera sensors (noise), occur.

The representation in 9 can be understood as a schematic representation of a convolution kernel resulting from convolution of the image value distribution from a camera image. In the central area 51e are two substructures 53 . 54 shown, which are not simultaneously observed in the same convolution kernel, but represent the possible variants of the substructure. The substructure 53 is, for example, a rectilinear structure that results from right to left in the figure one after the other lined up pixels. The length l corresponds to the amount of relative velocity taking into account the geometry of the detection structure and the geometry of the image. Also, the direction of movement with the constant speed can be from the substructure 53 if this is of interest and not already known by the characteristics of the surgical microscopy system.

The substructure 54 has a curved shape since the speed has changed direction during the exposure time interval. The curved course corresponds to the curved path of the movement. Although not shown, the length of the substructure may be 54 be determined as the length of the curved course. Thereby, the amount of the speed or the mean value of the amount of the speed is determined. The curved course contains the information about the direction of movement.

If the detection structure extends in a plane that is not perpendicular to the optical axis, the images of the individual tiles, which have the same pattern per se, shown as different patterns in the camera image. However, this can be taken into account in particular by using information about the inclination of the plane relative to the optical axis before, during and / or after the evaluation of the camera image.

10 shows temporal curves of the radiation flux density Φ on a sensor element of the camera, which generates a camera image pixel, the progressions are shown for three different movements. The camera pixel contains the integrated information about the radiation received by the sensor element over the exposure time interval, which begins at time t1 and ends at time t2. In all three cases shown, the movement begins at time t1 and ends at time t2. Until the time t1, ie until the beginning of the exposure period, the sensor element receives radiation at a maximum radiation flux density. However, by the movement, the radiation flux density decreases to 0 until the end of the exposure time interval at time t2. A first movement takes place at a constant speed during the exposure time interval. (solid line of Strahlungsflussdichteverlaufs). A second movement proceeds in the first part of the exposure time interval at a constant speed, then stops and is again at constant speed in the same direction as before (end of the exposure time interval) (dashed line of the radiant flux density trace). Further, the radiation flux density trajectory is shown for a third movement similar to the second movement, but the period of interruption of the movement in the middle of the exposure time interval is larger, and therefore the amount of speed at the beginning and at the end of the exposure time interval is larger. This third course is represented by a dotted line.

In all three cases, the sensor element receives the same amount of radiation. This follows from the illustration in 10 by integration of the radiation flux density courses over the time interval from t1 to t2. All three surfaces under the Strahlungsflussdichteverläufen are the same size. Therefore, the sensor element and the information provided by it can not be used to identify the course of the movement during the exposure time interval. However, this only relates to the illustrated ideal course of the movements, which start exactly at time t1 and end precisely at time t2. This also applies to the consideration of only a single pixel. As already stated, information about the time course during the integration time interval can be obtained on the basis of the consideration of several pixels. However, it is described below that even the observation of a single camera pixel leads to a possibility to increase the information content overall over the course of the movement in the camera image.

11 shows the integrated over a exposure time interval of a sensor element radiation flux density ∫φ (t) dt. At the end of an integration time interval at the times t1, t2 and t3, the integration value provided by the sensor element is represented by a small dark rectangle. In this case, a movement takes place, which with a solid line in 10 Radiation flux density curve shown corresponds to, ie the speed is constant during the integration time interval between the times t1 and t2 and before and after zero. Therefore, the integration value at time t1 is the maximum possible value. At time t2, the integration value is 50% of the maximum value, and at time t3 (that is, a complete integration time interval after the end of the movement), the integration value is zero.

12 shows the integration values at one opposite 11 modified situation. The modification is that, although not the duration of the integration time intervals has been changed, but the times of the beginning and the end of the respective integration time interval. The same results would be obtained if the beginning and end of the time interval of the movement had shifted at a constant speed.

The following will discuss the situation that as in 11 the time course of the radiation flux density, which is incident on the sensor element, by the solid line in 10 is shown. However, the exposure time interval always starts offset by half a time interval from that of 11 described situation. Therefore, the integration value in the representation is in 12 at the time of half an integration time interval before time t1 still at the maximum value. However, at the end of the next integration time interval, exactly midway between times t1 and t2, the integration value is only one-quarter below the maximum value at the end of the next integration time interval. In the middle of the times t2 and t3, the integration value is one quarter of the maximum value. Only at the end of the following integration time interval after the time t3 is the integration value at zero, since the movement ends only at the time t2 and only after the time t3 ended integration time interval is a complete integration time interval in which no radiation was received.

If, on the other hand, the movement takes place in 10 According to the course of the radiation flux density, which is shown with a dashed line, between the times t1 and t3, two other integration values are obtained, which are indicated by non-darkened rectangles in 12 are shown. This is because in the integration time interval ending between the times t1 and t2, the radiation flux density has dropped faster, but in the integration interval ending between the times t2 and t3, the radiation flux density has not fallen as fast as in the other one illustrated case with the solid line in 10 ,

From this, it can be concluded that the position of the integration time interval has an influence on which integration values and thus image values supply the individual sensor elements of the camera matrix.

This knowledge can be used in different ways to increase the information content over the time course of the movement and to determine information about the time course of the movement by evaluating the camera image or a plurality of camera images. In particular, this makes it possible to easily determine the magnitude and direction of the velocity, that is to say the velocity vector. In all cases, information is detected by at least one camera that contains integrated amounts of radiation over different integration periods. A single camera can already provide a camera image whose pixels have time-offset, but e.g. are the same length of integration periods. Such cameras are also referred to as so-called rolling shutter cameras (English: rolling shutter cameras). When using a single camera with temporally offset integration periods, it is preferred that the detection structure captured in total by the camera image contains redundant information in different subareas. For example, The camera detects a first half of the detection structure with the upper half of the camera image and the second half of the detection structure with the lower half of the camera image. When the integration periods of the upper half and the lower half of the camera image are different and there are redundant structure information in the first half and the second half of the detection structure, the information content in the camera image is increased. Alternatively or additionally, it is possible to use a plurality of cameras whose integration periods differ.

In all cases, the integration periods can not or not only differ in that they are temporally offset from each other. Rather, the duration or length of the integration periods may (also) differ.

A further consideration makes it clear that the speed of the movement and / or the time course of the movement can be determined from one or more camera images with image information which originates from different integration periods of the integration of radiation. In general terms, therefore, the camera image may comprise pixels whose image values each correspond to an amount of radiation integrated over an exposure time interval, the exposure time intervals of different pixels of the camera image differing from each other, and taking into account information about the different exposure time intervals, the speed of the relative movement and / or information about one the time course of the movement is / are determined. Already from a single camera image with image information, which comes from different integration periods, the orientation and / or the time course of the movement can / can be determined.

If, for example, a line-shaped structure that moves transversely to the longitudinal extent of the line is detected with pixels from mutually offset integration periods, this results in an offset of the line. For example, if a first pixel occupies the one end portion of the line over a first time interval, a second pixel occupies the middle portion of the line for a second time interval offset from the first time interval by one-third and adopts a third pixel to the other end portion of the line A third time interval offset by two thirds from the first time interval results in three lines in the entire camera image, each corresponding to one third of the three sections of the original line. Taking into account the time offset of the time intervals, the speed can be calculated and / or the time course of the speed and / or the orientation of the movement can be determined. Depending on the orientation, the arrangement of the three lines differs in the camera image.

Therefore, preferably, the detection structure is designed to contain information (eg, edge traces or a known binary structure) that can be used at different time intervals of radiation integration to determine the velocity. For example, can be achieved by appropriate selection of the object contents of the detection structure that calculates the smearing in the camera image can then be used to calculate the speed in the offset from the subregions (eg lines) of the image.

In particular, a digital camera known per se can be used which has a sensor matrix with at least one row of sensor elements whose integration values are successively read in the order of the sensor elements along the line and used as image values of the pixels of the camera image. The duration for reading out the image values of a line results in the line speed. In the case of a two-dimensional sensor matrix with sensor elements arranged in rows and columns, the individual lines (or alternatively the individual columns) can be read out in succession at the line speed. In particular, from the local offset (i.e., a positional shift) of the camera image of the detection structure detected by different rows or different columns of a two-dimensional camera image, the velocity can be determined.

An image of a detection structure taken by a single row of sensor elements can not initially be distinguished from zero velocity as the detection structure moves at the line speed of the camera. However, unlike a non-moving object, the movement of the object (the detection structure) causes smearing, depending on the speed and exposure time of the individual sensor elements. As described elsewhere in this description, therefore, the speed can be determined from the smearing or changing of the image of the detection structure with respect to the non-moving detection structure.

More generally, it is therefore preferable to combine the knowledge about the different integration time intervals with the speed measurement taking into account the information about the actual appearance of the detection structure. In particular, the detection structure is therefore designed and manufactured in such a way that it enables a determination of the relative movement of the camera and the detection structure both by smearing and by effects of the different integration time intervals.

Advantageously, a plurality of cameras, at least two cameras, with in particular the same number of rows and columns of the sensor elements and the same line speed can be used. In the case of the different cameras, however, the sequence in which the radiation quantities integrated in by the sensor elements over the individual integration time intervals are read out as image values differs. For example, when reading out the image values of two cameras, the readout in the first camera can begin with the sensor element in the first row and the first column (or with line scan cameras with the first sensor element) and can read out in the second camera with the sensor element in the first camera Last line and last column begin (or on line scan cameras with the last sensor element of the line). Commercially available cameras read the image values line by line at the same time. In this case, for the first camera, for example, the reading may begin with the first line and the second line with the last line. In another embodiment, readout in the central region of the matrix (eg, with respect to the lines and / or the two-dimensional matrix) may begin simultaneously at both or all of the cameras, but with the sensor elements being read out in reverse order. In the case of line-by-line simultaneous read-out, reading starts with the first camera, for example with the last line of the first half of the lines, and with the second camera with the first line of the second half of the lines. The readout takes place in particular at the same line speed and in two-dimensional arrays line by line instead of one after the other. This can generally be referred to as crossed readout. In this case, the matrices of the sensor elements of the different cameras are aligned in the same way in particular to the detection structure, i. the same or approximately the same area of the detection structure is detected by the first sensor element of the respective camera. The crucified reading can be achieved, for example, in a simple manner by aligning two similar cameras with rolling read-out relative to one another with their optical axes turned by 180 ° onto the detection structure. For example, the cameras may first be similarly aligned with the detection structure, and then one of the cameras rotated 180 degrees about its optical axis.

This specific example makes it clear that information about the relative movement is recorded solely by taking into account the local offset of the simultaneously recorded camera images of the two cameras, which information is preferably used to determine the speed and / or at least one other climatic variable of the movement.

Particularly when using more than two cameras, it is preferred that the order of reading out the sensor elements differs for all cameras and / or that corresponding to one another, the same area of the detection structure detecting sensor elements in the different cameras are read out of time.

Due to the different order of readout, a homogenization of the resolving power of the camera ensemble is achieved when detecting the movement in different spatial directions, in particular in the case of more than two cameras, so that the object speeds in the different spatial directions can be measured with approximately the same accuracy.

It is advantageous to produce the detection structure in such a way that it does not contain any periodic structural elements which lead to ambiguous images in the planned object velocity spectrum, as described e.g. in an apparent reverse rotation of a forward-rotating spoked wheel with a film camera at corresponding repeated frequencies of the image recording is the case.

The time-shifted operation of several cameras is also a way to increase the resolution of the speed measurement. Alternatively or additionally, this time-shifted operation enables a motion measurement with determination of depth information. e

A further advantageous way of avoiding the problem that irreversible or ambiguous results may occur with certain combinations of speeds and object contents is the use of rolling shutter readout methods in more than one camera in which not only one readout direction occurs , Examples have already been mentioned. In particular, the reading sequence when reading out the image values of the same camera, in particular after reading out a complete camera image, can also be changed.

Alternatively or additionally, readout methods can also be used which combine rolling readout to form a camera image in at least one camera with simultaneous readout of all sensor elements to form a camera image in at least one other camera in which the readout frequencies in the different cameras of the ensemble differ. This can ensure that unique image information is recorded.

Alternatively or additionally, the detection structure can be changed over time, for example by using at least one display, and / or to realize different structural elements by different colors (by reflection and / or emission of radiation). In the latter case, color sensitive cameras are used. However, this is based on the same basic approach as when using cameras or camera areas with different integration time intervals. Temporal variability of the detection structure can be achieved, for example, by virtue of the fact that the detection structure has at least one display which changes a gray scale distribution represented by the display over time. In this case, the state of the gray value distribution can be selected as a function of at least one parameter of the surgical microscopy system. In this way, in the frequency space, the state of the detection structure (in particular the frequency content of the detection structure) can be adapted to the instantaneous speed (intended, for example, by the drive control).

In one embodiment, the continuous gray value distribution can be displayed on at least one self-illuminating display depending on the operating parameter, the pixel raster of the display being particularly small compared to the resolution of the image on the camera image. In particular, the information about the measured speed of the movable part and optionally also the location determined from the camera image can be used to determine, taking into account the planned trajectory of the movement with an optional safety margin for expected path deviations, from the partial area of the camera image to be evaluated. In this way, the controlled variable actual speed and optionally also actual position for the regulation of the movement of the movable part can be generated from individual images.

The information obtained from the speed measurement can in particular also be used to correct the smearing of the detection structure in the camera image. Thus, the resolution and / or the accuracy for a determination of the position of the movable part can be increased by evaluating the camera image.

To stabilize measurements of location and velocity components in the direction of the optical axis or approximately in the direction of the optical axis of the image, stereoscopic images can be used. This does not exclude that more than two cameras, but for example, three or four cameras are used, and their images, for example, taking into account the position and Orientation of the camera to be evaluated together. In this way, depth information can be obtained, that is, a 3D image of the moving part can be created and evaluated. For example, known methods of separating polarized radiation by polarizing filters or separating radiation of different wavelengths (color) can be used. In stereoscopic methods, it is additionally possible to evaluate different camera images and / or partial regions of camera images taking into account different exposure time intervals, as has already been described.

Merely as an example of a particular type of surgical microscopy system, a system having a linear axis and five axes of rotation has previously been described. This total of six axes of motion can be referred to as stacked or cascaded axes of motion, since a movement of each logically superior axis of movement in the stack or the cascade leads to a movement of all subordinate axes in the stack or the cascade. In practice, however, not only stacked axes of motion occur.

So far, in conventional embodiments of the control and / or regulation of the movement at least with respect to a movement axis for the measurement of the movement was based on a coordinate system, which is based on the respective axis. Finds e.g. In an axis stack sequence, a motion measurement relative to an axis that is not the first axis in the stacking sequence, the coordinate system moves relative to the fixed machine coordinate system (which may also be referred to as a laboratory system or world coordinate system) when there is movement of at least one major axis. The measurement of a movement according to the invention now makes it possible in a simple manner to measure the movement of the movable part of a surgical microscopy system with respect to the laboratory coordinate system. For this purpose, the at least one camera (as already described above with examples) can either be arranged at the base of the system and thus fixed in the laboratory coordinate system or the detection structure can be fixedly arranged on the base and the camera can be moved with the moving part. The number of cameras used depends in particular on the type of motion measurement and / or on the design of the system and on the range of motion to be detected.

In particular, therefore, to generate the respective kinematic variable (such as position, speed and / or acceleration) of the movement, a camera image can be repeatedly recorded and the kinematic variable can be determined for each of the camera images. As a result, the value of the kinematic variable can be determined in each case currently for the last recorded camera image. Optionally, the orientation and / or the temporal course of the movement can also be determined repeatedly from one of the camera images. In particular, therefore, local measurement systems on moving parts (e.g., scale divisions and optical read heads on moving parts providing only pulse signals upon detection of a split mark) can be replaced by the camera image evaluation of the present invention. Even tachometers of drives can be replaced by this.

In the case of repeated detection of the detection structure, the detection structure or subregions of the detection structure can be tracked with respect to their relative movement, in particular identified from the various camera images taken one after the other. For example, In this case, a plurality of cameras is used, with which the movement of the same moving part is tracked. This also includes the already mentioned case in that the plurality of cameras is moved along the movable part and detects at least one stationary detection structure.

Overall, therefore, it is particularly possible to carry out the control and / or regulation of the movement of a moving part via at least one corresponding drive (e.g., electric motor) based on at least one kinematic quantity defined in the laboratory coordinate system and measured directly in that coordinate system. A conversion from a co-moving coordinate system in the laboratory coordinate system can therefore be omitted. Also, there is no need for a corresponding calibration by changing the position of the co-moving coordinate system, e.g. depending on the respective operating state (for example with different loading of a moving part of the system or with different temperature levels or temperature distributions).

In particular, the control and / or regulation of the movement of the movable part can be carried out using kinematic variables of the movement which are determined repeatedly from the camera images. In particular, both the position of the movable part and its speed and / or acceleration are determined from each camera image. Since only a single, up-to-date camera image is required for speed determination, the speed measurement value already exists with little delay after the acquisition of a single camera image for controlling and / or regulating the movement. The Response time of the control and / or regulation is therefore lower than in the evaluation of several consecutively recorded camera images. Disadvantages occurring in known controls and / or controls, such as oscillation of the output variable of the control and / or regulation due to delayed determination of the measured variable, can therefore be avoided or at least reduced.

For a simple position controller, it suffices to detect the current actual position (controlled variable) and, accordingly, the manipulated variable, e.g. to determine the target vehicle speed with respect to the axis of motion, based on the control deviation (ie difference between the target position and actual position) and pass it on to the drive train (controlled system).

13 shows an embodiment of a simple position controller. y (t) stands for the controlled variable at the output of the controlled system 71 on which, for example, a disturbance d (t) can act. About a return 73 becomes the controlled variable y (t) of a comparator 75 supplied, which thus compares the actual position with the value of a reference variable w (t) and the comparison result e (t) the controller 77 supplies. At the output of the regulator 77 the manipulated variable u (t) is sent to the controlled system 71 output.

A controller adapted to the controlled system is also able to reduce the control deviation e (t) to a minimum and thus to adjust the position to be controlled to the requested setpoint position w (t), even with conventional detection of the controlled variable at the output of the controlled system Maintain tolerance. How out 13 is apparent, the controller takes into account 77 directly only the position deviation e (t), but not the speed at the current time of execution of the control process and thus not the deviation from the target speed. The current actual speed can indeed be within the controller 77 (eg its transfer function) can be calculated by calculating the difference to an earlier position. In this case, however, outdated information is taken into account, namely the former position from an earlier cycle of execution of the control.

As a result, although the control deviation e (t) is approximately zero, the speed of the movable part at the location of the desired position is not zero. This is especially the case when the controlled system performs a mostly high-frequency mechanical oscillation. Previous regulators compensate this for predetermined frequency ranges or suitable combinations of resonance frequencies of the controller and the controlled system. However, in many cases they can not completely eliminate the effect of vibrations.

It is now proposed to use a control for regulating the movement of a moving part of a surgical microscopy system, of which in addition to the information about the actual position (optionally in the form of a deviation from the target position) also a speed measurement (optionally as a difference between a target speed and an actual speed ) is used. In particular, such a control may be a cascade control, i. a first controller (e.g., the position controller) and a second controller (e.g., the speed controller) are connected in series, i. cascaded. Optionally, the acceleration can also be used by the control as an input variable, in particular in a further cascaded stage of the control as a measured actual variable (possibly as a difference between a desired acceleration and an actual acceleration).

An embodiment of such a cascaded control shows 14 , Therein, s is the position of the moving part, v is the speed of the moving part and a is the acceleration of the moving part. These three, the movement of the movable part descriptive kinematic variables, which are available for the measurement at the output of a corresponding part of the controlled system are also used to form the control deviation with the index "is" for the actual value and the index "soll" for distinguished the setpoint.

While the position s can be measured via conventional position measuring systems of moving devices, such as scale graduation and optical read head, and / or is determined from a single camera image of the motion measuring system according to the invention, the speed is preferably determined directly from a camera image. In contrast, in conventional systems, the speed is determined, for example, by means of an electrical voltage applied to a DC motor and / or by measuring the rotational speed, for example by means of a tachometer. Conventionally, the acceleration a can be determined by the current supplied to the DC motor. Alternatively, at least one acceleration sensor can be arranged on the movable part. While this conventional determination of the acceleration is also possible, it is preferred that the acceleration either directly from the course of Movement is determined during the integration time interval of the camera image and / or from the time course of the speed, which is determined from a sequence of several camera images.

In the 14 Control shown is merely a special embodiment of a scheme by which the speed determined directly from a camera image is used. Alternative embodiments of such a scheme are possible. For example, the acceleration can not be a measured variable and, for example, instead be determined by calculation from the speed or ignored. That in concrete 14 illustrated embodiment, at its input shown on the left on a comparison device, which _is the target position s _soll with the actual position s _is compared and a first controller, the position controller Po, supplying. This controller may be, for example, a proportional controller. At the output of the position controller Po, the output control value, eg, a speed, a further comparison device supplied to _this control value, in particular with the measured actual velocity v and the target velocity v _soll compares and supplies the appropriate control error to a speed controller PID, which, for example, a regulator with proportional -, integral and Differenzialanteil is. At the output of the corresponding manipulated variable of a further comparison device is supplied to, for example, _is an acceleration with the actual acceleration a and the target acceleration a _{is to} compare. The corresponding control deviation is fed to an acceleration controller PI, which may be, for example, a proportional and integral controller. At the output of this acceleration controller PI is the manipulated variable U for the controlled system.

In 14 is the controlled system symbolized by a first PT ₁ member for modeling a time delay at the output of the acceleration a is measurable. A further downstream PT ₁ element models a further delay, at the output of which the speed v can be measured. Yet another element I models an integrating effect of the controlled system, at whose output the position s can be measured.

The 15 to 18 each show the location of a circular marker that is part of a detection structure captured by a recording image at four different times during a single exposure time interval. In each of the embodiments of the 15 to 18 the marker moves in a straight line relative to the camera. The location variable of the straight-line direction is denoted by x. Above the circles filled with hatchings, which represent the marker at the respective location, there is a respective time, which designates the time at which the marker is located at the respective location. In each case, the times t0 (at the beginning of the exposure time interval), the time t0 + τ / 3 (when one third of the exposure time interval has elapsed), the time t0 + 2τ / 3 (when two-thirds of the exposure time interval have elapsed) and the time t0 + τ at the end of the exposure time interval in the 15 to 18 specified.

Underneath is in each of the 15 to 18 a diagram in which over the entire exposure time interval away from the marker emanating and received by the sensor elements of the camera radiation amount S (x) is plotted as a function of the location x. This corresponds to the signal strength of the sensor at the corresponding location in the recording image. At time t0, the marker is located at location x0. However, the marker is not in all of the 15 to 18 at the time of the end of the exposure time interval at the location furthest from the location x0. This is only in the cases of 15 and 16 the case. In contrast, the movement in the cases of the 17 and 18 at time t0 + τ / 3.

Below the diagram, which represents the amount of radiation received as a function of location, is in each 15 to 18 another diagram illustrating the temporal intensity curve I (t) of the radiation emanating from the marker during the exposure time interval, ie the intensity I is a function of the time t. In the case of 15 to 17 the course is constant, ie radiation emanating from the marker during the exposure time interval with a constant radiation intensity. In the case of 18 on the other hand, the intensity I (t) decreases linearly with a constant gradient. In addition, a variant of the temporal intensity curve in 18 indicated by a dashed line. According to the variant, the intensity at the beginning of the exposure time interval is substantially greater and initially falls steeply, in order then to swivel into the course descending with a constant gradient. This corresponds in the representation of the received radiation amount S (x) a dashed line in the region of the location x0.

In the case of 15 The marker moves at constant speed in x-direction. As mentioned, the time course of the radiation intensity is also constant. The radiation amount S (x) received by the camera in the exposure time interval has a symmetrical shape to the center of the location area, where the marker moves during the exposure time interval. Therefore, the movement of the marker can not be distinguished in its orientation from the reverse movement of the marker.

This also applies to the case of 16 in that the marker executes a linear movement in the x-direction slowing down during the exposure time interval. Instead, if the marker performs an accelerating motion in the reverse direction, the same amount of radiation S (x) as a function of location, as in 16 shown arise.

The from the cases of the 15 and 16 The conclusion drawn can be explained mathematically as shown below. In this case, reference is made to the above representation of the gray value distribution GW. However, the convolution shown in equation (1) is used to illustrate the facts of the cases in 15 and 16 shown in a different way. In the following equation (7), the quantities I (t) denote the illumination intensity as a function of the time t and σ the location-dependent image function of the detection structure taken by the camera image. Since it is a relative movement between the detection structure and the camera, the location x changes in the course of time by the path s (t), ie the path s is also a function of time: GW (x) = ∫ ₀ ^T I (t) σ (x + s (t)) dt (7)

If the path s (t) is called variable u, we get: t = s ^-1 (u) => du = v (t) dt (8)

That is, the time t is equal to the inverse function of the path s as a function of the variable u and from this follows the expression on the right in the equations (8) which multiplies the infinitesimal value du of the variable u by the first derivative of the path s times the infinitesimal value dt equated to time t. From this follows again the following transformed expression:

mit:

The expressions in equations (8) and (9) can now be substituted into equation (7) to substitute time t. In this case, a rest of the movement and therefore a change of the direction of movement are not allowed. The first time derivative v of the path, ie the speed, therefore does not become zero:

With:

In this case, the first term k (u), ie the fraction, in the integral in equation (10) can be understood as the convolution kernel and operator which causes the smearing of the detection structure in the recorded recording image. This term will be referred to hereafter as a smearing operator. In the case of constant radiation intensity I (t) = 1, equation (10) simplifies as follows:

From the following equation (12) obtained from the equation (11) by substituting the variable u by its negative variable -w (ie, u = -w and du = -dw), it can be seen that the orientation of the motion in the Case of constant radiation intensity can not be determined. This follows from the equality of the expressions on the right-hand sides of equations (11) and (12), where, because of the substitution, the bounds of integration and the signs in front of the variables u and w are reversed:

Thus, empirically by the cases of 15 and 16 presented facts mathematically proven. However, if the radiation intensity I (t) changes over time, the gray scale distribution GW (x) can be written as in the second row of equation (10) and the radiation intensity I (t ) is not constant. Rather, the smear operator then depends on the time course of the radiation intensity and on the speed of the movement as a function of time, that is, on the time course of the movement. The smear operator can be determined from one or more recorded images.

Once the smear operator has been determined, the time course of the movement, eg the time-dependent speed change, can be determined from this. It can already be seen from the expression of the smear operator in the second line of equation (10) that, given an unknown time profile of the radiation intensity and unknown velocity change, there is an ambiguity between the radiation intensity and the velocity profile. This expression can be reshaped as follows: v (t) = I (t) / k (u)

The expression contains both the radiation intensity I and the velocity v as a function of the time t, that is to say their temporal courses. However, if the time course of the radiation intensity is known, as is the case here, the only question left is with which orientation the relative movement takes place. This ambiguity is expressed in the transformed expression in that the expression is not only the convolution kernel of the integral according to Equation (10), but also the corresponding integral with the direction of inverse integration and the opposite direction of the motion. These two integrals behave the same as the integrals in equations (11) and (12) with the exception of a non-constant radiation intensity I (t) occurring in them.

However, the ambiguity with respect to the orientation can be eliminated even when evaluating a single recording image, if the time course of the radiation intensity is chosen so that in theoretically conceivable reverse direction of movement, a non-realistic speed profile of the movement is the result. For example, it is possible, has already been mentioned above and is based on the variant of 18 to explain that the radiation intensity is chosen to be much larger within a short sub-period at the beginning of the exposure time interval than during the remainder of the exposure time interval. Illustratively, this process can be described as "burning in" of the object scene into the recording image.

However, additional ambiguities with respect to the determination of the orientation and / or the time course of the movement may result when evaluating only a single recorded image, if there is a reversal of the movement during the exposure time interval. This problem can be solved by the duration of the exposure time interval is selected short enough and / or by a separate evaluation of several consecutively recorded recording images an already determined orientation of the movement is confirmed and / or an already determined time course of the movement is plausibly continued. In addition, depending on the masses of the moving parts, information about the inertia of the moving system can be made. Furthermore, taking into account information about the possible driving forces, it can be determined which dynamics the movement can have. In this way, it is possible to exclude unrealistic movement patterns from several conceivable movement progresses which could have taken place during an exposure time interval.

As also mentioned above and as with reference to 19 is briefly explained, additional information can be obtained by a single recording image, for example, if the temporal course of the radiation intensity in different spectral ranges is varied in different ways and / or sensor elements of at least one camera with at least two read-out sequences as already described above.

wird in dem konkreten Beispiel wie bereits beschrieben von lediglich zwei Spektralkomponenten mit den Strahlungsintensitätsverläufen I1(t) und I2(t) ausgegangen:

A concrete case for the different temporal variation of the radiation intensity in different spectral ranges is represented mathematically using the following equations. It is assumed that during the first half of the exposure time interval only the radiation intensity of the first spectral component of the radiation is not equal to zero and in the second half of the exposure time interval only the radiation intensity of the other, second spectral component is not equal to zero. As a result, the smearing of the first spectral component is clearly offset from smearing by the other spectral component. While the following equation (13) in the counter of the smear operator and, accordingly, to the left of the equal sign in the gray value distribution or radiation distribution GW contains only the general index i and thus stands for any integer number of spectral components,

In the concrete example, as already described, only two spectral components with the radiation intensity profiles I1 (t) and I2 (t) are assumed:

This is followed by the equations (15.1) and (15.2) for the intensity distribution of the two spectral components:

According to Equations (15), the first spectral component contributes to the localized radiation distribution only in the first half of the exposure time interval, ie, until time T / 2, and provides the second spectral component only in the second half of the exposure time interval, beginning at time T / 2 , a contribution to the radiation distribution. It is thus clearly possible with which orientation the movement takes place. The sensor elements of the at least one camera, which have received radiation of the first spectral component, have detected the detection structure in the first half of the exposure time interval. By contrast, the sensor elements which received radiation of the second spectral component detected the detection structure in the second half of the exposure time interval. From the arrangement of the sensor elements, therefore, the orientation of the movement can be determined beyond doubt.

There are other ways to implement the principle of obtaining additional information by varying the temporal variation of the spectral components of the radiation. In the already described above and based on 19 explained variation of at least three spectral components λ1, λ2, λ3, the spectral intensities, for example, each have a sinusoidal time course, but with phase offset to each other. In the specific embodiment, the frequency of the intensity change is the same for all three spectral components. Is shown in 19 the intensity profile for the three spectral components in a single exposure time interval. By known mathematical methods, which are known as "phase shift" from the field of detection of three-dimensional surface structures, the orientation and / or the temporal course of the movement can be determined unambiguously from a single recording image if no reversal occurs during the exposure time interval the movement took place. For example, describes Jason Geng in Structured-light 3D Surface Imaging: A Tutorial, Advances in Optics and Photonics 3, 128-160 (2011) doi: 10.1364 / AOP.3000128 , such a mathematical procedure. Other mathematical methods described therein for detecting three-dimensional surface structures can also be transferred to the field of application of motion detection on which the present invention is based.

The situation described above, according to which a temporally constant radiation intensity, in the case of a single marker and a single camera with the same exposure time interval for all sensor elements, does not allow sufficient information for determining the orientation and / or the time profile of the movement from a single recording image, is for another case in 17 shown. As already mentioned, also in the case of 17 a reversal of the movement of the marker takes place. Starting from the top left in 17 At the time t0 of the beginning of the exposure time interval, the marker reaches its maximum distance from location x0 after one third of the duration of the exposure time interval. As the movement progresses during the exposure time interval, it will move in the reverse direction, but at a slower speed than in the first third of the exposure time interval. The resulting irradiation amount S (x) as a function of the location x, which is recorded by the sensor elements, shows qualitatively the in the upper diagram of 17 illustrated course. At the location reached by the marker at the end of the exposure time interval, there is a nearly jump-like increase in the amount of radiation S (x) since the marker has traversed this location and all locations farther from location x0 twice during the exposure time interval.

Even though 18 For the same case, if the movement of the marker shows a constantly decreasing radiation intensity, it can not be unambiguously determined from a single recording image when the marker has moved with which orientation during the exposure time interval. The by a solid line in 18 illustrated course of the received radiation from the sensor elements S (x) is the in 17 shown qualitatively similar. Only the two plateaus of the course are slightly changed and drop to larger values of x in places, with the right plateau initially slightly increasing again towards the maximum x value, because there the reversal of the movement has taken place and the residence time of the marker there is relatively high was long.

However, the variant of the time course of the radiation intensity I (t) shown in dashed lines and the resulting variant of the radiation amount S (x) show the value of the additionally obtained information when the radiation intensity decreases over time with a large rate of change. For the reverse case of an increase with a large rate of change applies accordingly. The radiation intensity is particularly large in the first tenth of the exposure time interval and this leads to a maximum of the radiation quantity near the location x0. Thus, the information is obtained that the marker was most likely located at the beginning of the exposure time interval.

20 shows a pseudo-random variation of the radiation intensity I (t). This can be the total radiation intensity or a different radiation intensity, eg a radiation intensity within a known and / or predetermined spectral range of the radiation. Furthermore, this may alternatively be the radiation intensity of radiation of a specific polarization direction. For example, the radiation intensity within other known and / or predetermined spectral regions of the radiation may also be varied pseudorandomly with different pseudo-random intensity level jumps in the different spectral regions of the radiation. The same applies to radiation components with other polarization directions.

The radiation intensity is constant in each case within sub-time intervals and jumps at the end of the sub-time interval in steps to another intensity level. 20 shows the period of an exposure time interval between the start time t0 and the end time t0 + τ. Within the sub-time interval, there are eight sub-time intervals in the exemplary embodiment in which the radiation intensity in each case has a constant value.

The pseudorandom intensity distribution is not limited to eight sub-time intervals per exposure time interval, but rather may have any other suitable number and duration of sub-time intervals. Also, it is not mandatory that the sub-time intervals are all the same length. For example, in the type of in 18 As a variant of the course of the radiation intensity of a single part-time interval have a much higher value of the radiation intensity than the immediately adjacent part-time intervals or as all sub-time intervals within the same exposure time interval. In this way, in the sense described above, the scene is "burned" into the recording image during this part-time interval.

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited non-patent literature

• "The Non-parametric Sub-pixel Local Point Spread Function Estimation Is a Well-Posed Problem" by Mauricio Delbracio et al., International Journal of Computer Vision (2012) 96, pages 175-195, Springer [0060]
• Jason Geng in Structured-light 3D Surface Imaging: A Tutorial, Advances in Optics and Photonics 3, 128-160 (2011) doi: 10.1364 / AOP.3.000128 [0268]

Claims (17)

Method for operating a surgical microscope system ( 1 ), wherein the surgical microscope system ( 1 ): - a microscope optic ( 21 ), which is designed to image an object area for consideration in an operation, - a movement device ( 18 ), which is designed to the microscope optics ( 21 ) in such a way that a position of the object area is changed, - at least one image recording device ( 23 . 24 ), (a) either with the microscope optics ( 21 ) and upon movement of the microscope optics ( 21 ) is moved, wherein the image recording device ( 23 . 24 ) a separate from the microscope optics ( 21 ) ( 28 . 29 ) when moving the microscope optics ( 21 ) is not moved, detected by at least one recording image, (b) or separately from the microscope optics ( 21 ), upon movement of the microscope optics ( 21 ) is not moved and a collection structure ( 28 . 29 ) through the microscope optics ( 21 ) and / or by at least one fixed with the microscope optics ( 21 ) connected part and / or on the microscope optics ( 21 ) and / or on the at least one fixed to the microscope optics ( 21 ) is detected by at least one recording image, using information about an actual appearance of the detection structure ( 28 . 29 ) from differences of the at least one recording image to the actual appearance of the detection structure ( 28 . 29 ) a speed of relative movement of the image recording device ( 23 . 24 ) and the collection structure ( 28 . 29 ) is determined. The method of claim 1, wherein the recording image is captured during a time interval of detection of the detection structure ( 28 . 29 ) and in the case of the relative movement of the image recording device ( 23 . 24 ) and the collection structure ( 28 . 29 ) an actual appearance of the collection structure ( 28 . 29 ) appears changed due to the relative movement in the recording image. Method according to claim 1 or 2, wherein the information about the actual appearance of the detection structure ( 28 . 29 ) a reference image of the collection structure ( 28 . 29 ) and wherein by evaluating differences of the reference image and the at least one of the image recording device (10) 23 . 24 ) recorded recording image, the speed of the relative movement is determined. Method according to Claim 3, wherein the mathematical convolution of the reference image with the region of the recording image in which the detection structure ( 28 . 29 ), a convolution kernel of the convolution is determined and from the convolution kernel the velocity of the relative movement is determined. The method of claim 4, wherein the convolution kernel is interpreted as a geometric structure whose outer dimensions correspond to the outer dimensions of the reference image and the outer dimensions of the region of the recorded image in which the detection structure ( 28 . 29 ) and wherein the speed of the relative movement of at least one geometric property of a substructure ( 53 . 54 ) of the convolution kernel. The method of claim 5, wherein the recording image and the reference image are two-dimensional images, and wherein an amount and / or a direction of the velocity of the relative movement from a geometry of the substructure ( 54 ) of the convolution kernel. Method according to one of claims 1-6, wherein the detection structure ( 28 . 29 ) is a structure whose spatial function transformed into frequency space has no zero within a frequency range beginning at zero but not including frequency zero and ending at a given maximum frequency.  The method of claim 7, wherein the maximum frequency is set so that it is not smaller than the Nyquist frequency of the camera. Method according to claim 7 or 8, wherein the function value of the spatial function of the detection structure transformed into the frequency domain ( 28 . 29 ) is greater than a predetermined minimum value in the entire frequency range. The method of claim 9, wherein the predetermined minimum value is greater than a statistical fluctuation amplitude of image values of the recorded image caused by the recording of the recorded image and by the determination of the velocity. Method according to one of claims 7-10, wherein the function value of the spatial function of the detection structure ( 28 . 29 ) is constant in the entire frequency range.  A method according to any one of claims 1-11, wherein the recording image comprises pixels whose image values each correspond to an amount of radiation integrated over an exposure time interval, the exposure time intervals of different pixels of the recording image differing from each other, and taking into account information about the different exposure time intervals, the speed of the relative movement and / or information about a time course of the movement is / are determined. Method according to one of claims 1-11, wherein, using the information about an actual appearance of the detection structure ( 28 . 29 ) from differences of the at least one recording image to the actual appearance of the detection structure ( 28 . 29 ) an orientation and / or a temporal course of the relative movement of the image recording device ( 23 . 24 ) and the collection structure ( 28 . 29 ) is determined. Motion measuring system ( 19 ) for a surgical microscopy system ( 1 ) with a microscope optic ( 21 ), which is designed to image an object area for consideration in an operation, wherein the movement measuring system comprises: - at least one image recording device ( 23 . 24 ), (a) during operation with either the microscope optics ( 21 ) and upon movement of the microscope optics ( 21 ) is moved, wherein the image recording device ( 23 . 24 ) a separate from the microscope optics ( 21 ) ( 28 . 29 ) when moving the microscope optics ( 21 ) is not moved, detected by at least one recording image, (b) or during operation separate from the microscope optics ( 21 ), upon movement of the microscope optics ( 21 ) is not moved and a collection structure ( 28 . 29 ) through the microscope optics ( 21 ) and / or by at least one fixed with the microscope optics ( 21 ) connected part and / or on the microscope optics ( 21 ) and / or on the at least one fixed to the microscope optics ( 21 ) is detected by at least one recording image, - a detection device that is configured using information about an actual appearance of the detection structure ( 28 . 29 ) from differences of the at least one recording image to the actual appearance of the detection structure ( 28 . 29 ) a speed of relative movement of the image recording device ( 23 . 24 ) and the collection structure ( 28 . 29 ) to investigate. Motion measuring system ( 19 ) according to claim 14, wherein the determining device ( 4 . 19 ) using the information about an actual appearance of the collection structure ( 28 . 29 ) from differences of the at least one recording image to the actual appearance of the detection structure ( 28 . 29 ) an orientation and / or a temporal course of the relative movement of the relative movement of the image recording device ( 23 . 24 ) and the collection structure ( 28 . 29 ) to investigate. Motion measuring system according to claim 14 or 15, wherein the movement measuring system comprises a movement control ( 4 ), which is designed taking into account the information provided by the determining device ( 4 . 19 ) determined speed a movement device ( 18 ) for moving the microscope optics ( 21 ) to control. Surgical microscopy system ( 1 ), comprising: - the motion measuring system ( 19 ) according to one of claims 14 to 16, - a microscope optics ( 21 ), which is designed to image an object area for consideration in an operation, - a movement device ( 18 ), which is designed to the microscope optics ( 21 ) to move such that a position of the object area is changed.

Images