WO2019091570A1

WO2019091570A1 - Time-resolved examination of a sample by microscopy

Info

Publication number: WO2019091570A1
Application number: PCT/EP2017/078868
Authority: WO
Inventors: Heinrich Spiecker
Original assignee: Lavision Biotec Gmbh
Priority date: 2017-11-10
Filing date: 2017-11-10
Publication date: 2019-05-16
Also published as: EP3707543A1

Abstract

The apparatus for time-resolved examination of a sample (1) by microscopy comprises a sample holder (3), a microscope (5) with a data recording unit (11) for recording data from a recording region (9) defined by the microscope (5), a setting unit (13) for changeably setting a relative position between the recording region (9) and the sample holder (3), and a control device (25) for actuating the setting unit (13) and the data recording unit (11). For the purposes of recording a reference data record, data are recorded in a repeated alternating fashion and the relative position between the recording region (9) and the sample holder (3) is changed. The reference data record is evaluated for characterizing surroundings of a preferred relative position of the recording region (9) in relation to the sample (1). Thereupon, the following steps are carried out multiple times in iterative fashion for the purposes of recording a measurement data record: a) recording data; b) determining a displacement vector between the preferred and a current relative position of the recording region (9) in relation to the sample (1) on the basis of the recorded data and the evaluated reference data record; and c) modifying the relative position between the recording region (9) and the sample holder (3) for the purposes of at least partly compensating the displacement vector.

Description

Time-resolved microscopy of a sample

The present invention relates to the field of time-resolved microscopy. Advantageously, the invention can be applied to the microscopy of a living sample.

Modern microscopy techniques are increasingly being used in vivo. For example, in vivo microscopy is used to study temporal progressions. For example, in the neurosciences experiments are carried out with awake animals, in which the connection between brain activity and behavior is examined. The problem here is that the tissues to be examined do not rest. For example, motion artefacts resulting from the heartbeat or the respiration of the test animal or from muscle contractions may complicate the evaluation of recorded time courses. A movement of the tissue to be examined can take place in all three spatial dimensions.

In some microscopy methods, not only temporal courses are recorded, but the sample is also locally stimulated. For example, in the field of optogenetics, light is stimulated. Since the local stimulation can only be as accurate as the relative position between sample and microscope, the influence of movements of the tissue to be examined is also detrimental here.

Rough suppression of motion artifacts is achieved in practice by fixing the sample. However, a complete suppression of the movements of the sample is generally not possible.

In practice, it is also known to use compensation methods in which a counter-movement is actively generated in order to keep the sample stable. With such compensation methods, the movement of the sample is detected by means of an external sensor and, based on this, the countermovement is generated. A disadvantage of this method is that the sensory measurement of the movement takes place only locally and thus not all components of the movement of the entire system are taken into account. In addition, the measurement of the movement does not take place at the location of the data acquisition, so that relative movements between the measuring point of the external sensor and the receiving area of the microscope are not taken into account.

In addition, it is known from practice to suppress the occurrence of motion artifacts by triggering the recordings. As a trigger signal, for example, an electrocardiogram (ECG) or a clock of a ventilator can be used, which each substantially synchronously with a corresponding movement of the sample. The image acquisition is then always recorded with a fixed relative phase to the cycle of the movement. The disadvantage here is that the image repetition time is limited to the periodicity of the movement pattern, which limits the achievable time resolution. In addition, movements of the sample that do not follow the periodic pattern still cause motion artifacts in the images.

As a post-processing method, it is also known to extract a master image after taking a time-resolved image series. With respect to this master image, a lateral displacement vector is calculated by correlation methods for each frame of the image series. The frames are then shifted backward by this vector. A disadvantage of this method is that a shift perpendicular to the image plane can not be compensated. In addition, the image quality suffers due to the undefined orientation between sample and microscope during the measurement. Since there is no stable relative position between sample and microscope during the measurement time, it is also not possible to carry out stimulation processes in the sample precisely localized.

It is an object of the invention to provide a device and a method for time-resolved microscopy of a sample, which allow the recording of time-resolved data with high quality, even if the sample is not completely stationary or motionless, such. In in vivo microscopy applications.

This object is achieved with respect to the device by the subject matter of claim 1 and with respect to the method by the subject matter of claim 11. The dependent claims indicate advantageous developments of the invention.

A device according to the invention for time-resolved microscopy of a sample comprises a sample holder for receiving the sample, a microscope, an adjustment unit and a control device. The microscope has a data acquisition unit which is designed to record data from a receiving area defined by the microscope. The data acquisition unit may comprise, for example, a CCD camera or a photomultiplier. The setting unit is configured to variably set a relative position between the receiving area and the sample holder. In particular, the setting unit can be designed to change only the position of the receiving area or only the position of the sample holder. Alternatively, the setting unit may be configured to adjust both the position of the receiving area and the position of the sample holder. The controller is configured to drive the setting unit and the data acquisition unit. In particular, the control device may comprise a computing unit and a memory unit with suitable program instructions.

The controller is configured to drive the adjustment unit and the data acquisition unit to acquire a reference data set by repeatedly alternately receiving data by the data acquisition unit and changing the relative position between the acquisition area and the sample holder. For recording the reference data record, the control device thus controls the setting unit and the data recording unit in such a way that data is recorded several times, wherein the relative position between the receiving area and the sample holder is changed in a controlled manner between the individual recordings. It is irrelevant whether as a first step data is recorded or as a first step, the relative position between the receiving area and the sample holder is changed. It is of course possible that between two successive shots, the relative position between the receiving area and the sample holder is changed in several steps. In addition, it is conceivable to take several shots before the relative position between the receiving area and the sample holder is changed again. In addition to the recorded data, in each case the associated relative position between the receiving area and the sample holder can be stored. It may also be sufficient to store in each case the change in the relative position between the receiving area and the sample holder carried out before and / or after a specific recording, or to know it from a program for recording the reference data set, as long as the changes in the relative position are subsequently comprehensible.

The control device is further configured to evaluate the reference data set for characterizing an environment of a preferred relative position of the recording area with respect to the sample after the recording of the reference data record. The preferred relative position of the receiving area with respect to the sample may correspond to the relative position of the receiving area with respect to the sample, in which a recording of the reference data set particularly well represents a region of the sample to be examined. The selection of the preferred relative position of the receiving area with respect to the sample can be done manually or automatically according to respectively desired criteria based on the reference data set, for example by selecting a recording of the reference data set. It is not absolutely necessary that the corresponding mutual spatial relationship between the receiving area and the sample of the preferred relative position is actually known. In order to be able to characterize the environment of the preferred relative position of the recording area with respect to the sample as accurately as possible, it is advantageous if the reference data record has as large a number of recordings as possible which are assigned to different relative positions between the recording area and the sample holder. For example, the reference data set may have more than 20, more than 50, more than 100, more than 150, more than 200, or more than 500 shots.

Since the environment of the preferred relative position of the recording area in relation to the sample is characterized by means of the reference data record, it is advantageous if the reference data record is recorded as quickly as possible. Preferably, the time within which the reference data set is taken is small compared to a characteristic time within which a relevant movement of the sample takes place. In particular, in in vivo microscopy applications, the recording of the reference data set may be small with respect to a periodicity of a pulse beat or respiration of the test subject, for example a factor of at least 20, at least 15, at least 10, or at least 5 smaller. By recording the reference data set sufficiently fast, the accuracy of the characterization of the environment of the preferred relative position of the recording area with respect to the sample can be increased since the influences of the intrinsic movement of the sample on the reference data record are minimized. In practice, it is generally not possible to record the reference data set so fast that effects resulting from movements of the sample will not be reflected in the recordings of the reference data set. The characterization of the environment of the preferred relative position of the receiving area with respect to the sample can therefore be influenced in practice by motion artifacts. Nevertheless, based on the evaluated reference data set, as described below, a position stabilization can take place during a subsequent acquisition of a measurement data set which significantly improves the quality of the recordings of the measurement data set.

According to one embodiment, it would be conceivable to make the recordings of the reference data record triggered, wherein the trigger signal depicts a periodic movement of the sample, for example a heartbeat or a respiration of a test animal. The recordings of the reference data set, according to such an embodiment, could all have a fixed phase with respect to the periodic movement of the sample, so that motion artifacts in the referendum record are further suppressed. The slowing down of the measurement associated with a triggered recording can be accepted when recording the reference data set. The control device is configured to iteratively perform the subsequently described steps a), b) and c) several times after the evaluation of the reference data record for recording a measurement data record.

At step a), the controller controls the data acquisition unit to acquire data.

In step b), a displacement vector between the preferred relative position of the recording area with respect to the sample and a current relative position of the recording area with respect to the sample is determined based on the data recorded in step a) and the evaluated reference data set.

In step c), the setting unit for changing the relative position between the receiving area and the sample holder is controlled in such a way that the displacement vector determined in step b) is at least partially compensated.

Thus, there is always a correction of the relative position between the recording area and the sample holder during recording of the measurement data record between individual recordings. Thus, a movement of the sample, which leads to an undesired shifting of the relative position between the sample and the recording area between successive recordings of the measurement data set, can be at least partially compensated live during the measurement. Therefore, recordings are obtained with greatly increased quality. An evaluation of the data obtained is greatly simplified, since movement artefacts are already suppressed by the corrections of the relative position between the receiving area and the sample holder during the measurement.

With the invention, better results can be achieved compared to a procedure in which an external reference signal is used for movement correction or as a trigger for the measurement recordings, since the motion correction is calculated from the recordings and thus a direct correction takes place without further assumptions on a Relationship between a selected sensor reading or a selected trigger condition and an actual movement of the sample must be made.

It is conceivable, but not mandatory, for steps a), b) and c) to be executed strictly separately in chronological succession. For example, it would be possible for data to be recorded in step a) of an iteration before completion of the adjustment of the relative position between the receiving area and the sample holder in step c) of the preceding step. Iteration takes place in order to be able to comply with an image acquisition rate for a long calculation or traversing time. It would therefore be conceivable that a position correction comes into effect only in a later image, the readjustment of the relative position between the recording area and the sample, so to speak, runs after it. Nevertheless, a motion correction takes place live during the recording of the measurement data record.

According to one embodiment, the control device is configured to control the setting unit when recording the measurement data set in step c) in such a way that the displacement vector between the preferred and the current relative position of the recording area is at least substantially equalized to zero with respect to the sample. This ensures that the relative position between receiving area and sample is kept as constant as possible during the recording of the measured data set. This is particularly suitable for the stabilized recording of a time series, which should always show the same image section of the sample.

In contrast, according to another embodiment, the control device is configured to control the setting unit when recording the measurement data set in step c) in such a way that the displacement vector between the preferred and the current relative position of the recording area with respect to the sample is at least substantially equal to a predetermined offset vector becomes. The offset vector can be different from recording to recording, ie from iteration to iteration of steps a), b) and c). In particular, the amount of offset vector from frame to frame may increase by a fixed increment value while the direction of the offset vector may remain constant. Such an embodiment is particularly suitable for the stabilized recording of a time series, which shows a different image section of the sample from picture to picture.

Mixed forms are also conceivable, for example such that in certain iterations of step c) the displacement vector is compensated to zero and in other iterations of step c) the displacement vector is compensated for a non-zero offset vector.

It is in particular conceivable that a control loop according to a control method known from the prior art, for example a PID method, is carried out from the sequence of the displacement vectors determined during the repeated image recording, with the result that the motion correction is particularly fast or is particularly accurate and a tendency to control vibrations is reduced. It can also be beneficial be to use a model-based control behavior, in which knowledge about the type of movement history are incorporated into the control behavior. Thus, it may be advantageous that the system itself recognizes a regularity from the displacement vectors known from the past, and thus an improved correction of the motion artifacts can be achieved with this a priori knowledge and the currently calculated displacement vector.

According to a simple embodiment, the measurement data set is used for later evaluation of the experiment. The advantage here is that the motion stabilization in such a procedure is based directly on the actual measurement and thus is particularly efficient.

However, it would also be conceivable for an additional measuring channel (hereinafter: experimental channel) to be provided in addition to the measuring channel assigned to the reference data set and the measured data record (in the following: stabilizing channel). The stabilization channel can then be used for motion stabilization and the actual data of the experiment can be obtained via the experimental channel. In particular, in such an embodiment, it is not necessarily required that the recordings of the measurement data record are also stored. Nevertheless, storing the measurement data record is conceivable. The stabilization channel and the experiment channel may be measurement channels of the same microscope and may be formed, for example, as different color channels. It is advantageous here that the stabilization channel can be designed in such a way that it primarily detects non-dynamic signals which originate exclusively from a movement of the sample, but which are relatively insensitive to the dynamics actually to be investigated. By contrast, the experiment channel can be designed such that it primarily detects signals which originate from the dynamic to be investigated.

The adjustment unit may comprise one or more of the following components: a lens drive device designed as a motorized drive device or a piezo-driven focus drive for changing the position of an objective of the microscope, a sample holder drive device for in particular motorized modification of the position of the sample holder, a lens with variably adjustable Focal length, an adaptive mirror in the beam path of the microscope, or a piezo tripod for three-dimensional displacement of the lens of the microscope. Basically, all measures are suitable as parts of the setting unit, which allow a relative displacement of the receiving area relative to the object. The skilled person is beyond the measures mentioned here various other measures available. There are any combinations of said components conceivable. Motorized drive devices are relatively robust and can be easily integrated and controlled. For particularly accurate measurement results may be advantageous in certain applications piezogetriebene adjustment because they are very fast adjustable. If the microscope is designed as a laser scanning microscope, the setting unit can also be designed to add an offset to a control voltage of the scanner mirror or the scanner mirror in order to shift the recording area. In the case of camera-based microscopes, the camera chip or the entire camera or, alternatively, the tube lens can be moved to move the recording area.

According to a simple embodiment, it is sufficient if, by means of the setting unit, the relative position between the receiving area and the sample holder is variable by a relative linear movement along an optical axis of the microscope. This ensures that the individual recordings of the measurement data set always show the desired sample detail in good image quality. However, in order to improve the movement correction, it is advantageous if the setting unit has more degrees of freedom, that is, the relative position between the receiving area and the sample holder is variable in several dimensions. For example, additionally or alternatively to a relative displacement along the optical axis of the microscope, a lateral displacement between the receiving area and the sample holder and / or a tilting or rotating of the relative position between receiving area and sample holder may be possible. As a result, movements of the sample can be compensated, which do not run parallel to the optical axis of the microscope.

According to one embodiment, a receiving area corresponds in each case to a two-dimensional area in space. In particular, the receiving area can lie in a respective focal plane of the microscope. In such an embodiment, the data acquisition unit can have, for example, a CCD camera for recording the reference data record and the measurement data record. However, it would also be conceivable that the microscope is embodied as a laser scanning microscope in which a scanner for scanning the two-dimensional recording area with excitation light is provided and the data recording unit comprises a photomultiplier for recording corresponding signal light. The laser scanning microscope can be eg a confocal microscope or a multiphoton microscope. For example, galvanometric scanners, resonant scanners, polygon scanners or acousto-optical scanners can be used as scanners. In the case of Using acousto-optical scanners, a shift of the recording area in all three dimensions can take place by means of an adaptation of the control parameters of the scanner.

Alternatively, a capture area may correspond to a line or polygon scan path in space, respectively. When talking about a line over polygon scan path, a scan of a sequence of individual points may be included in the space. A change in the position of the recording area in the room can be done by linear displacement of the line or Polygonskanpfads. In particular, the microscope can be embodied as a laser scanning microscope with a scanner that is configured to scan the line or polygon scanning path with excitation light during a recording. The data acquisition unit may comprise a photomultiplier for receiving corresponding signal light. To adjust the recording area, for example, the starting point at which the scanner begins to scan path to be changed.

Mixed forms are also conceivable. For example, when recording the reference data set, a recording area may correspond to a two-dimensional area in space, and when recording the measurement data set, a recording area may correspond to a line or polygon scanning path in space.

The control device may be configured to control the setting unit such that changing the relative position between the receiving area and the sample holder during recording of the reference data set corresponds in each case to a linear displacement of the receiving area relative to the sample holder along an optical axis of the microscope. In such an embodiment, during the recording of the reference data record, a plurality of recordings are taken, each with a mutually shifted focal plane. On the basis of the information thus obtained, the position of the receiving area with respect to the sample can be stabilized during the subsequent recording of the measured data record.

The control device can be configured to control the setting unit and the data recording unit in such a way that a series of line or polygon scans offset from one another is recorded as a reference data record. The line or polygon scans can be linearly offset from each other, in particular in the lateral direction and / or along an optical axis of the microscope. The control device may be configured to control the setting unit such that changing the relative position between the receiving area and the sample holder during recording of the measurement data set comprises at least one displacement of the recording area relative to the sample holder along an optical axis of the microscope.

The measurement data set can correspond to a time-resolved two-dimensional image series.

The measurement data set may correspond to a time-resolved series of lines or polygon scans. The inventive movement correction can suppress unwanted motion artifacts resulting from a change in the position of the line or polygon scan path with respect to the sample due to an intrinsic motion of the sample.

In one embodiment, the control device may be configured to analyze the reference data set by data analysis for at least several of the recordings forming a reference data record between the preferred relative position of the recording area with respect to the sample and a relative position of the recording area associated with the respective recording to determine the sample.

For example, when evaluating the reference data record, it is initially possible to select a recording which corresponds to the preferred relative position of the recording area with respect to the sample. The selection can be made manually or automatically according to basically any aspect. For example, the image can be selected on which a particularly interesting region of the sample is imaged as sharply and completely as possible. Subsequently, for at least several of the remaining recordings of the reference data record, the control device can determine the displacement vector between the preferred relative position of the recording area with respect to the sample and the relative position of the recording area associated with the respective recording with respect to the sample, since the control device knows in which way and the relative position between the receiving area and the sample holder has been changed between the individual recordings of the reference data record by driving the setting unit.

When, after taking a picture of the measurement data set, the displacement vector between the preferred relative position of the recording area with respect to the sample and the current relative position of the recording area with respect to the sample are determined should, the controller can compare the current recording with the individual recordings of the reference data set.

According to a simple embodiment, the control device determines, based on evaluation criteria, a recording of the reference data record whose displacement relative to the preferred relative position of the recording area with respect to the sample best matches the displacement of the current recording with respect to the preferred relative position of the recording area with respect to the sample. This selection of a recording of the reference data record can take place by means of various methods of image processing and in particular based on a correlation method and / or an intensity comparison between the recordings. For controlling the setting unit for changing the relative position between the receiving area and the sample holder for at least partially compensating the current displacement vector before the next recording, the control unit can resort to the displacement vector associated with the selected recording of the reference data set.

By interpolating between the individual recordings of the reference data record and / or smoothing the reference data record, an even better motion compensation can be achieved according to a further developed embodiment. For example, in evaluating the reference data record, the control device could be configured by data analysis from a recording of the reference data record corresponding to a preferred relative position of the recording area with respect to the sample, based on the data of the remaining recordings of the reference data record and the known displacement vectors between the two single images at several feature points to calculate three-dimensional intensity gradients. Based on the intensity gradients, local Taylor developments of intensity could be determined at the feature points.

During the acquisition of the measurement data set, the control unit could minimize a difference between the local Taylor developments of the intensities at the feature points and the intensities of the respective current image at the feature points, for example by means of a least square method, and thus a global estimate for the current displacement vector is obtained.

The estimation of the current displacement vector by the control device may also be iterative. Optionally, the control device for this purpose include a graphics card. The number and selection of feature points can be over the iterations be constant or can also be customized. In particular, it is possible to perform the iteration in different resolution levels, offering a procedure according to a picture pyramid. The image pyramid can be calculated in the currently analyzed image of the measurement data set and in the same way in the reference data set. The iteration can first take place within a magnification stage and then be continued step by step in higher magnification levels of the image pyramid.

Furthermore, there is the possibility of suppressing image noise and smoothing, both the recordings of the reference data record and the recordings of the measured data record. Preferably, smoothing in both data sets is only calculated locally at the feature points. So the smoothing can be done within a short time.

Based on the current estimated displacement vector, the adjustment unit for varying the relative position between the recording area and the sample holder can be actuated to perform motion correction during the recording of the measurement data record.

The invention also relates to a method for time-resolved microscopy of a sample. The device according to the invention is suitable for carrying out the method, designed and configured. Features described with respect to the device can be transferred to the process, and vice versa.

In a method according to the invention for time-resolved microscopy of a sample, the sample is held with a sample holder. By repeated alternately recording data from a data acquisition range defined by a microscope by means of a data acquisition unit of the microscope and changing the relative position between the receiving area and the sample holder, in particular by means of a setting unit, a reference data record is recorded. After recording the reference data record, the reference data record is evaluated, in particular by means of a control device, for characterizing an environment of a preferred relative position of the recording region with respect to the sample. After the evaluation of the reference data set, the following steps a), b) and c) are carried out iteratively several times to record a measured data record.

In step a), data is recorded by means of the data acquisition unit. In step b), a displacement vector between the preferred relative position of the recording area with respect to the sample and a current relative position of the recording area with respect to the sample is determined based on the data recorded in step a) and the evaluated reference data set.

In step c), the relative position between the receiving area and the sample holder for at least partially compensating the current displacement vector is changed.

Advantageous developments of the method according to the invention will become apparent from the above description of the device and from the following description of the figures.

The invention is applicable, for example, to wide-field microscopy, in particular wide-field microscopy with high aperture, fluorescence microscopy, laser scanning microscopy, multiphoton micoscopy and light-sheet microscopy.

In the following, the invention will be explained with reference to embodiments with reference to the figures. Showing:

1 shows a schematic representation of a device for time-resolved microscopy according to an embodiment with a camera-based Datenaufnahme unit;

FIG. 2 shows a schematic illustration for explaining the recording of a reference data record according to an embodiment with a two-dimensional surface in the space as a receiving region; FIG.

3 shows a schematic representation of a device for time-resolved microscopy according to an embodiment in which the microscope is designed as a laser scanning microscope and the adjustment unit comprises a lens drive device;

4 shows a schematic representation of a device for time-resolved microscopy according to an embodiment in which the microscope is designed as a laser scanning microscope and the setting unit comprises a telescope;

Fig. 5 is a schematic representation of a device for time-resolved microscopy according to an embodiment in which the microscope as Laser scanning microscope is formed and the adjustment unit comprises an adaptive mirror;

Fig. 6 is a schematic diagram for explaining the recording of a reference data set according to an embodiment in which the recording area corresponds to a line or polygon scanning path in space; and

7 shows a schematic representation of a device for time-resolved microscopy according to an embodiment in which the microscope is designed as a laser scanning microscope and a further receiving channel is provided.

FIG. 1 shows an apparatus for time-resolved microscopy of a sample 1 according to an embodiment. The device comprises a sample holder 3 for receiving the sample 1. Preferably, the sample 1 can be fixed to the sample holder 3. The device further comprises a microscope 5 with an objective 7. The microscope 5, in particular its objective 7, defines a receiving region 9 of the microscope 5. In the embodiment shown in FIG. 1, the receiving region 9 is a two-dimensional surface in space Focus plane of the microscope 5 is located. The microscope 5 comprises a data recording unit 11 for recording data from the recording area 9.

The apparatus further comprises a setting unit 13 for variably setting a relative position between the receiving area 9 and the sample holder 3. In the illustrated embodiment, the setting unit 13 comprises a sample holder setting unit 15 and a microscope setting unit 17. The sample holder setting unit 15 is configured to to change the position of the sample holder 3 in the room. The sample holder setting unit 15 may include, for example, a sample holder driving means for motorized changing the position of the sample holder 3. The microscope setting unit 17 is designed to change the position of the receiving area 9 in space. In the embodiment shown, the microscope setting unit 17 comprises a piezo tripod 19 for changing the position of the objective 7 of the microscope 5. The piezo tripod 19 may comprise three with respect to the optical axis 21 of the microscope 5 symmetrically arranged piezo elements 23, by means of which Lens 7 is variable in its position. Such a piezo tripod 19 allows precise adjustment of the position of the objective 7. By jointly driving the three piezo elements 23, the objective 7 can be displaced along the optical axis 21 of the microscope 5, whereby the receiving region 9 is likewise displaced along the optical axis 21 , By different activation of the piezo elements 23 or driving Only one or two of the piezo elements 23 can tilt the lens 7, resulting in a shift of the receiving area 9. Alternatively or in addition to the piezo tripod 19, the microscope setting unit 17, for example, also a lens drive means for motorized change in the position of the lens 7 of the microscope 5 include. This could, for example, allow a displacement of the objective 7 along the optical axis 21 of the microscope 5. The additional or alternative provision of a piezo-driven focus drive for changing the position of a focal plane of the microscope 5 would be conceivable.

The time-resolved microscopy apparatus further comprises a control device 25, which is configured to drive the setting unit 13 and the data acquisition unit 11. The control device 25 may include a computing unit 27 and a memory unit 29 with suitable program instructions. In the embodiment shown, the control device 25 for adjusting the relative position between the receiving area 9 and the sample holder 3 can drive both the sample holder setting unit 15 and the microscope setting unit 17. The setting of the relative position between the receiving area 9 and the sample holder 3 can thus be achieved by a combination of a change in the position of the sample holder 3 in the room and a change in the position of the receiving area 9 in the room. But it would also be conceivable that, for example, the sample holder 3 is held stationary and only the receiving area 9 is moved in space. It would also be conceivable to leave the receiving area 9 stationary in the room and to change only the position of the sample holder 3 in the room.

The controller 25 is configured to drive the setting unit 13 and the data acquisition unit 11 to receive a reference data set. For this purpose, data are repeatedly taken from the receiving area 9 alternately alternately by the data recording unit 11, and then the relative position between the receiving area 9 and the sample holder 3 is changed by means of the setting unit 13. Preferably, the reference data set comprises more than 20, more than 50, more than 100, more than 150, more than 200 or more than 500 recordings thus obtained, each corresponding to different relative positions between the receiving area 9 and the sample holder 3. As explained below, the reference data set is used to suppress in a sequence data set movement of the sample 1, such as a movement due to a heartbeat or respiration, resulting motion artifacts. So that the reference data set itself is largely unaffected by such movements of the sample 1, For example, the measurement of the reference data set could take place within a time period which is small compared to the time scale relevant to the respective movement of the sample 1.

Figure 2 shows a schematic representation of an example of different relative positions between the sample holder 3 and the receiving area 9 for the individual recordings of a reference data set. The respective receiving areas 9 are in this case areal areas of a focus plane of the microscope 5 whose relative position with respect to the sample holder 3 has been changed from recording to recording of the reference data record by means of the setting unit 13. For the purposes of illustration, it is assumed that the reference data record is recorded sufficiently quickly that the position of the sample 1 with respect to the sample holder 3 can be regarded as constant during the recording of the reference data record. This is not necessarily the case. FIG. 2 shows a group of relative positions of the receiving region 9 with respect to the sample holder 3, which is obtained by a linear displacement of the receiving region 9 relative to the sample holder 3 along the optical axis 21 of the microscope 5. Other relative positions could also be provided, in which tilt positions are present between the receiving area 9 and the sample holder 3.

After receiving the reference data record, the control device 25 is configured to evaluate the reference data record for characterizing an environment of a preferred relative position of the receiving region 9 with respect to the sample 1.

The control device 25 is configured to iteratively perform the steps a), b) and c) for recording a measurement data set several times after the evaluation of the reference data record. In step a), the data acquisition unit 11 is driven to receive data. In step b), a displacement vector between the preferred relative position of the receiving area 9 with respect to the sample 1 and a current relative position of the receiving area 9 with respect to the sample 1 is then determined based on the data recorded in step a) and the evaluated reference data record.

In step c), the setting unit 13 for changing the relative position between the receiving area 9 and the sample holder 3 is actuated for at least partial compensation of the displacement vector determined in step b). By iteratively carrying out steps a), b) and c), during the recording of the measurement data set, movement artifacts resulting from a movement of the sample 1 can be suppressed by tracking the relative position between recording area 9 and sample 1. For applications in which a stabilized time series is to be recorded, which always shows the same image section of the sample 1, the adjustment unit 13 can be controlled in step c) such that the displacement vector between the preferred and the current relative position of the receiving region 9 with respect to Sample 1 is at least substantially equalized to zero. However, the invention can also be used for the stabilized recording of a time series in which a different image detail of the sample 1 is recorded from one image to another. For example, it is possible to scan through sample 1 in this way. For this purpose, the setting unit 13 can be controlled in step c) so that the displacement vector between the preferred and the current relative position of the receiving area 9 with respect to the sample 1 is not compensated to zero but to a predetermined offset vector. The offset vector may be different from shot to shot. In particular, the offset vector can be changed from one recording to the next by one increment.

The evaluation of the reference data record by the control device 25 and the determination of the displacement vector between the preferred and the current relative position of the recording region 9 with respect to the sample 1 in step b) can be carried out in various ways.

According to a simple embodiment, the control device 25 for evaluating the reference data record from the recordings of the reference data record can select a preferred recording, which then corresponds to the preferred relative position between the recording region 9 and the sample 1. This selection could also be based on user input. It is not absolutely necessary to know which actual relative position exists between the receiving area 9 and the sample 1 during the recording of the corresponding recording of the reference data set. However, the associated relative position between receiving area 9 and sample holder 3 is known. The control device 25 can then determine displacement vectors between the preferred relative position of the receiving region 9 with respect to the sample 1 and the relative positions of the receiving region 9 associated with the remaining images of the reference data set with respect to the sample 1 and thus characterize the environment of the preferred relative position. For this purpose, the control device 25 accesses the known relative movements between the receiving area 3 and the sample holder 1 during the recording of the reference data set. In step b) when recording the measured data set, the control device 25 can determine, for example by means of correlative methods by comparing the recorded in step a) recording with the recordings of the reference data record a record of the reference data set, which was made at a similar relative position between receiving area 9 and sample 1 like the picture taken in step a). The displacement vector between the preferred relative position of a recording area 9 with respect to the sample 1 and the current relative position of the recording area 9 with respect to the sample 1 can then be considered as the displacement vector between the recording of the reference data record corresponding to the preferred relative position and the selected other recording of the reference data set are determined.

According to another variant, the control device 25 could first also select from the recordings of the reference data record a preferred recording which corresponds to the preferred relative position between the recording area 9 and the sample 1. According to this variant, the control device 25 can be configured to analyze the reference data record by means of data analysis from the recording of the reference data record which corresponds to the preferred relative position of the recording area with respect to the sample, based on the data of the remaining recordings of the reference data record and the known data Displacement vectors between the individual images at several feature points to calculate three-dimensional intensity gradients. Based on the intensity gradients, local Taylor evolution of intensity could be determined at the feature points.

In step b) during the acquisition of the measurement data set, a difference between the local Taylor developments of the intensities at the feature points and the intensities of the respectively current image at the feature points could be minimized by the control unit 25, for example by means of a least square method, and thus obtaining a global estimate for the current displacement vector. Based on the current estimated displacement vector, the setting unit 13 could then be actuated in step c) for changing the relative position between the receiving area 9 and the sample holder 3 in order to perform a motion correction live.

Since, according to the invention, motion compensation takes place while the measurement data record is being recorded, the measurement data set itself represents a motion-stabilized, evaluable data record and can be used for later evaluation of the corresponding experiment. It is therefore in principle sufficient if the microscope 5 diglich has a single measuring channel, which is used for recording the reference data set and the measurement data set. For this purpose, the data acquisition unit 1 1 may comprise, for example, a first CCD camera 31.

However, it would also be conceivable that the measuring channel assigned to the reference data set and the measured data set is provided merely as a stabilization channel and an additional measuring channel (hereinafter: experimental channel) is provided for the actual experimental data or additional experimental data. The stabilization channel can then be used for motion stabilization as described above. During the acquisition of the measurement data set in the stabilization channel, further experimental data can be recorded via the experiment channel. The experimental channel may be coupled to the stabilizing channel such that the relative position between the receiving region 9 and the sample 1 is identical for both channels. By way of example, the data acquisition unit 11 in addition to the first CCD camera 31 can have a second CCD camera 33 which is assigned to the experiment channel. It would also be conceivable, for example, for the data acquisition unit 11 to have a photomultiplier associated with the experiment channel. The stabilization channel and the experiment channel can be designed, for example, as different color channels. The stabilization channel is preferably designed such that it primarily detects non-dynamic signals which originate exclusively from an intrinsic movement of the sample 1, but which are relatively insensitive to the dynamics actually to be investigated. In contrast, the experiment channel can be designed to primarily detect signals originating from a dynamic to be investigated. It would also be conceivable, for example, for a line scan or polygon scan to be performed in the sample 1 via the experiment channel, while two-dimensional images are recorded in the stabilization channel.

FIGS. 3, 4 and 5 show devices for time-resolved microscopy of a sample 1 according to further embodiments. These embodiments differ from the embodiment of FIG. 1 in that the microscope 5 is designed as a laser scanning microscope. The microscope 5 here comprises a scanner 35 with which a scan path as a receiving region 9 can be scanned through the microscope 5. For this purpose, the scanner 35 is preferably controlled by the control device 25 and may comprise one or more, in particular two, scanner mirrors 36. With regard to the features and modifications described in FIG. 1, it is possible to transfer to the embodiments of FIGS. 3, 4 and 5, and vice versa. According to the embodiments of FIGS. 3, 4 and 5, the setting unit 13 for variably setting a relative position between the receiving area 9 and the sample holder 3 again comprises a microscope setting unit 17 and a sample holder setting unit 15. However, it would also be conceivable to use only the microscope Adjustment unit 17 or the sample holder setting unit 15 provide. The embodiments illustrated in FIGS. 3, 4 and 5 differ from one another in terms of the configuration of the microscope setting unit 17. In the embodiment shown in FIG. 3, the microscope setting unit 17 comprises an objective drive device 37 for displacing the objective 7 along the optical axis 21 of the microscope 5. The lens drive device 37 may be provided, for example, as a motorized drive device or as a piezo-driven focus drive. FIG. 4 shows an embodiment in which the objective 7 is stationary and the microscope setting unit 17 is instead provided as a motorized telescope 39. By moving a lens 41 of the motorized telescope 39, a focus of the microscope 5 along the optical axis 21 of the microscope 5 can be moved. FIG. 5 shows a further possibility of how the microscope setting unit 17 can alternatively be formed. In this embodiment, an adaptive mirror 43 is provided with which a scanning deposit of the microscope 5, in particular along the optical axis 21 of the microscope 5 and / or laterally displaceable. Via a control signal from the control device 25, a mirror surface of the adaptive mirror 43 can be curved convexly or concavely. In this way, the focus of the excitation beam 45 can be changed in particular without astigmatism in its position. In addition or as an alternative to the variants of the microscope setting unit 17 shown, it would also be conceivable that the setting unit 25 is designed to add an offset to a control voltage of the scanner mirror 36 or the scanner mirror 36 in order to displace the receiving area 9.

The microscopes 5 of the embodiments shown in FIGS. 3, 4 and 5 each have a data acquisition unit 11 in the form of a photomultiplier 47 for recording data from the respective recording area 9. For a recording, the control device 25 controls the scanner 35 in each case to scan a predefined scan path, which corresponds to the recording area 9. If the relative position between the receiving region 9 and the sample holder 3 is to be changed, a starting point of the scanning path is displaced relative to the sample holder 3, for example, by means of the microscope setting unit 17 and / or the sample holder setting unit 15.

Also in the embodiments shown in Figures 3, 4 and 5, the receiving area 9, similar to the embodiment shown in Figure 1, as a two-dimensional surface present in the room. For this purpose, the control device 25 can control the scanner 35 to scan a two-dimensional surface when taking a picture of the reference data record and recording a picture of the measured data record. The evaluation of the reference data set and the stabilization of the measured data set can be carried out analogously to the embodiment described in FIG.

However, according to the embodiments of FIGS. 3, 4 and 5, it is also conceivable to select a line or polygonal scanning path in each case. Also in this case, the control device 25 controls the setting unit 13 and the data recording unit 1 1 to record a reference data set by repeating, alternately picking up data by the data recording unit 11 and changing the relative position between the recording area 9 and the sample holder 3. To record a recording of the reference data set, the line or polygonal scanning path representing the recording area 9 is scanned by means of the scanner 35, the recording being recorded by means of the photomultiplier 47 in the meantime. Between two successive such recordings of the reference data record, the position of the starting point of the line or polygonal scanning path with respect to the sample holder 3 is changed by means of the setting unit 13, that is to say the relative position between receiving area 9 and sample holder 3 is changed. FIG. 6 shows, in a schematic perspective view, the relative positions between individual receiving areas 9, which are designed as line or polygonal scanning paths, and the sample holder 3, which are each assigned to a recording of the reference data record. In the illustration shown, the receiving areas 9 are shifted relative to one another along the optical axis 21 of the microscope 5. In addition or alternatively, receiving areas 9 could also be traversed, which are displaced laterally, for example, with respect to the receiving areas 9 shown or are tilted with respect to them.

After the recording of the reference data set, the control device 25 evaluates the reference data record for characterizing an environment of a preferred relative position of the recording region 9 with respect to the sample 1. For this purpose, the components of the intensity gradients for each point of the line or Polygonscanpfads can be calculated, for example, based on a recording which corresponds to a preferred position of the receiving area 9 with respect to the sample 1, by simply subtraction with the other recordings of the reference data set. With knowledge of the intensity gradients, feature points can then be determined from the line or polygonal scan path. For the selection of the feature points z. B. a histogram of the components of the intensity gradients are analyzed, the feature points are preferably selected can be that the global aperture problem is solved and no feature points are selected with strong image noise. In the simplest case, however, all points of the line or polygon scan path (or else of a two-dimensional image scan path) can also be included for the stabilization. Based on the intensity gradients, local Taylor evolutions of intensity could be determined at the feature points or at all points of the line or polygon scan path (or even a two-dimensional image scan path).

After the evaluation of the reference data record, the control device 25 carries out the steps a), b) and c) iteratively several times to record a measured data record.

In step a), the data acquisition unit 11 is driven to record data. In this case, the line or polygonal scanning path representing the recording area 9 is traced by means of the scanner 35, the recording being recorded by means of the photomultiplier 47 in the meantime.

In step b), a displacement vector between the preferred relative position of the receiving region 9 with respect to the sample 1 and a current relative position of the receiving region 9 with respect to the sample 1 is determined. This is done based on the data recorded in step a) and the evaluated reference data record. During the acquisition of the measurement data set, the control unit could minimize a difference between the local Taylor developments of the intensities at the feature points and the intensities of the respective current image at the feature points, for example by means of a least square method, and thus a global estimate for the current displacement vector is obtained. This could, as explained above, also be iterative, for example by calculating a picture pyramid.

In step c), the setting unit 13 for changing the relative position between the receiving area 9 and the sample holder 3 is actuated for at least partially compensating the displacement vector.

It would also be hybrid forms between embodiments in which the receiving area 9 is always a two-dimensional surface, and embodiments in which the receiving area 9 is always a line or Polygonscanpfad conceivable. For example, during the recording of the reference data record, a recording area 9 could each correspond to a two-dimensional area in the room and then during recording of the measurement data set a recording area 9 correspond in each case to a line or polygon scanning path in the room. So could be used as a reference data set for stabilization even when recording a line or Polygonscanpfads as a measurement data set on two-dimensional areal recordings.

FIG. 7 shows a schematic representation of a device for time-resolved microscopy according to a further embodiment, in which the microscope 5 is designed as a laser scanning microscope. In its basic structure, the device according to this embodiment is identical except for the differences explained below or analogous to the embodiments of Figures 3, 4 and 5. The microscope setting unit 17 is shown in Figure 7 as a piezo tripod 19 for moving the focal plane of the lens 7, but could for example also be designed as described with regard to FIGS. 3, 4 and 5.

In contrast to the embodiments shown in FIGS. 3, 4 and 5, the device according to FIG. 7 comprises, in addition to the measurement channel for recording the reference data set and the measurement data set (stabilization channel), a further measurement channel (experimental channel) for recording (additional) experimental data. The stabilization channel may be used, for example, as described with reference to Figures 3, 4 and 5, for motion stabilization and the actual data of the experiment or additional data may be obtained via the experimental channel. Since the stabilization channel and the experiment channel are measurement channels of the same microscope 5, motion-stabilized measurement data are obtained via the experiment channel without additional effort if the measurement is performed simultaneously with the motion-stabilized recording of the measurement data set via the stabilization channel.

In addition to the scanner 35 of the stabilization channel, in the arrangement according to FIG. 7 the microscope 5 comprises a further scanner 49 to which a split-off part of the excitation beam 45 is supplied. The further scanner 49 can in turn have one or more, in particular two, scanner mirrors 51 with which it can scan a one-dimensional or a two-dimensional scan pattern independently of the scanner 35 of the stabilization channel.

In this embodiment, the data acquisition unit 11 includes, in addition to the photomultiplier 47 assigned to the stabilization channel, a further photomultiplier 53, which is assigned to the experiment channel. By means of a spectral separation element, in particular a dichroic mirror 55, the signal radiation from the specimen 1 can be spectrally spectrally transmitted to the photomultiplier 47 of the stabilization channel and the photomultiplier 53 of the experiment. be divided. The spectral distribution can be such that the stabilization channel primarily detects non-dynamic signals, which mainly result from a movement of the sample, but is relatively insensitive to the dynamics actually to be investigated. By contrast, the experiment channel can primarily detect signals that originate from the dynamics to be investigated.

Claims

claims

An apparatus for time-resolved microscopy of a sample (1), comprising: a sample holder (3) for receiving the sample (1); a microscope (5) having a data acquisition unit (1 1) which is adapted to receive data from a receiving area (9) defined by the microscope (5); an adjusting unit (13) for variably setting a relative position between the receiving area (9) and the sample holder (3); and control means (25) configured to drive the setting unit (13) and the data receiving unit (11), characterized in that the control means (25) is configured to drive the setting unit (13) and the data receiving unit (11) thereto to record a reference data set by repeatedly alternately receiving data by the data acquisition unit (11) and changing the relative position between the receiving area (9) and the sample holder (3), the control device (25) being configured to record the reference data record for characterization after the reference data set has been acquired an environment of a preferred relative position of the receiving area (9) with respect to the sample (1) to evaluate; and the control device (25) is configured, after the evaluation of the reference data record, to iteratively perform the following steps several times to record a measurement data record:

Driving the data acquisition unit (11) to record data;

Determining a displacement vector between the preferred relative position of the imaging region (9) with respect to the sample (1) and a current relative position of the imaging region (9) relative to the sample (1) based on the acquired data and the evaluated reference data set; and

Activating the setting unit (13) for changing the relative position between the receiving area (9) and the sample holder (3) for at least partially compensating the displacement vector.

2. Device according to claim 1, wherein the setting unit (13) comprises one or more of the following components: a lens drive device (37) designed as a motorized drive device or as a piezo-driven focusing drive for changing the position of an objective (7) of the microscope (5). a sample holder drive device (15) for motorized modification of the position of the sample holder (3), a lens (41) with variably adjustable focal length, an adaptive mirror (43) in the beam path of the microscope (5) or a piezo tripod (19) for displacement of the objective (7) of the microscope (5).

3. Device according to one of the preceding claims, wherein a receiving area (9) each corresponds to a two-dimensional surface in space.

4. Device according to one of the preceding claims, wherein a receiving area (9) each corresponds to a line or Polygonscanpfad in space.

5. Device according to one of the preceding claims, wherein the control device (25) is configured to control the setting unit (13) such that changing the relative position between the receiving area (9) and the sample holder (3) when receiving the reference data set one each linear displacement of the receiving area (9) relative to the sample holder (3) along an optical axis (21) of the microscope (5).

6. Device according to one of the preceding claims, wherein the control device (25) is configured to control the setting unit (13) and the data acquisition unit (1 1) such that a series of mutually displaced line or polygon scans is recorded as a reference data record.

7. Device according to one of the preceding claims, wherein the control device (25) is configured to control the setting unit (13) such that the changing of the relative position between the receiving area (9) and the sample holder (3) when recording the measured data set at least one Displacement of the receiving area (9) relative to the sample holder (3) along an optical axis (21) of the microscope (5).

8. Device according to one of the preceding claims, wherein the measurement data set corresponds to a time-resolved two-dimensional image series.

9. Device according to one of the preceding claims, wherein the measurement data set corresponds to a time-resolved series of line or polygon scans.

10. Device according to one of the preceding claims, wherein the control device (25) is configured, when evaluating the reference data record by means of data analysis, starting from a recording of the reference data record which corresponds to the preferred relative position of the receiving region (9) with respect to the sample (1). corresponds to calculate three-dimensional intensity gradients at several feature points based on the data of the other recordings of the reference data set and known displacement vectors between the individual recordings of the reference data record.

A method of time-resolved microscopy of a sample (1) comprising: holding the sample (1) with a sample holder (3);

Recording a reference data set by repeatedly alternately picking up data from a receiving area (9) defined by a microscope (5) by means of a data acquisition unit (11) of the microscope (5) and changing the relative position between the receiving area (9) and the sample holder (3); after recording the reference data record, evaluating the reference data record to characterize an environment of a preferred relative position of the recording area (9) with respect to the sample (1); and after evaluating the reference data set, iteratively performing the following steps to record a measurement data set:

Receiving data by the data acquisition unit (11);

Changing the relative position between the receiving area (9) and the sample holder (3) for at least partially compensating the displacement vector.

12. The method of claim 1 1, wherein changing the relative position between the receiving area (9) and the sample holder (3) when recording the reference data set in each case a linear displacement of the receiving area (9) relative to the sample holder (3) along an optical axis ( 21) of the microscope (5), so that the reference data set is recorded as a picture stack.

13. The method of claim 1 1 or 12, wherein a series of mutually displaced line or Polygonscans is recorded as a reference data record.

14. The method according to any one of claims 11 to 13, wherein a receiving area (9) each corresponds to a two-dimensional area in space or a line or Polygonscanpfad in space.

15. The method according to claim 11, wherein during the evaluation of the reference data record by means of data analysis for each of a plurality of recordings forming the reference data record a displacement vector between the preferred position of the recording area (9) with respect to the sample (1) and one of the recordings associated relative position of the receiving area (9) with respect to the sample (1) is determined.