WO2019091570A1 - Microscopie à résolution temporelle d'un échantillon - Google Patents

Microscopie à résolution temporelle d'un échantillon Download PDF

Info

Publication number
WO2019091570A1
WO2019091570A1 PCT/EP2017/078868 EP2017078868W WO2019091570A1 WO 2019091570 A1 WO2019091570 A1 WO 2019091570A1 EP 2017078868 W EP2017078868 W EP 2017078868W WO 2019091570 A1 WO2019091570 A1 WO 2019091570A1
Authority
WO
WIPO (PCT)
Prior art keywords
sample
relative position
recording
reference data
receiving area
Prior art date
Application number
PCT/EP2017/078868
Other languages
German (de)
English (en)
Inventor
Heinrich Spiecker
Original Assignee
Lavision Biotec Gmbh
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Lavision Biotec Gmbh filed Critical Lavision Biotec Gmbh
Priority to PCT/EP2017/078868 priority Critical patent/WO2019091570A1/fr
Priority to EP17803838.6A priority patent/EP3707543A1/fr
Publication of WO2019091570A1 publication Critical patent/WO2019091570A1/fr

Links

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/365—Control or image processing arrangements for digital or video microscopes
    • G02B21/367—Control or image processing arrangements for digital or video microscopes providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/0004—Microscopes specially adapted for specific applications
    • G02B21/002—Scanning microscopes
    • G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
    • G02B21/008—Details of detection or image processing, including general computer control
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/24—Base structure
    • G02B21/241—Devices for focusing
    • G02B21/244—Devices for focusing using image analysis techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/207—Analysis of motion for motion estimation over a hierarchy of resolutions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B2207/00—Coding scheme for general features or characteristics of optical elements and systems of subclass G02B, but not including elements and systems which would be classified in G02B6/00 and subgroups
    • G02B2207/114—Two photon or multiphoton effect
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10056—Microscopic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20172—Image enhancement details
    • G06T2207/20182—Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering

Definitions

  • the present invention relates to the field of time-resolved microscopy.
  • the invention can be applied to the microscopy of a living sample.
  • Modern microscopy techniques are increasingly being used in vivo.
  • in vivo microscopy is used to study temporal progressions.
  • in the neurosciences experiments are carried out with awake animals, in which the connection between brain activity and behavior is examined.
  • the problem here is that the tissues to be examined do not rest.
  • motion artefacts resulting from the heartbeat or the respiration of the test animal or from muscle contractions may complicate the evaluation of recorded time courses.
  • a movement of the tissue to be examined can take place in all three spatial dimensions.
  • a trigger signal for example, an electrocardiogram (ECG) or a clock of a ventilator can be used, which each substantially synchronously with a corresponding movement of the sample.
  • ECG electrocardiogram
  • the image acquisition is then always recorded with a fixed relative phase to the cycle of the movement.
  • the disadvantage here is that the image repetition time is limited to the periodicity of the movement pattern, which limits the achievable time resolution.
  • movements of the sample that do not follow the periodic pattern still cause motion artifacts in the images.
  • a post-processing method it is also known to extract a master image after taking a time-resolved image series. With respect to this master image, a lateral displacement vector is calculated by correlation methods for each frame of the image series. The frames are then shifted backward by this vector.
  • a disadvantage of this method is that a shift perpendicular to the image plane can not be compensated.
  • the image quality suffers due to the undefined orientation between sample and microscope during the measurement. Since there is no stable relative position between sample and microscope during the measurement time, it is also not possible to carry out stimulation processes in the sample precisely localized.
  • the microscope has a data acquisition unit which is designed to record data from a receiving area defined by the microscope.
  • the data acquisition unit may comprise, for example, a CCD camera or a photomultiplier.
  • the setting unit is configured to variably set a relative position between the receiving area and the sample holder.
  • the setting unit can be designed to change only the position of the receiving area or only the position of the sample holder.
  • the setting unit may be configured to adjust both the position of the receiving area and the position of the sample holder.
  • the controller is configured to drive the setting unit and the data acquisition unit.
  • the control device may comprise a computing unit and a memory unit with suitable program instructions.
  • the controller is configured to drive the adjustment unit and the data acquisition unit to acquire a reference data set by repeatedly alternately receiving data by the data acquisition unit and changing the relative position between the acquisition area and the sample holder.
  • the control device thus controls the setting unit and the data recording unit in such a way that data is recorded several times, wherein the relative position between the receiving area and the sample holder is changed in a controlled manner between the individual recordings. It is irrelevant whether as a first step data is recorded or as a first step, the relative position between the receiving area and the sample holder is changed. It is of course possible that between two successive shots, the relative position between the receiving area and the sample holder is changed in several steps.
  • the control device is further configured to evaluate the reference data set for characterizing an environment of a preferred relative position of the recording area with respect to the sample after the recording of the reference data record.
  • the preferred relative position of the receiving area with respect to the sample may correspond to the relative position of the receiving area with respect to the sample, in which a recording of the reference data set particularly well represents a region of the sample to be examined.
  • the selection of the preferred relative position of the receiving area with respect to the sample can be done manually or automatically according to respectively desired criteria based on the reference data set, for example by selecting a recording of the reference data set. It is not absolutely necessary that the corresponding mutual spatial relationship between the receiving area and the sample of the preferred relative position is actually known.
  • the reference data record has as large a number of recordings as possible which are assigned to different relative positions between the recording area and the sample holder.
  • the reference data set may have more than 20, more than 50, more than 100, more than 150, more than 200, or more than 500 shots.
  • the reference data record is recorded as quickly as possible.
  • the time within which the reference data set is taken is small compared to a characteristic time within which a relevant movement of the sample takes place.
  • the recording of the reference data set may be small with respect to a periodicity of a pulse beat or respiration of the test subject, for example a factor of at least 20, at least 15, at least 10, or at least 5 smaller.
  • the accuracy of the characterization of the environment of the preferred relative position of the recording area with respect to the sample can be increased since the influences of the intrinsic movement of the sample on the reference data record are minimized.
  • the characterization of the environment of the preferred relative position of the receiving area with respect to the sample can therefore be influenced in practice by motion artifacts. Nevertheless, based on the evaluated reference data set, as described below, a position stabilization can take place during a subsequent acquisition of a measurement data set which significantly improves the quality of the recordings of the measurement data set.
  • the recordings of the reference data record triggered wherein the trigger signal depicts a periodic movement of the sample, for example a heartbeat or a respiration of a test animal.
  • the recordings of the reference data set could all have a fixed phase with respect to the periodic movement of the sample, so that motion artifacts in the referendum record are further suppressed.
  • the slowing down of the measurement associated with a triggered recording can be accepted when recording the reference data set.
  • the control device is configured to iteratively perform the subsequently described steps a), b) and c) several times after the evaluation of the reference data record for recording a measurement data record.
  • the controller controls the data acquisition unit to acquire data.
  • step b) a displacement vector between the preferred relative position of the recording area with respect to the sample and a current relative position of the recording area with respect to the sample is determined based on the data recorded in step a) and the evaluated reference data set.
  • step c) the setting unit for changing the relative position between the receiving area and the sample holder is controlled in such a way that the displacement vector determined in step b) is at least partially compensated.
  • steps a), b) and c) are executed strictly separately in chronological succession.
  • data it would be possible for data to be recorded in step a) of an iteration before completion of the adjustment of the relative position between the receiving area and the sample holder in step c) of the preceding step. Iteration takes place in order to be able to comply with an image acquisition rate for a long calculation or traversing time. It would therefore be conceivable that a position correction comes into effect only in a later image, the readjustment of the relative position between the recording area and the sample, so to speak, runs after it. Nevertheless, a motion correction takes place live during the recording of the measurement data record.
  • control device is configured to control the setting unit when recording the measurement data set in step c) in such a way that the displacement vector between the preferred and the current relative position of the recording area is at least substantially equalized to zero with respect to the sample. This ensures that the relative position between receiving area and sample is kept as constant as possible during the recording of the measured data set. This is particularly suitable for the stabilized recording of a time series, which should always show the same image section of the sample.
  • control device is configured to control the setting unit when recording the measurement data set in step c) in such a way that the displacement vector between the preferred and the current relative position of the recording area with respect to the sample is at least substantially equal to a predetermined offset vector becomes.
  • the offset vector can be different from recording to recording, ie from iteration to iteration of steps a), b) and c).
  • the amount of offset vector from frame to frame may increase by a fixed increment value while the direction of the offset vector may remain constant.
  • Such an embodiment is particularly suitable for the stabilized recording of a time series, which shows a different image section of the sample from picture to picture.
  • step c) the displacement vector is compensated to zero and in other iterations of step c) the displacement vector is compensated for a non-zero offset vector.
  • a control loop according to a control method known from the prior art is carried out from the sequence of the displacement vectors determined during the repeated image recording, with the result that the motion correction is particularly fast or is particularly accurate and a tendency to control vibrations is reduced.
  • a model-based control behavior in which knowledge about the type of movement history are incorporated into the control behavior.
  • the system itself recognizes a regularity from the displacement vectors known from the past, and thus an improved correction of the motion artifacts can be achieved with this a priori knowledge and the currently calculated displacement vector.
  • the measurement data set is used for later evaluation of the experiment.
  • the advantage here is that the motion stabilization in such a procedure is based directly on the actual measurement and thus is particularly efficient.
  • an additional measuring channel hereinafter: experimental channel
  • the stabilization channel can then be used for motion stabilization and the actual data of the experiment can be obtained via the experimental channel.
  • the recordings of the measurement data record are also stored.
  • storing the measurement data record is conceivable.
  • the stabilization channel and the experiment channel may be measurement channels of the same microscope and may be formed, for example, as different color channels.
  • the stabilization channel can be designed in such a way that it primarily detects non-dynamic signals which originate exclusively from a movement of the sample, but which are relatively insensitive to the dynamics actually to be investigated.
  • the experiment channel can be designed such that it primarily detects signals which originate from the dynamic to be investigated.
  • the adjustment unit may comprise one or more of the following components: a lens drive device designed as a motorized drive device or a piezo-driven focus drive for changing the position of an objective of the microscope, a sample holder drive device for in particular motorized modification of the position of the sample holder, a lens with variably adjustable Focal length, an adaptive mirror in the beam path of the microscope, or a piezo tripod for three-dimensional displacement of the lens of the microscope.
  • a lens drive device designed as a motorized drive device or a piezo-driven focus drive for changing the position of an objective of the microscope
  • a sample holder drive device for in particular motorized modification of the position of the sample holder
  • a lens with variably adjustable Focal length an adaptive mirror in the beam path of the microscope
  • a piezo tripod for three-dimensional displacement of the lens of the microscope.
  • All measures are suitable as parts of the setting unit, which allow a relative displacement of the receiving area relative to the object.
  • the skilled person is beyond the measures mentioned here various other measures available.
  • the setting unit can also be designed to add an offset to a control voltage of the scanner mirror or the scanner mirror in order to shift the recording area.
  • the camera chip or the entire camera or, alternatively, the tube lens can be moved to move the recording area.
  • the relative position between the receiving area and the sample holder is variable by a relative linear movement along an optical axis of the microscope. This ensures that the individual recordings of the measurement data set always show the desired sample detail in good image quality.
  • the setting unit has more degrees of freedom, that is, the relative position between the receiving area and the sample holder is variable in several dimensions. For example, additionally or alternatively to a relative displacement along the optical axis of the microscope, a lateral displacement between the receiving area and the sample holder and / or a tilting or rotating of the relative position between receiving area and sample holder may be possible. As a result, movements of the sample can be compensated, which do not run parallel to the optical axis of the microscope.
  • a receiving area corresponds in each case to a two-dimensional area in space.
  • the receiving area can lie in a respective focal plane of the microscope.
  • the data acquisition unit can have, for example, a CCD camera for recording the reference data record and the measurement data record.
  • the microscope is embodied as a laser scanning microscope in which a scanner for scanning the two-dimensional recording area with excitation light is provided and the data recording unit comprises a photomultiplier for recording corresponding signal light.
  • the laser scanning microscope can be eg a confocal microscope or a multiphoton microscope.
  • galvanometric scanners for example, galvanometric scanners, resonant scanners, polygon scanners or acousto-optical scanners can be used as scanners.
  • acousto-optical scanners a shift of the recording area in all three dimensions can take place by means of an adaptation of the control parameters of the scanner.
  • a capture area may correspond to a line or polygon scan path in space, respectively.
  • a scan of a sequence of individual points may be included in the space.
  • a change in the position of the recording area in the room can be done by linear displacement of the line or Polygonskanpfads.
  • the microscope can be embodied as a laser scanning microscope with a scanner that is configured to scan the line or polygon scanning path with excitation light during a recording.
  • the data acquisition unit may comprise a photomultiplier for receiving corresponding signal light. To adjust the recording area, for example, the starting point at which the scanner begins to scan path to be changed.
  • a recording area when recording the reference data set, a recording area may correspond to a two-dimensional area in space, and when recording the measurement data set, a recording area may correspond to a line or polygon scanning path in space.
  • the control device may be configured to control the setting unit such that changing the relative position between the receiving area and the sample holder during recording of the reference data set corresponds in each case to a linear displacement of the receiving area relative to the sample holder along an optical axis of the microscope.
  • a plurality of recordings are taken, each with a mutually shifted focal plane.
  • the position of the receiving area with respect to the sample can be stabilized during the subsequent recording of the measured data record.
  • the control device can be configured to control the setting unit and the data recording unit in such a way that a series of line or polygon scans offset from one another is recorded as a reference data record.
  • the line or polygon scans can be linearly offset from each other, in particular in the lateral direction and / or along an optical axis of the microscope.
  • the control device may be configured to control the setting unit such that changing the relative position between the receiving area and the sample holder during recording of the measurement data set comprises at least one displacement of the recording area relative to the sample holder along an optical axis of the microscope.
  • the measurement data set can correspond to a time-resolved two-dimensional image series.
  • the measurement data set may correspond to a time-resolved series of lines or polygon scans.
  • the inventive movement correction can suppress unwanted motion artifacts resulting from a change in the position of the line or polygon scan path with respect to the sample due to an intrinsic motion of the sample.
  • control device may be configured to analyze the reference data set by data analysis for at least several of the recordings forming a reference data record between the preferred relative position of the recording area with respect to the sample and a relative position of the recording area associated with the respective recording to determine the sample.
  • the control device can determine the displacement vector between the preferred relative position of the recording area with respect to the sample and the relative position of the recording area associated with the respective recording with respect to the sample, since the control device knows in which way and the relative position between the receiving area and the sample holder has been changed between the individual recordings of the reference data record by driving the setting unit.
  • the controller can compare the current recording with the individual recordings of the reference data set.
  • the control device determines, based on evaluation criteria, a recording of the reference data record whose displacement relative to the preferred relative position of the recording area with respect to the sample best matches the displacement of the current recording with respect to the preferred relative position of the recording area with respect to the sample.
  • This selection of a recording of the reference data record can take place by means of various methods of image processing and in particular based on a correlation method and / or an intensity comparison between the recordings.
  • the control unit can resort to the displacement vector associated with the selected recording of the reference data set.
  • the control device could be configured by data analysis from a recording of the reference data record corresponding to a preferred relative position of the recording area with respect to the sample, based on the data of the remaining recordings of the reference data record and the known displacement vectors between the two single images at several feature points to calculate three-dimensional intensity gradients. Based on the intensity gradients, local Taylor developments of intensity could be determined at the feature points.
  • control unit could minimize a difference between the local Taylor developments of the intensities at the feature points and the intensities of the respective current image at the feature points, for example by means of a least square method, and thus a global estimate for the current displacement vector is obtained.
  • the estimation of the current displacement vector by the control device may also be iterative.
  • the control device for this purpose include a graphics card.
  • the number and selection of feature points can be over the iterations be constant or can also be customized.
  • the image pyramid can be calculated in the currently analyzed image of the measurement data set and in the same way in the reference data set. The iteration can first take place within a magnification stage and then be continued step by step in higher magnification levels of the image pyramid.
  • both the recordings of the reference data record and the recordings of the measured data record are only calculated locally at the feature points. So the smoothing can be done within a short time.
  • the adjustment unit for varying the relative position between the recording area and the sample holder can be actuated to perform motion correction during the recording of the measurement data record.
  • the invention also relates to a method for time-resolved microscopy of a sample.
  • the device according to the invention is suitable for carrying out the method, designed and configured. Features described with respect to the device can be transferred to the process, and vice versa.
  • a sample is held with a sample holder.
  • a reference data record is recorded.
  • the reference data record is evaluated, in particular by means of a control device, for characterizing an environment of a preferred relative position of the recording region with respect to the sample.
  • step a) data is recorded by means of the data acquisition unit.
  • step b) a displacement vector between the preferred relative position of the recording area with respect to the sample and a current relative position of the recording area with respect to the sample is determined based on the data recorded in step a) and the evaluated reference data set.
  • step c) the relative position between the receiving area and the sample holder for at least partially compensating the current displacement vector is changed.
  • the invention is applicable, for example, to wide-field microscopy, in particular wide-field microscopy with high aperture, fluorescence microscopy, laser scanning microscopy, multiphoton micoscopy and light-sheet microscopy.
  • FIG. 1 shows a schematic representation of a device for time-resolved microscopy according to an embodiment with a camera-based determining unit
  • FIG. 2 shows a schematic illustration for explaining the recording of a reference data record according to an embodiment with a two-dimensional surface in the space as a receiving region;
  • FIG. 3 shows a schematic representation of a device for time-resolved microscopy according to an embodiment in which the microscope is designed as a laser scanning microscope and the adjustment unit comprises a lens drive device;
  • FIG. 4 shows a schematic representation of a device for time-resolved microscopy according to an embodiment in which the microscope is designed as a laser scanning microscope and the setting unit comprises a telescope;
  • Fig. 5 is a schematic representation of a device for time-resolved microscopy according to an embodiment in which the microscope as Laser scanning microscope is formed and the adjustment unit comprises an adaptive mirror;
  • Fig. 6 is a schematic diagram for explaining the recording of a reference data set according to an embodiment in which the recording area corresponds to a line or polygon scanning path in space;
  • FIG. 7 shows a schematic representation of a device for time-resolved microscopy according to an embodiment in which the microscope is designed as a laser scanning microscope and a further receiving channel is provided.
  • FIG. 1 shows an apparatus for time-resolved microscopy of a sample 1 according to an embodiment.
  • the device comprises a sample holder 3 for receiving the sample 1.
  • the sample 1 can be fixed to the sample holder 3.
  • the device further comprises a microscope 5 with an objective 7.
  • the receiving region 9 is a two-dimensional surface in space Focus plane of the microscope 5 is located.
  • the microscope 5 comprises a data recording unit 11 for recording data from the recording area 9.
  • the apparatus further comprises a setting unit 13 for variably setting a relative position between the receiving area 9 and the sample holder 3.
  • the setting unit 13 comprises a sample holder setting unit 15 and a microscope setting unit 17.
  • the sample holder setting unit 15 is configured to to change the position of the sample holder 3 in the room.
  • the sample holder setting unit 15 may include, for example, a sample holder driving means for motorized changing the position of the sample holder 3.
  • the microscope setting unit 17 is designed to change the position of the receiving area 9 in space.
  • the microscope setting unit 17 comprises a piezo tripod 19 for changing the position of the objective 7 of the microscope 5.
  • the piezo tripod 19 may comprise three with respect to the optical axis 21 of the microscope 5 symmetrically arranged piezo elements 23, by means of which Lens 7 is variable in its position. Such a piezo tripod 19 allows precise adjustment of the position of the objective 7. By jointly driving the three piezo elements 23, the objective 7 can be displaced along the optical axis 21 of the microscope 5, whereby the receiving region 9 is likewise displaced along the optical axis 21 , By different activation of the piezo elements 23 or driving Only one or two of the piezo elements 23 can tilt the lens 7, resulting in a shift of the receiving area 9.
  • the microscope setting unit 17, for example, also a lens drive means for motorized change in the position of the lens 7 of the microscope 5 include. This could, for example, allow a displacement of the objective 7 along the optical axis 21 of the microscope 5.
  • the additional or alternative provision of a piezo-driven focus drive for changing the position of a focal plane of the microscope 5 would be conceivable.
  • the time-resolved microscopy apparatus further comprises a control device 25, which is configured to drive the setting unit 13 and the data acquisition unit 11.
  • the control device 25 may include a computing unit 27 and a memory unit 29 with suitable program instructions.
  • the control device 25 for adjusting the relative position between the receiving area 9 and the sample holder 3 can drive both the sample holder setting unit 15 and the microscope setting unit 17.
  • the setting of the relative position between the receiving area 9 and the sample holder 3 can thus be achieved by a combination of a change in the position of the sample holder 3 in the room and a change in the position of the receiving area 9 in the room. But it would also be conceivable that, for example, the sample holder 3 is held stationary and only the receiving area 9 is moved in space. It would also be conceivable to leave the receiving area 9 stationary in the room and to change only the position of the sample holder 3 in the room.
  • the controller 25 is configured to drive the setting unit 13 and the data acquisition unit 11 to receive a reference data set.
  • data are repeatedly taken from the receiving area 9 alternately alternately by the data recording unit 11, and then the relative position between the receiving area 9 and the sample holder 3 is changed by means of the setting unit 13.
  • the reference data set comprises more than 20, more than 50, more than 100, more than 150, more than 200 or more than 500 recordings thus obtained, each corresponding to different relative positions between the receiving area 9 and the sample holder 3.
  • the reference data set is used to suppress in a sequence data set movement of the sample 1, such as a movement due to a heartbeat or respiration, resulting motion artifacts. So that the reference data set itself is largely unaffected by such movements of the sample 1, For example, the measurement of the reference data set could take place within a time period which is small compared to the time scale relevant to the respective movement of the sample 1.
  • Figure 2 shows a schematic representation of an example of different relative positions between the sample holder 3 and the receiving area 9 for the individual recordings of a reference data set.
  • the respective receiving areas 9 are in this case areal areas of a focus plane of the microscope 5 whose relative position with respect to the sample holder 3 has been changed from recording to recording of the reference data record by means of the setting unit 13.
  • the reference data record is recorded sufficiently quickly that the position of the sample 1 with respect to the sample holder 3 can be regarded as constant during the recording of the reference data record. This is not necessarily the case.
  • FIG. 2 shows a group of relative positions of the receiving region 9 with respect to the sample holder 3, which is obtained by a linear displacement of the receiving region 9 relative to the sample holder 3 along the optical axis 21 of the microscope 5.
  • Other relative positions could also be provided, in which tilt positions are present between the receiving area 9 and the sample holder 3.
  • control device 25 After receiving the reference data record, the control device 25 is configured to evaluate the reference data record for characterizing an environment of a preferred relative position of the receiving region 9 with respect to the sample 1.
  • the control device 25 is configured to iteratively perform the steps a), b) and c) for recording a measurement data set several times after the evaluation of the reference data record.
  • step a) the data acquisition unit 11 is driven to receive data.
  • step b) a displacement vector between the preferred relative position of the receiving area 9 with respect to the sample 1 and a current relative position of the receiving area 9 with respect to the sample 1 is then determined based on the data recorded in step a) and the evaluated reference data record.
  • step c) the setting unit 13 for changing the relative position between the receiving area 9 and the sample holder 3 is actuated for at least partial compensation of the displacement vector determined in step b).
  • steps a), b) and c) during the recording of the measurement data set, movement artifacts resulting from a movement of the sample 1 can be suppressed by tracking the relative position between recording area 9 and sample 1.
  • the adjustment unit 13 can be controlled in step c) such that the displacement vector between the preferred and the current relative position of the receiving region 9 with respect to Sample 1 is at least substantially equalized to zero.
  • the invention can also be used for the stabilized recording of a time series in which a different image detail of the sample 1 is recorded from one image to another.
  • the setting unit 13 can be controlled in step c) so that the displacement vector between the preferred and the current relative position of the receiving area 9 with respect to the sample 1 is not compensated to zero but to a predetermined offset vector.
  • the offset vector may be different from shot to shot.
  • the offset vector can be changed from one recording to the next by one increment.
  • the evaluation of the reference data record by the control device 25 and the determination of the displacement vector between the preferred and the current relative position of the recording region 9 with respect to the sample 1 in step b) can be carried out in various ways.
  • the control device 25 for evaluating the reference data record from the recordings of the reference data record can select a preferred recording, which then corresponds to the preferred relative position between the recording region 9 and the sample 1. This selection could also be based on user input. It is not absolutely necessary to know which actual relative position exists between the receiving area 9 and the sample 1 during the recording of the corresponding recording of the reference data set. However, the associated relative position between receiving area 9 and sample holder 3 is known. The control device 25 can then determine displacement vectors between the preferred relative position of the receiving region 9 with respect to the sample 1 and the relative positions of the receiving region 9 associated with the remaining images of the reference data set with respect to the sample 1 and thus characterize the environment of the preferred relative position.
  • the control device 25 accesses the known relative movements between the receiving area 3 and the sample holder 1 during the recording of the reference data set.
  • the control device 25 can determine, for example by means of correlative methods by comparing the recorded in step a) recording with the recordings of the reference data record a record of the reference data set, which was made at a similar relative position between receiving area 9 and sample 1 like the picture taken in step a).
  • the displacement vector between the preferred relative position of a recording area 9 with respect to the sample 1 and the current relative position of the recording area 9 with respect to the sample 1 can then be considered as the displacement vector between the recording of the reference data record corresponding to the preferred relative position and the selected other recording of the reference data set are determined.
  • control device 25 could first also select from the recordings of the reference data record a preferred recording which corresponds to the preferred relative position between the recording area 9 and the sample 1.
  • control device 25 can be configured to analyze the reference data record by means of data analysis from the recording of the reference data record which corresponds to the preferred relative position of the recording area with respect to the sample, based on the data of the remaining recordings of the reference data record and the known data Displacement vectors between the individual images at several feature points to calculate three-dimensional intensity gradients. Based on the intensity gradients, local Taylor evolution of intensity could be determined at the feature points.
  • step b) during the acquisition of the measurement data set, a difference between the local Taylor developments of the intensities at the feature points and the intensities of the respectively current image at the feature points could be minimized by the control unit 25, for example by means of a least square method, and thus obtaining a global estimate for the current displacement vector.
  • the setting unit 13 Based on the current estimated displacement vector, the setting unit 13 could then be actuated in step c) for changing the relative position between the receiving area 9 and the sample holder 3 in order to perform a motion correction live.
  • the measurement data set itself represents a motion-stabilized, evaluable data record and can be used for later evaluation of the corresponding experiment. It is therefore in principle sufficient if the microscope 5 diglich has a single measuring channel, which is used for recording the reference data set and the measurement data set.
  • the data acquisition unit 1 1 may comprise, for example, a first CCD camera 31.
  • the measuring channel assigned to the reference data set and the measured data set is provided merely as a stabilization channel and an additional measuring channel (hereinafter: experimental channel) is provided for the actual experimental data or additional experimental data.
  • the stabilization channel can then be used for motion stabilization as described above.
  • further experimental data can be recorded via the experiment channel.
  • the experimental channel may be coupled to the stabilizing channel such that the relative position between the receiving region 9 and the sample 1 is identical for both channels.
  • the data acquisition unit 11 in addition to the first CCD camera 31 can have a second CCD camera 33 which is assigned to the experiment channel.
  • the stabilization channel and the experiment channel can be designed, for example, as different color channels.
  • the stabilization channel is preferably designed such that it primarily detects non-dynamic signals which originate exclusively from an intrinsic movement of the sample 1, but which are relatively insensitive to the dynamics actually to be investigated.
  • the experiment channel can be designed to primarily detect signals originating from a dynamic to be investigated. It would also be conceivable, for example, for a line scan or polygon scan to be performed in the sample 1 via the experiment channel, while two-dimensional images are recorded in the stabilization channel.
  • FIGS. 3, 4 and 5 show devices for time-resolved microscopy of a sample 1 according to further embodiments. These embodiments differ from the embodiment of FIG. 1 in that the microscope 5 is designed as a laser scanning microscope.
  • the microscope 5 here comprises a scanner 35 with which a scan path as a receiving region 9 can be scanned through the microscope 5.
  • the scanner 35 is preferably controlled by the control device 25 and may comprise one or more, in particular two, scanner mirrors 36.
  • the control device 25 may comprise one or more, in particular two, scanner mirrors 36.
  • the setting unit 13 for variably setting a relative position between the receiving area 9 and the sample holder 3 again comprises a microscope setting unit 17 and a sample holder setting unit 15.
  • the microscope setting unit 17 comprises an objective drive device 37 for displacing the objective 7 along the optical axis 21 of the microscope 5.
  • the lens drive device 37 may be provided, for example, as a motorized drive device or as a piezo-driven focus drive.
  • FIG. 4 shows an embodiment in which the objective 7 is stationary and the microscope setting unit 17 is instead provided as a motorized telescope 39.
  • the microscope setting unit 17 is instead provided as a motorized telescope 39.
  • a lens 41 of the motorized telescope 39 By moving a lens 41 of the motorized telescope 39, a focus of the microscope 5 along the optical axis 21 of the microscope 5 can be moved.
  • FIG. 5 shows a further possibility of how the microscope setting unit 17 can alternatively be formed.
  • an adaptive mirror 43 is provided with which a scanning deposit of the microscope 5, in particular along the optical axis 21 of the microscope 5 and / or laterally displaceable. Via a control signal from the control device 25, a mirror surface of the adaptive mirror 43 can be curved convexly or concavely. In this way, the focus of the excitation beam 45 can be changed in particular without astigmatism in its position.
  • the setting unit 25 is designed to add an offset to a control voltage of the scanner mirror 36 or the scanner mirror 36 in order to displace the receiving area
  • the microscopes 5 of the embodiments shown in FIGS. 3, 4 and 5 each have a data acquisition unit 11 in the form of a photomultiplier 47 for recording data from the respective recording area 9.
  • the control device 25 controls the scanner 35 in each case to scan a predefined scan path, which corresponds to the recording area 9. If the relative position between the receiving region 9 and the sample holder 3 is to be changed, a starting point of the scanning path is displaced relative to the sample holder 3, for example, by means of the microscope setting unit 17 and / or the sample holder setting unit 15.
  • the receiving area 9 similar to the embodiment shown in Figure 1, as a two-dimensional surface present in the room.
  • the control device 25 can control the scanner 35 to scan a two-dimensional surface when taking a picture of the reference data record and recording a picture of the measured data record.
  • the evaluation of the reference data set and the stabilization of the measured data set can be carried out analogously to the embodiment described in FIG.
  • the control device 25 controls the setting unit 13 and the data recording unit 1 1 to record a reference data set by repeating, alternately picking up data by the data recording unit 11 and changing the relative position between the recording area 9 and the sample holder 3.
  • the line or polygonal scanning path representing the recording area 9 is scanned by means of the scanner 35, the recording being recorded by means of the photomultiplier 47 in the meantime.
  • FIG. 6 shows, in a schematic perspective view, the relative positions between individual receiving areas 9, which are designed as line or polygonal scanning paths, and the sample holder 3, which are each assigned to a recording of the reference data record.
  • the receiving areas 9 are shifted relative to one another along the optical axis 21 of the microscope 5.
  • receiving areas 9 could also be traversed, which are displaced laterally, for example, with respect to the receiving areas 9 shown or are tilted with respect to them.
  • the control device 25 evaluates the reference data record for characterizing an environment of a preferred relative position of the recording region 9 with respect to the sample 1.
  • the components of the intensity gradients for each point of the line or Polygonscanpfads can be calculated, for example, based on a recording which corresponds to a preferred position of the receiving area 9 with respect to the sample 1, by simply subtraction with the other recordings of the reference data set.
  • feature points can then be determined from the line or polygonal scan path. For the selection of the feature points z. B.
  • the feature points are preferably selected can be that the global aperture problem is solved and no feature points are selected with strong image noise. In the simplest case, however, all points of the line or polygon scan path (or else of a two-dimensional image scan path) can also be included for the stabilization. Based on the intensity gradients, local Taylor evolutions of intensity could be determined at the feature points or at all points of the line or polygon scan path (or even a two-dimensional image scan path).
  • control device 25 After the evaluation of the reference data record, the control device 25 carries out the steps a), b) and c) iteratively several times to record a measured data record.
  • step a the data acquisition unit 11 is driven to record data.
  • the line or polygonal scanning path representing the recording area 9 is traced by means of the scanner 35, the recording being recorded by means of the photomultiplier 47 in the meantime.
  • step b) a displacement vector between the preferred relative position of the receiving region 9 with respect to the sample 1 and a current relative position of the receiving region 9 with respect to the sample 1 is determined. This is done based on the data recorded in step a) and the evaluated reference data record.
  • the control unit could minimize a difference between the local Taylor developments of the intensities at the feature points and the intensities of the respective current image at the feature points, for example by means of a least square method, and thus a global estimate for the current displacement vector is obtained. This could, as explained above, also be iterative, for example by calculating a picture pyramid.
  • step c) the setting unit 13 for changing the relative position between the receiving area 9 and the sample holder 3 is actuated for at least partially compensating the displacement vector.
  • a recording area 9 could each correspond to a two-dimensional area in the room and then during recording of the measurement data set a recording area 9 correspond in each case to a line or polygon scanning path in the room. So could be used as a reference data set for stabilization even when recording a line or Polygonscanpfads as a measurement data set on two-dimensional areal recordings.
  • FIG. 7 shows a schematic representation of a device for time-resolved microscopy according to a further embodiment, in which the microscope 5 is designed as a laser scanning microscope.
  • the device according to this embodiment is identical except for the differences explained below or analogous to the embodiments of Figures 3, 4 and 5.
  • the microscope setting unit 17 is shown in Figure 7 as a piezo tripod 19 for moving the focal plane of the lens 7, but could for example also be designed as described with regard to FIGS. 3, 4 and 5.
  • the device according to FIG. 7 comprises, in addition to the measurement channel for recording the reference data set and the measurement data set (stabilization channel), a further measurement channel (experimental channel) for recording (additional) experimental data.
  • the stabilization channel may be used, for example, as described with reference to Figures 3, 4 and 5, for motion stabilization and the actual data of the experiment or additional data may be obtained via the experimental channel. Since the stabilization channel and the experiment channel are measurement channels of the same microscope 5, motion-stabilized measurement data are obtained via the experiment channel without additional effort if the measurement is performed simultaneously with the motion-stabilized recording of the measurement data set via the stabilization channel.
  • the microscope 5 comprises a further scanner 49 to which a split-off part of the excitation beam 45 is supplied.
  • the further scanner 49 can in turn have one or more, in particular two, scanner mirrors 51 with which it can scan a one-dimensional or a two-dimensional scan pattern independently of the scanner 35 of the stabilization channel.
  • the data acquisition unit 11 includes, in addition to the photomultiplier 47 assigned to the stabilization channel, a further photomultiplier 53, which is assigned to the experiment channel.
  • a spectral separation element in particular a dichroic mirror 55
  • the signal radiation from the specimen 1 can be spectrally spectrally transmitted to the photomultiplier 47 of the stabilization channel and the photomultiplier 53 of the experiment. be divided.
  • the spectral distribution can be such that the stabilization channel primarily detects non-dynamic signals, which mainly result from a movement of the sample, but is relatively insensitive to the dynamics actually to be investigated.
  • the experiment channel can primarily detect signals that originate from the dynamics to be investigated.

Abstract

Le dispositif de microscopie à résolution temporelle d'un échantillon (1) comprend un porte-échantillon (3), un microscope (5) pourvu d'une unité d'acquisition de données (11) destinée à acquérir des données d'une région d'acquisition (9) définie par le microscope (5), une unité de réglage (13) destinée à régler de manière variable une position relative entre la région de réception (9) et le porte-échantillon (3) et un moyen de commande (25) destiné à commander l'unité de réglage (13) et l'unité d'acquisition de données (11). Pour acquérir un ensemble de données de référence, des données sont acquises alternativement de manière répétée et la position relative entre la région d'acquisition (9) et le porte-échantillon (3) est modifiée. L'ensemble de données de référence est évalué pour caractériser un environnement d'une position relative préférée de la zone d'acquisition (9) par rapport à l'échantillon (1). Ensuite, les étapes suivantes sont effectuées plusieurs fois de manière itérative pour acquérir un ensemble de données de mesure : a) acquérir des données; b) déterminer un vecteur de déplacement entre la position relative préférée et la position relative actuelle de la région d'acquisition (9) par rapport à l'échantillon (1) sur la base des données acquises et de l'ensemble de données de référence évalué ; et c) modifier la position relative entre la région d'acquisition (9) et le porte-échantillon (3) pour compenser au moins partiellement le vecteur de déplacement.
PCT/EP2017/078868 2017-11-10 2017-11-10 Microscopie à résolution temporelle d'un échantillon WO2019091570A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/EP2017/078868 WO2019091570A1 (fr) 2017-11-10 2017-11-10 Microscopie à résolution temporelle d'un échantillon
EP17803838.6A EP3707543A1 (fr) 2017-11-10 2017-11-10 Microscopie à résolution temporelle d'un échantillon

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/EP2017/078868 WO2019091570A1 (fr) 2017-11-10 2017-11-10 Microscopie à résolution temporelle d'un échantillon

Publications (1)

Publication Number Publication Date
WO2019091570A1 true WO2019091570A1 (fr) 2019-05-16

Family

ID=60450614

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2017/078868 WO2019091570A1 (fr) 2017-11-10 2017-11-10 Microscopie à résolution temporelle d'un échantillon

Country Status (2)

Country Link
EP (1) EP3707543A1 (fr)
WO (1) WO2019091570A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3988989A1 (fr) * 2020-10-14 2022-04-27 Abberior Instruments GmbH Procédé et microscope doté d'un dispositif de détection des déplacements d'un échantillon par rapport à une lentille

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH07248452A (ja) * 1994-03-14 1995-09-26 Nikon Corp 標本像自動追尾装置
WO2008153836A2 (fr) * 2007-05-31 2008-12-18 President And Fellows Of Harvard College Microscopie confocale d'acquisition à verrouillage de la cible en temps réel (tarc)
US9007453B2 (en) * 2011-09-12 2015-04-14 Olympus Corporation Time lapse observation method, and time lapse observation apparatus and multiphoton microscope used therefor
US20150278625A1 (en) * 2012-12-14 2015-10-01 The J. David Gladstone Institutes Automated robotic microscopy systems
KR20170049244A (ko) * 2015-10-28 2017-05-10 한국과학기술원 공초점 현미경 시스템

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH07248452A (ja) * 1994-03-14 1995-09-26 Nikon Corp 標本像自動追尾装置
WO2008153836A2 (fr) * 2007-05-31 2008-12-18 President And Fellows Of Harvard College Microscopie confocale d'acquisition à verrouillage de la cible en temps réel (tarc)
US9007453B2 (en) * 2011-09-12 2015-04-14 Olympus Corporation Time lapse observation method, and time lapse observation apparatus and multiphoton microscope used therefor
US20150278625A1 (en) * 2012-12-14 2015-10-01 The J. David Gladstone Institutes Automated robotic microscopy systems
KR20170049244A (ko) * 2015-10-28 2017-05-10 한국과학기술원 공초점 현미경 시스템

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
ANONYMOUS: "Resonant Scanning in Laser Confocal Microscopy | MicroscopyU", 19 August 2016 (2016-08-19), XP055491906, Retrieved from the Internet <URL:https://www.microscopyu.com/techniques/confocal/resonant-scanning-in-laser-confocal-microscopy> [retrieved on 20180711] *
DANIEL VON WANGENHEIM ET AL: "Live tracking of moving samples in confocal microscopy for vertically grown roots", ELIFE, vol. 6, 19 June 2017 (2017-06-19), XP055491442, DOI: 10.7554/eLife.26792 *
H PINKARD ET AL: "Micro-Magellan: open-source, sampleadaptive, acquisition software for optical microscopy", NO.10 | OCTOBER 2016, 29 September 2016 (2016-09-29), pages 807, XP055491550, Retrieved from the Internet <URL:https://www.nature.com/articles/nmeth.3991> [retrieved on 20180710] *
PENG CHENG ET AL: "Real-Time Continuous Image Registration Enabling Ultraprecise 2-D Motion Tracking", IEEE TRANSACTIONS ON IMAGE PROCESSING, IEEE SERVICE CENTER, PISCATAWAY, NJ, US, vol. 22, no. 5, 31 May 2013 (2013-05-31), pages 2081 - 2090, XP011497806, ISSN: 1057-7149, DOI: 10.1109/TIP.2013.2244608 *
RABUT G ET AL: "Automatic real-time three-dimensional cell tracking by fluorescence microscopy", JOURNAL OF MICROSCOPY, BLACKWELL SCIENCE, GB, vol. 216, 2 November 2004 (2004-11-02), pages 131 - 137, XP002504212, ISSN: 0022-2720, DOI: 10.1111/J.0022-2720.2004.01404.X *
SUNGON LEE ET AL: "Image Stabilization for In Vivo Microscopy by High-Speed Visual Feedback Control", IEEE TRANSACTIONS ON ROBOTICS, IEEE SERVICE CENTER, PISCATAWAY, NJ, US, vol. 23, no. 1, 24 February 2008 (2008-02-24), pages 45 - 54, XP011204852, ISSN: 1552-3098 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3988989A1 (fr) * 2020-10-14 2022-04-27 Abberior Instruments GmbH Procédé et microscope doté d'un dispositif de détection des déplacements d'un échantillon par rapport à une lentille
US11967090B2 (en) 2020-10-14 2024-04-23 Abberior Instruments Gmbh Method of and microscope comprising a device for detecting movements of a sample with respect to an objective

Also Published As

Publication number Publication date
EP3707543A1 (fr) 2020-09-16

Similar Documents

Publication Publication Date Title
DE102014206309B4 (de) System und Verfahren zum Erhalten von Bildern mit Versatz zur Verwendung für verbesserte Kantenauflösung
EP2107884B1 (fr) Procede et appareil de diagnostic de la retine
DE19722790B4 (de) Anordnung und Verfahren zur zeitaufgelösten Messung nach dem Scannerprinzip
DE102009059978A1 (de) Endoskopsystem mit Abtastfunktion
DE102014223387A1 (de) Bildverarbeitungs-Inspektionssystem und Verfahren zum Erhalten eines Bilds mit erweiterter Schärfentiefe
DE102014102080A1 (de) Verfahren zur Bildaufnahme und Bildaufnahmesystem
DE112010002934T5 (de) Mikroskop mit einem Strahl geladener Teilchen und Messverfahren dafür
DE102005047200A1 (de) Verfahren zur Korrektur einer Steuerung eines optischen Scanners in einer Vorrichtung zur scannenden Abbildung einer Probe und Vorrichtung zur Durchführung des Verfahrens
DE102016217628B4 (de) Verfahren zum Betreiben eines Operationsmikroskopiesystems, Bewegungsmesssystem für ein Operationsmikroskopiesystem und Operationsmikroskopiesystem
DE112014003528T5 (de) Bildverarbeitungsvorrichtung und Bildverarveitungsverfahren
DE102015119400A1 (de) Mehrfachkameravorrichtung und Verfahren zum synchronisierten Autofokussieren
DE102005047218A1 (de) Verfahren zur Korrektur einer Steuerung eines optischen Scanners und optischer Scanner
EP3244197A1 (fr) Procédé et dispositif de réglage de la position focale d&#39;un laser d&#39;excitation dans des vaisseaux sanguins pour mesures optiques destinées à déterminer le sexe d&#39; oeufs de volaille
WO1998050938A2 (fr) Dispositif d&#39;imagerie et/ou de balayage a compensation des degradations d&#39;image dues aux facteurs environnementaux
WO2019091570A1 (fr) Microscopie à résolution temporelle d&#39;un échantillon
WO2007039054A1 (fr) Procede pour produire des images de representation a partir d&#39;images determinees et organes permettant de realiser ledit procede
DE102010062975A1 (de) Verfahren zur Erzeugung einer vierdimensionalen Darstellung eines einer periodischen Bewegung unterworfenen Zielgebiets eines Körpers
DE102023115998A1 (de) System und verfahren zur hochgeschwindigkeits-autofokussierung eines mikroskops
EP1548485B1 (fr) Méthode de correction de la dérive d&#39;un appareil optique
DE10251345B4 (de) Verfahren und Vorrichtung zur Untersuchung von Schichten von Geweben in lebenden Tieren mit einem Mikroskop
EP3548836B1 (fr) Détermination de l&#39;agencement d&#39;un objet échantillon par éclairage angulaire sélectif
DE102021126658B3 (de) Medizinisches Visualisierungsystem und Verfahren zur Video-Stabilisierung bei einem solchen System
DE102019219506A1 (de) System und verfahren zum eichen eines linsensystems mit variabler brennweite unter verwendung eines eichobjekts mit einer ebenen geneigten musterfläche
EP4187306A1 (fr) Procede d&#39;evaluation de donnees de mesure d&#39;un microscope a champ lumineux et dispositif de microscopie a champ lumineux
DE102007005376A1 (de) Verfahren zum Bestimmen von EKG-getriggerten Aufnahmezeitpunkten für die Bildgebung zur Unterstützung von interventionellen und diagnostischen Eingriffen am Herzen

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17803838

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2017803838

Country of ref document: EP

Effective date: 20200610