WO2021160723A1

WO2021160723A1 - Microscope apparatus

Info

Publication number: WO2021160723A1
Application number: PCT/EP2021/053296
Authority: WO
Inventors: Xian HU; Felix Margadant; Oddmund BAKKE
Original assignee: Universitetet I Oslo
Priority date: 2020-02-12
Filing date: 2021-02-11
Publication date: 2021-08-19
Also published as: GB202001902D0

Abstract

A microscope apparatus (10) comprises: an imaging volume with orthogonal X, Y and Z axes; a camera (211) having a focal plane; a light source (110, 120) for illumination of the imaging volume; and a Z-stage movement system (131, 132) for relative movement of the imaging volume and the focal plane in the Z axis. Images of microscopic biological objects S are obtained using the microscope apparatus (10) via a method comprising: using the camera (211) to obtain an image in a single exposure whilst using the Z-stage movement system (131, 132) for relative movement of the imaging volume and the focal plane; wherein the Z axis movement is a step-wise movement in which there are multiple resting steps providing periods of no relative motion in between multiple periods of relative motion; and wherein during the periods of relative motion the light source is controlled to reduce the illumination of the imaging volume.

Description

MICROSCOPE APPARATUS

The present invention relates to a method of operating a microscope apparatus for obtaining images of microscopic biological objects, such as cells and/or organelles, as well as to a corresponding apparatus, a related method for adapting a microscope apparatus, and a computer programme product.

There are many reasons for imaging of microscopic biological objects, such as in order to identify, count and/or categorise such objects. In many cases the biological objects can be dynamic objects, subject to changes from on-going biological and/or biochemical processes. It can be important to be able to accurately obtain images of such biological objects with a suitable degree of time resolution as well as spatial resolution across a sample, which will occupy a three dimensional imaging volume.

It is known to use a microscope apparatus with a motorized or mechanized imaging stage, such as with a system as described in US 2019/025213. In that publication the disclosed apparatus has an electric stage unit with motors that can move the stage in an X-Y direction to move the position of the field of view of the microscope's lens, and can move the stage in the Z direction to move the focal plane of the lens along the optical axis of the imaging volume. In order to obtain the required image of a 3D part inside the imaging volume, the microscope apparatus is controlled to take images at different focal depths including sweep capturing in which the stage is moved in the Z direction continuously, at a steady speed, whilst the image is exposed. Microscope devices of this type are controlled as in US 2019/025213 and as described in other prior publications using differing combinations of steps for image capturing and image processing in order to try to obtain successful results.

One particular example of the use of such microscopes is imaging intracellular trafficking vesicles with sufficient time resolution while also minimizing photobleaching. Various challenges arise in connection with this. Endosomes for example, often grow from resolution limited spots to blobs several microns in diameter while migrating from the membrane to the perinuclear area. They can travel along the cytoskeleton structure at variable speeds alongside numerous other vesicles in the 3D volume of a cytosol.

Commonly, to follow and study the dynamics of these vesicles, Z-stacks are recorded via live cell confocal microscopy, i.e. stacks of images extending in the Z-direction of an imaging volume with orthogonal X, Y and Z axes. However, even the more rapid spinning disk confocal microscope devices with fast piezo Z drives consume more time than a second to record a stack with 20 slices. Membrane vesicles often travel micrometer distances in that time, leaving very visible motion artefacts in the recording which then lead to inaccurate measurements or estimates. Imaging and tracking the behaviour of fluorescence labelled endosome activity in live cells, therefore remains a challenging task.

Viewed from a first aspect, the present invention provides a method of operating a microscope apparatus for obtaining images of microscopic biological objects, the microscope apparatus comprising: an imaging volume with orthogonal X, Y and Z axes; a camera with a focal plane; a light source for illumination of the imaging volume; and a Z-stage movement system for relative movement of the imaging volume and the focal plane along the Z axis; the method comprising: using the camera to obtain an image in a single exposure whilst using the Z-stage movement system for relative movement of the imaging volume and the focal plane; wherein the Z axis movement is a step-wise movement in which there are multiple resting steps providing periods of no relative motion in between multiple periods of relative motion; and wherein during the periods of relative motion the light source is controlled to reduce the illumination of the imaging volume.

With this method the camera may capture multiple slices/sections in a single exposure during the step-wise relative movement of camera and/or the imaging volume in the Z direction, hence forming a projection of the X-Y plane along the Z-axis. The method can be considered as a new hardware and/or control adaptation permitting a form of "extended focus" that greatly improves the imaging speed by generating a projection along the depth (Z) axis. Effectively, the method foregoes one dimension (e.g. as may be possible with prior art sweep capturing methods, such as US 2019/025213) in order to achieve enhanced temporal resolution. By use of this movement in the Z direction, synchronised with control of the light source, the method can obtain horizontal sections of high quality, inherently merging these multiple optical sections into a single exposure image, with the resultant image hence being a sum or average projection.

The biological objects may be cells and/or organelles, for example. The method may include obtaining and then storing image data for one or more images of the biological objects, and/or transmitting the image data to an external computer device. It will be appreciated that in some examples the biological objects can include intracellular trafficking vesicles, where the proposed method gives particularly good results in terms of improving over alternative techniques, whilst still using existing/relatively simple hardware for the microscope apparatus.

In order to make best use of the extended focus allowed for by the step-wise relative motion of the camera and the imaging volume the light source is controlled to reduce illumination during motion, and hence to have increased illumination during the resting steps within the exposure. The method may involve deactivation of the light source during the periods of motion to give maximum contrast between the increased and decreased illumination levels via use of illuminated and non-illuminated states, and hence the control of the light source may be an on-off control synchronised with the step-wise relative motion of the camera and the imaging volume. The light source may be controlled, such as via a logic signal, with the control providing pulsed illumination timed based on the periods of no relative motion.

In some examples the light source comprises a laser, such as a cooled laser diode, a DPSS laser, or other suitable forms of continuous wave laser with a modulation or shutter system. The light source may include multiple sources of light and/or optical systems for controlling the direction of the light, such as for illumination of the imaging volume from above and/or from below. Advantageously, the method may use pre-existing light sources of a microscope apparatus. Typical systems that may be used include Light Emitting Diodes and Solid State Lasers as sources and LCD, AOTF, AOM, MEMS, or Digital Micro-mirrors as shutters.

The Z-stage movement system is arranged for relative movement of the imaging volume and the camera. This may be achieved by movement of the focal plane through the imaging volume, which may be done by movement of the imaging volume, by movement of the camera, or both. The method may use pre-existing Z-stage arrangements with relative motion in accordance with those achievable via known microscope apparatus. There may be a driver for actuating the relative motion, such as one or more motor(s) and/or other actuator devices. The microscope apparatus may also include the ability for X and/or Y movement, such as via one or more motor(s) and/or other actuator devices for X/Y movement of the imaging volume and/or of the camera.

In example embodiments the relative motion in the Z direction is provided by a piezoelectric Z-stage movement system, using piezo-electric actuators. This type of actuator gives advantages for a step-wise movement as required in the present method, since such step-wise movements can be achieved accurately and with quick transitions between the moving and non-moving states of the device.

The step-wise movement may be a stop-start motion with a frequency of the periods of non-motion in the range 10-500 Hz. The total distance of travel during the single exposure may for example be in the range of 0.5 -1000 pm, or optionally 1-200 pm, with a step size that is in the range 50-1000 nm. The step size may be identical for each step, with the frequency and step size being fixed across the entire movement range.

The microscope apparatus may include one or more control systems for control of the Z-stage movement system, the camera, and/or the light source. For example, there may be one or more microprocessor controllers. The method can thus involve controlling the Z- stage movement system, the camera, and/or the light source via the control system(s) in order to carry out the steps discussed herein. The method may make use of pre-existing controllers in this way and thus can be implemented via adaptation to existing controllers as discussed below in connection with the second aspect of the invention. The microscope apparatus may comprise a single control system for controlling all of the Z-stage movement system, the operation of the camera, and the illumination from the light source.

The microscope apparatus may be configured to operate confocally or with a shallow depth of field. Thus, it may be a confocal microscope apparatus and/or the method may include deactivation of focussing functions of the microscope and/or its camera, such as deactivation of an autofocus mechanism of the microscope apparatus.

In example embodiments the microscope is a spinning disc microscope, such as with a spin speed in the range 4000-20000 RPM. The microscope may have a pin-hole disc with pin-hole of diameter in the range 10-60 pm. It has been found that the proposed method gives beneficial results with spinning disc microscopes.

The microscope may generate the logic signal for when a camera exposure starts and then use a hardware timer or the camera timer to stop the exposure after the desired time has passed. This so called trigger signal, can be used to start the multi-step exposure described herein. The microscope could operate without being aware that it records Z- projections rather than single image slices. This way, no modification to the existing microscope is required. When the extended focus is not needed, the trigger signal from the camera is simply turned off.

The method can be repeated to obtain multiple exposures, each extending across a travel range for the Z-stage movement system, and the multiple exposures may cover the same or different ranges of relative movement between the camera and the imaging volume. Repeating the method at different times allows for time series at a high temporal resolution.

As discussed above, the method can be implemented with pre-existing microscope devices. The method may include modifying an existing microscope apparatus, such as via hardware additions or software changes in order to configure the microscope apparatus for carrying out the method. For example, the method may in some cases involve adding a dedicated stage control apparatus to an existing Z-stage movement system in order to make use of a pre-existing Z-stage movement system of suitable capabilities, such as a preexisting piezo drive Z-stage device. Alternatively or additionally the method may involve programming a control system for a microscope apparatus with suitable commands in order that it will carry out the method as discussed herein. It is commonplace for microscope apparatus of this type to have control systems that are configured to receive commands in order to control parameters such as exposure time, movement (e.g. of the Z-stage movement system) and/or illumination via the light source. The method may include modifying the hardware and/or software for such a control system, such as by providing suitable code to the control system and/or via suitable interaction with the interface of the control system. It will be understood that the modification of an existing microscope apparatus to enable it to perform the method of the first aspect is novel and inventive in its own right, as a method that results in a novel and inventive apparatus.

Thus, viewed from a second aspect, the invention provides a method for adapting a microscope apparatus in order to perform the method of the first aspect, the method for adapting comprising software and/or hardware modifications to configure the microscope apparatus to perform the method of the first aspect.

This method may hence provide a microscope apparatus comprising: an imaging volume with orthogonal X, Y and Z axes; a camera having a focal plane; a light source for illumination of the imaging volume; and a Z-stage movement system for relative movement of the imaging volume and the focal plane in the Z axis; wherein the microscope apparatus is configured to use the camera to obtain an image in a single exposure whilst using the Z- stage movement system for relative movement of the imaging volume and the focal plane; wherein the Z axis movement is a step-wise movement in which there are multiple resting steps providing periods of no relative motion in between multiple periods of relative motion; and wherein during the periods of relative motion the light source is controlled to reduce the illumination of the imaging volume.

The method for adapting a microscope apparatus may include providing any of the other features discussed above in connection with the method of the first aspect.

The microscope apparatus prior to adaptation may include all of the imaging volume, the camera, the light source and the Z-stage movement system, with the method of adaptation hence primarily or solely comprising modifications to the method of operation of the microscope apparatus, such as to the control thereof. For example, the method may include adjusting a control system of the microscope apparatus in order to operate the microscope apparatus as described in relation to the first aspect or optional features thereof. The method for adapting the microscope apparatus may involve software modifications in order to adjust the control system accordingly. It will be appreciated that a computer programme product for this purpose would be novel and inventive in its own right.

Thus, viewed from a third aspect, the invention provides a computer programme product comprising instructions for execution on a control system for a microscope apparatus comprising: an imaging volume with orthogonal X, Y and Z axes; a camera having a focal plane; a light source for illumination of the imaging volume; and a Z-stage movement system for relative movement of the imaging volume and the focal plane in the Z axis; wherein the instructions, when executed, will configure the control system to: use the camera to obtain an image in a single exposure whilst using the Z-stage movement system for relative movement of the imaging volume and the focal plane; wherein the Z axis movement is a step-wise movement in which there are multiple resting steps providing periods of no relative motion in between multiple periods of relative motion; and wherein during the periods of relative motion the light source is controlled to reduce the illumination of the imaging volume.

The instructions of the computer programme product may optionally be arranged to configure the control system of the microscope apparatus to perform any of the other steps of the methods described above.

Viewed from a fourth aspect, the invention provides a microscope apparatus for obtaining images of microscopic biological objects, the microscope apparatus comprising: an imaging volume with orthogonal X, Y and Z axes; a camera having a focal plane; a light source for illumination of the imaging volume; and a Z-stage movement system for relative movement of the imaging volume and the focal plane in the Z axis; wherein the microscope apparatus is configured to use the camera to obtain an image in a single exposure whilst using the Z-stage movement system for relative movement of the imaging volume and the focal plane; wherein the Z axis movement is a step-wise movement in which there are multiple resting steps providing periods of no relative motion in between multiple periods of relative motion; and wherein the microscope apparatus is arranged such that during the periods of relative motion the light source is controlled to reduce the illumination of the imaging volume.

The microscope of this aspect may include any of the other features discussed above in relation to the method aspects. This microscope may have been originally manufactured with the above Z-stage movement capability or it may have been adapted after manufacture in accordance with the second aspect.

The microscope apparatus may be arranged for obtaining and then storing image data for one or more images of the biological objects, and/or transmitting the image data to an external computer device. Thus, the microscope apparatus may include data storage, such as a computer memory, and/or a communication system for wired or wireless communications with the external computer device.

The light source is controlled to reduce illumination during motion, and hence to have increased illumination during the resting steps within the exposure. The microscope apparatus may be configured to deactivate the light source during the periods of motion to give maximum contrast between the increased and decreased illumination levels via use of illuminated and non-illuminated states, and hence the control of the light source may be an on-off control synchronised with the step-wise relative motion of the camera and the imaging volume, with the microscope apparatus including a suitable control device for the light source, such as one or more control system(s) as discussed below. The light source may be controlled, such as via a logic signal, with the control providing pulsed illumination timed based on the periods of no relative motion. In some examples the light source comprises a laser, such as a cooled laser diode, a DPSS laser, or other suitable forms of continuous wave laser with a modulation or shutter system. The light source may include multiple sources of light and/or optical systems for controlling the direction of the light, such as for illumination of the imaging volume from above and/or from below.

The Z-stage movement system is arranged for relative movement of the imaging volume and the camera. This may be achieved by movement of the focal plane through the imaging volume, which may be done by movement of the imaging volume, by movement of the camera, or both. There may be a driver for actuating the relative motion, such as one or more motor(s) and/or other actuator devices. The microscope apparatus may also include the ability for X and/or Y movement, such as via one or more motor(s) and/or other actuator devices for X/Y movement of the imaging volume and/or of the camera.

In example embodiments the microscope apparatus comprises a piezo-electric Z- stage movement system, using piezo-electric actuators, for providing the relative motion in the Z direction. The step-wise movement may be a stop-start motion with a frequency of the periods of non-motion in the range 10-500 Hz. The total distance of travel during the single exposure may for example be in the range of 0.5 -1000 pm, or optionally 1-200 pm, with a step size that is in the range 50-1000 nm. The step size may be identical for each step, with the frequency and step size being fixed across the entire movement range.

The microscope apparatus may include one or more control system(s) for control of the Z-stage movement system, the camera, and/or the light source. For example, there may be one or more microprocessor controllers. The control system(s) may be configured to control the Z-stage movement system, the camera, and/or the light source via the control system(s) in order to carry out the steps of the method discussed herein. The microscope apparatus may comprise a single control system for controlling all of the Z-stage movement system, the operation of the camera, and the illumination from the light source.

The microscope apparatus may be configured to operate confocally or with a shallow depth of field. Thus, it may be a confocal microscope apparatus and/or may be configured to obtain the single exposure image with deactivation of focussing functions of the microscope and/or its camera, such as deactivation of an autofocus mechanism of the microscope apparatus.

The microscope apparatus may be a spinning disc microscope, such as with a spin speed in the range 4000-20000 RPM. The microscope may have a pin-hole disc with pin hole of diameter in the range 10-60 pm.

The microscope apparatus and/or a control system thereof may be configured to operate in line with the methods discussed above. Certain example embodiments of the invention will now be described by way of example only and with reference to the accompanying drawings in which:

Figure 1 is a schematic diagram of a microscope apparatus;

Figure 2 shows a general epi-illumination microscope with dual stage drive;

Figure 3 illustrates a microscope of alternative configuration to Figure 2, where fine movement is done by moving the lens;

Figures 4a to 4c show, respectively, an image of a cross-section across the centre of a spherical shell, an image of the volume of the sphere, and an image of its projection;

Figures 5a to 5c show the impact of a motion artefact on the images of Figures 4a to 4c, respectively for the cross-section, the volume, and the projection;

Figures 6a and 6b show the contribution of motion artefacts to a projection image;

Figures 7a and 7b show a real-world example with a simulated cross-section in 7a and a projection image as proposed herein at 7b; and

Figure 8 illustrates a signal generated for control of the microscope apparatus to capture extended focus (EF) images.

As shown in Figure 1 a microscope apparatus 10 includes an arm 100 having a substantially C shape; a lens tube 102 and an eyepiece unit 103 that are supported by the arm 100 through a trinocular lens unit 101; an epi-illumination unit 110 and a transmitted- light illumination unit 120 provided on the arm 100; an electric stage unit 130 including a stage 131 on which a subject S is placed; and an objective lens 140 that is provided on one end side of the lens tube 102 with the trinocular lens unit 101 interposed therebetween, so as to oppose the stage 131 and that focuses observation light from the subject S. This microscope apparatus is as disclosed in US 2019/025213 and hence in terms of the structure of the device it is a known apparatus. The operation of known microscopes such as this can advantageously be modified according to the method discussed herein in order to provide a new image capture system that greatly improves the imaging speed for microscopic biological objects by generating a projection along the depth axis, via a new technique referred to herein as “Extended Focus” (EF).

The objective lens 140, the lens tube 102 connected with the trinocular lens unit 101 interposed therebetween, and an imaging unit 211 provided on the other end side of the lens tube 102 constitute a microscopy optical system (optical capturing system) 104. The imaging unit 211 may be considered as the camera.

The epi-illumination unit 110 includes an epi-illumination light source 111 and an epi- illumination optical system 112, and emits epi-illumination light to the subject S. The epi- illumination optical system 112 includes various optical members for focusing illumination light output from the epi-illumination light source 111 and guiding the light in the direction of an optical axis L of the microscopy optical system 104. Specifically, the epi-illumination optical system 112 includes a cube having a filter that reflects light at an excitation wavelength of fluorescence and that passes fluorescence occurring by being excited due to light at the excitation wavelength, a shutter, a field stop, an aperture diaphragm, or the like.

The transmitted-light illumination unit 120 includes a transmitted-light illumination light source 121 and a transmitted-light illumination optical system 122, and emits transmitted-light illumination light to the subject S. The transmitted-light illumination optical system 122 includes various optical members for focusing illumination light output from the transmitted-light illumination light source 121 and guiding the light in the direction of the optical axis L. Specifically, the transmitted-light illumination optical system 122 includes a filter unit, a shutter, a field stop, an aperture diaphragm, or the like.

The epi-illumination units 110, 120 may be controlled to reduce or increase illumination as required, such as in accordance with the EF system described below.

The electric stage unit 130 includes the stage 131; a stage drive unit 132 that moves the stage 131; and a position detecting unit 133. The stage drive unit 132 is configured by using, for example, a motor. A subject placement surface 131a of the stage 131 is provided so as to be perpendicular to the optical axis of the objective lens 140. Hereinafter, the subject placement surface 131a is an XY plane, and the normal direction to the XY plane, i.e., the direction parallel to the optical axis, is the Z direction. With regard to the Z direction, a downward direction in the figure, i.e., the direction in which the stage 131 (the subject placement surface 131a) moves away from the objective lens 140, may be referred to as a plus direction.

By moving the stage 131 within the XY plane, the position of the field of view of the objective lens 140 may be moved. Furthermore, by moving the stage 131 in the Z direction, the focal plane of the objective lens 140 may be moved along the optical axis L. That is, the electric stage unit 130 is a moving unit that moves the focal plane or the position of the field of view by moving the stage 131 under the control of an imaging controller.

In this example, the stage 131 is moved in order to move the position of the field of view, while the position of the microscopy optical system 104 including the lens tube 102 to the objective lens 140 is fixed. However, the microscopy optical system 104 may be moved, while the position of the stage 131 is fixed, on the contrary. Alternatively, the stage 131 and the microscopy optical system 104 may be moved in opposite directions to each other. That is, any configuration may be employed as long as the microscopy optical system 104 and the subject S on the stage 131 are movable relative to each other. Furthermore, it is also possible that the focal plane is moved by moving the microscopy optical system 104 in the Z direction and the position of the field of view is moved by moving the stage 131 along the XY plane. The position detecting unit 133 is configured by using an encoder that detects the amount of rotation of the stage drive unit 132 that is for example a motor, and it detects the position of the stage 131 and outputs a detected signal. Furthermore, instead of the stage drive unit 132 and the position detecting unit 133, a pulse generating unit that generates pulses under the control of the imaging controller 22 described later and a stepping motor may be provided.

The objective lens 140 of Figure 1 is attached to a revolver 142 that is capable of supporting multiple objective lenses (e.g., objective lenses 140, 141) having different magnifications. By rotating the revolver 142 and changing the objective lenses 140, 141 opposed to the stage 131, an imaging magnification is changeable. Here, Figure 1 illustrates a state where the objective lens 140 is opposed to the stage 131.

Another microscope device 10 is shown in Figure 2. This is a schematic diagram for a general epi-iilumination microscope with a Z-stage 131 having a dual purpose stage drive 132. A first set of pistons/actuators (e.g. piezo-electric actuators) provides the long distance focus adjustment, and a second set of pistons/actuators (e.g. piezo-electric actuators) allows for very small movements at the sub-millisecond time scale. A bubble 150 between the lens and the specimen is an immersion, which can be a liquid, a solid, or also air. The upper stage Z and the specimen S are relatively light, so it can all be moved more than 100 times a second. The microscope device 10 of Figure 2 further includes an eyepiece unit 103 and an epi-illumination unit 110. The epi-illumination unit 110 includes an epi-illumination light source 111 and an epi-illumination optical system 112, and emits epi-illumination light to the specimen S via an objective lens 140. The camera views the specimen via the objective lens 140, such as via the optics of the eyepiece unit 103, or via an imaging unit located similar to the imaging unit 211 described above in relation to Figure 1.

An alternative widespread configuration is shown in Figure 3. This is similar to Figure 2 aside from that the fine movement is performed by inching the objective lens up and down. Thus, all parts are similar to Figure 2 in this example aside from that the dual purpose stage drive 132 includes pistons for movement of the Z-stage 131 as well as for movement of the lens 140. Thus, a first set of pistons/actuators (e.g. piezo-electric actuators) provides the long distance focus adjustment by movement of the Z-stage 131 , and a second set of pistons/actuators (e.g. piezo-electric actuators) allows for very small movements of the objective lens 140. This different arrangement for the relative Z movement does not impair the image quality as the light from the back port of the lens is not focused. This means that up and down movement of the lens 140 by a maximum of perhaps 0.2 mm will not distort the image.

Via the use of the EF system proposed herein, a camera based imaging system can synchronize the movement of the Z-stage 131, in particular a piezo driven stage, and the image capture by the camera of the microscope device 10. The EF approach then combines multiple optical sections into a single exposure. The image generated by EF equals a sum or average projection.

Although the EF system can be adapted to various camera based imaging platforms, it has its highest potential for confocal or structured illumination scopes, as it preserves the super-resolving feature, unlike any other EF method. The EF system may be used with spinning disk confocal microscope systems. The confocal mechanism is superior in confining the recorded volume compared to other competing approaches such as wide field fluorescence microscopy (a feature referred to as “sectioning” and it has the consequence that this EF method makes very neat and well defined rectilinear projections, albeit at the cost wasting more light than competing EF approaches). EF also gives excellent results on instant-SIM spinning disk systems.

In broad terms, the EF imaging requires a microscope apparatus 10 comprising: an imaging volume with orthogonal X, Y and Z axes; a camera 211; a light source 111, 121 for illumination of the imaging volume; and a Z-stage movement system 131, 132 for relative movement of the of the focal plane through the imaging volume, such as by moving the Z- stage 131 relative to the camera 211. The basic EF method involves using the camera 211 to obtain an image in a single exposure; using the Z-stage movement system 131,132 for relative movement of the imaging volume and the camera in a step-wise fashion, whilst the illumination from the light source is reduced/deactivated during movement. The step-wise movement synchronised with control of the light source results in multiple periods of no relative motion, with increased light, in between multiple periods of relative motion, with decreased light/no light from the light source. By way of example, the Z-stage movement system can be a piezo-electric Z-stage 131, 132 and the light source can be a laser.

As the piezo Z-stage does not travel during light exposure with the structured illumination, the EF projection can be recorded natively super-resolved, as discussed further in the examples below. With an instant-SIM system the EF projection preserves the instant- SIM super resolution feature. The advantages in resolution are probably the most unique advantage for the proposed EF system over other extended depth of field methods. This greatly aids size and area measurements of the imaged objects, which may for example be cells or organelles. The inherent higher resolution paired with the projection allows for very low-light, highly accurate and sensitive tracking.

It will of course be appreciated that the proposed EF system can give benefits for any form of confocality or structured illumination as, due to the stepwise motion, the volume is logically and optically collapsed into a single slice, and the sampling effects and cross correlation that allow for the resolution enhancement in these methods is preserved into the projection thereof. The mere projection without resolution enhancement works for any light microscope that has a distinct focal plane at any time. I.e. it will function with all existing technologies, i.e. phase contrast, DIC, fluorescence, dark field, backscattering, transmission, and all point scanning techniques with the exception of other EF methods and holographic microscopy

The proposal herein thus involves a co-moving field of view microscopy approach where the image is formed by a form of projection and the projection requires a non- negligible amount of time to be acquired, long enough to cause visible motion artifacts.

Such situations commonly occur in but are not limited to: point scanning microscopy where image pixels are recorded one after another or by z-projections where an image is accumulated through several focal planes.

The first premise is that if the recording coordinate frame is moving along with the observed structures, the projection will be artifact depleted. This setup can be achieved by predicting the motion parameters from the artifacts in a single projection or measured from consecutive projections via interpolation.

The second premise is that the EF system can be used to rapidly record a projection - at a much shortened exposure time - hence greatly reducing the observable motion artifacts. For this latter approach, the relative movement of the sample and the focal plane does not necessarily have to follow the object precisely or at all.

The third premise is that more constant and repeatable projections are obtained by “streaking” along the axis of the projection. If the projection changes only slowly over time - due to deformation of rotation - then the lateral corrections can be derived from the direction of maximum blur.

A narrower conclusion and consequence of all three premises is that projections of three dimensional specimen will become more constant and consistent as two degrees of freedom are removed from the observation. A third is removed by the nature of the projection. This allows for a more stringent comparison of the residual motion between consecutive recorded frames. Centroid methods (such as those described in R.E.

Thompson, D.R. Larson, and W.W. Webb, Biophysical Journal, 82:2775-2783, 2002), image moments, and edge detectors such as Canny edge detection (known from A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 8, 1986, pp. 679-698) can be used, finally allowing superresolution tracking of complex objects.

Motion artifacts in projections have the same impact on localization accuracy as for 2D or 3D objects but they are a lot less evident in the image material itself. This can be seen with reference to Figures 4a-4c and 5a-5c. Figures 4a to 4c show, respectively, an image of a cross-section across the centre of a spherical shell, an image of the volume of the sphere, and an image of its projection. Figures 5a to 5c show the impact of a motion artefact on the images of Figures 4a to 4c. It is clear that the projection image of Figure 5c is not affected by the motion artefact in the same way as the cross-section or the volume.

Projections are thus very attractive for tracking as they are bright and can be generated faster than volumes and hence lend themselves for accurate long term localization. The artefacts from generating projections can also be corrected more readily than for volumes, as seen in Figures 6a and 6b.

Figures 7a and 7b show a real world example, a moving endosome. In Figure 7a the image is a simulated representation of the cross section of the moving endosome. The image in Figure 7b shows captured images from a slow projection using the EF system as described herein. It will be appreciated that the image captured by the EF system shows high degree of similarity to the simulated cross-section image, with a motion artefact that is remarkably similar to that simulated in Figure 6b. This motion artefact can be suppressed by either following the movement of the sample or by accelerating the projection recording.

As explained above, the EF system uses a stepwise movement of the focal plane relative to the imaging volume/sample S, such as a motion generated using a piezo electric movement system, with simultaneous control of illumination in order to suppress illumination during periods of movement. Figure 8 shows an example of a control signal for carrying out such a method using a typical microscope device 10. For the rapid recording of 21 consecutive sections on a spinning disk confocal, a ramp generator is devised that records the slices in a single camera exposure. The generator issues a trigger for the camera exposure, then steps on the piezo stage 0.3 pm every 6 spinning disk sectors. During the step, the lasers are blanked and then re-engaged after another full sector has passed. The settling times over these small steps is about 2 ms.

The xy motion is corrected at a much slower pace. The stage compensation motion is kept at constant speed between observations and the target is to keep the observed motion at zero. The speed correction is hence a first order compensation dx/dt~0. To limit the range of observation, discontinuities are inserted in between frames with (x,y):=(0,0). This allows for long observation time windows of 30 minutes or more.

It will be appreciated that the control signal can readily be adapted for differing control systems of the microscope device, depending on the particular requirements thereof for control of the Z-movement and control of the light source. In many cases it is possible to utilise existing microscope devices, such as those shown in Figures 1 to 3, with modification of the control thereof and without the need to upgrade the hardware. It is also possible to manufacture a microscope device 10 with EF focus as an OEM capability.

Claims

CLAIMS:

1. A method of operating a microscope apparatus for obtaining images of microscopic biological objects, the microscope apparatus comprising: an imaging volume with orthogonal X, Y and Z axes; a camera having a focal plane; a light source for illumination of the imaging volume; and a Z-stage movement system for relative movement of the imaging volume and the focal plane in the Z axis; the method comprising: using the camera to obtain an image in a single exposure whilst using the Z-stage movement system for relative movement of the imaging volume and the focal plane; wherein the Z axis movement is a step-wise movement in which there are multiple resting steps providing periods of no relative motion in between multiple periods of relative motion; and wherein during the periods of relative motion the light source is controlled to reduce the illumination of the imaging volume.

2 A method as claimed in claim 1, comprising obtaining horizontal sections during each period of no motion and combining these multiple horizontal sections into a single exposure image.

3. A method as claimed in claim 1 or 2, wherein the biological objects are cells and/or organelles, and the method includes obtaining image data for one or more images of the biological objects and then storing the image data or transmitting the image data to an external computer device.

4. A method as claimed in claim 1, 2 or 3, comprising deactivation of the light source during the periods of motion of the Z-stage movement system, such that the control of the light source is an on-off control synchronised with the step-wise relative motion of the camera and the imaging volume.

5. A method as claimed in any preceding claim, wherein the light source can be directly modulated or switched on and off by a shutter system.

6. A method as claimed in any preceding claim, wherein the Z-stage movement system is arranged for relative movement of the imaging volume and the camera by movement of the imaging volume, by movement of the camera, or both; and wherein the Z- stage movement system comprises a driver for actuating the relative motion.

7. A method as claimed in any preceding claim, wherein the Z-stage movement system is a piezo-electric Z-stage movement system using piezo-electric actuators.

8. A method as claimed in any preceding claim, wherein the step-wise movement is a stop-start motion with a frequency in the range 1-500 Hz for the periods of no relative motion.

9. A method as claimed in any preceding claim, wherein the total distance of travel during the single exposure is in the range of 0.5-1000 pm, with a step size for the step-size Z axis movement that is in the range 50-1000 nm.

10. A method as claimed in any preceding claim, wherein the microscope apparatus includes one or more control system(s) for control of the Z-stage movement system, the camera, and/or the light source.

11. A method as claimed in any preceding claim, wherein the microscope apparatus is configured to operate confocally or with a shallow depth of field.

12. A method as claimed in claim 11 , wherein the microscope apparatus is a confocal microscope apparatus

13. A method as claimed in claim 11 or 12, including deactivating focussing functions of the microscope and/or its camera.

14. A method as claimed in any preceding claim, comprising modifying an existing microscope apparatus, such as via hardware additions or software changes in order to configure the microscope apparatus for carrying out the method.

15. A method as claimed in claim 14, comprising modifying the hardware and/or software for a control system of the microscope apparatus.

16. A method for adapting a microscope apparatus, the method comprising: performing software and/or hardware modifications to the microscope apparatus in order configure the microscope apparatus to perform the method of any preceding claim.

17. A computer programme product comprising instructions for execution on a control system for a microscope apparatus comprising: an imaging volume with orthogonal X, Y and Z axes; a camera having a focal plane; a light source for illumination of the imaging volume; and a Z-stage movement system for relative movement of the imaging volume and focal plane in the Z axis; wherein the instructions, when executed, will configure the control system to: use the camera to obtain an image in a single exposure whilst using the Z-stage movement system for relative movement of the imaging volume and the focal plane; wherein the Z axis movement is a step-wise movement in which there are multiple resting steps providing periods of no relative motion in between multiple periods of relative motion; and wherein during the periods of relative motion the light source is controlled to reduce the illumination of the imaging volume; optionally wherein the instructions, when executed, will configure the microscope apparatus to carry out the method of any of claims 1 to 13.

18. A microscope apparatus for obtaining images of microscopic biological objects, the microscope apparatus comprising: an imaging volume with orthogonal X, Y and Z axes; a camera having a focal plane; a light source for illumination of the imaging volume; and a Z-stage movement system for relative movement of the imaging volume and the focal plane in the Z axis; wherein the microscope apparatus is configured to use the camera to obtain an image in a single exposure whilst using the Z-stage movement system for relative movement of the imaging volume and the focal plane; wherein the Z axis movement is a step-wise movement in which there are multiple resting steps providing periods of no relative motion in between multiple periods of relative motion; and wherein the microscope apparatus is arranged such that during the periods of relative motion the light source is controlled to reduce the illumination of the imaging volume.

19. A microscope apparatus as claimed in claim 18, wherein the microscope apparatus is configured to operate in accordance with the method of any of claims 1 to 13.