WO2010055362A2 - Method and measuring system for scanning a region of interest - Google Patents

Method and measuring system for scanning a region of interest Download PDF

Info

Publication number
WO2010055362A2
WO2010055362A2 PCT/HU2009/000095 HU2009000095W WO2010055362A2 WO 2010055362 A2 WO2010055362 A2 WO 2010055362A2 HU 2009000095 W HU2009000095 W HU 2009000095W WO 2010055362 A2 WO2010055362 A2 WO 2010055362A2
Authority
WO
WIPO (PCT)
Prior art keywords
scanning
trajectory
laser beam
interest
trajectories
Prior art date
Application number
PCT/HU2009/000095
Other languages
French (fr)
Other versions
WO2010055362A3 (en
Inventor
BaIázs RÓZSA
Gergely Katona
E. Szilveszter Vizi
Original Assignee
Femtonics Kft.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from HU0800690A external-priority patent/HU0800690D0/en
Priority claimed from HU0800782A external-priority patent/HU0800782D0/en
Application filed by Femtonics Kft. filed Critical Femtonics Kft.
Publication of WO2010055362A2 publication Critical patent/WO2010055362A2/en
Publication of WO2010055362A3 publication Critical patent/WO2010055362A3/en

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64—Fluorescence; Phosphorescence
    • G01N21/645—Specially adapted constructive features of fluorimeters
    • G01N21/6456—Spatial resolved fluorescence measurements; Imaging
    • G01N21/6458—Fluorescence microscopy
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64—Fluorescence; Phosphorescence
    • G01N21/6428—Measuring fluorescence of fluorescent products of reactions or of fluorochrome labelled reactive substances, e.g. measuring quenching effects, using measuring "optrodes"
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/0004—Microscopes specially adapted for specific applications
    • G02B21/002—Scanning microscopes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
    • G06V10/147—Details of sensors, e.g. sensor lenses
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
    • G06V20/693—Acquisition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/14—Image acquisition
    • G06V30/144—Image acquisition using a slot moved over the image; using discrete sensing elements at predetermined points; using automatic curve following means
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N2201/00—Features of devices classified in G01N21/00
    • G01N2201/10—Scanning
    • G01N2201/105—Purely optical scan
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/0004—Microscopes specially adapted for specific applications
    • G02B21/002—Scanning microscopes
    • G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
    • G02B21/0036—Scanning details, e.g. scanning stages
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B26/00—Optical devices or arrangements for the control of light using movable or deformable optical elements
    • G02B26/08—Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
    • G02B26/10—Scanning systems

Definitions

  • the present invention relates to a method for carrying out measurements on at least one region of interest within a sample via a laser scanning microscope having focusing means for focusing a laser beam and having electro-mechano- optic deflecting means for deflecting the laser beam.
  • the invention further relates to a measuring system for realising the inventive method.
  • Two-dimensional (2D) and three-dimensional (3D) laser scanning technologies have great importance in performing measurements on biological specimens (including scanning, imaging, detection, excitation, etc.) e.g. imaging biological structures or mapping fluorescent markers of cell surface receptors or performing measurements such as uncaging, photo-stimulation, FRET (Fluorescence resonance energy transfer), FLIM (Fluorescence lifetime imaging), etc.
  • the present invention relates to, but is not limited to two-photon laser scanning microscope technology, e.g. the present invention may be implemented on a one-photon laser scanning microscope or on a confocal laser scanning microscope.
  • scanning of a sample may be performed by moving the sample stage e.g. via stepping motors, or by changing the position of the focal spot of a laser beam traversing the microscope optics relative to the sample.
  • the first solution - i.e. moving the sample stage - is complicated to implement when using submerge specimen chambers or when electrical recording is performed on the biological specimen with microelectrodes. Accordingly, in the case of measuring biological specimens it is often preferred to move the focus spot of the laser beam instead of moving the specimen.
  • XY plane focal plane
  • Z axis optical axis
  • electro-mechano-optic technologies exist for deflecting the laser beam in an XY plane, e.g. via deflecting mirrors mounted on galvanometric scanners.
  • the whole of the minimal raster scan area (or volume) is scanned and superfluous data obtained from regions not belonging to the regions of interest are discarded.
  • the prior art rectangular raster scan area (or cuboid volume) may comprise extensive amount of superfluous data, for example in the case of scanning a curved region of interest or a longitudinal region of interest which is diagonal to the scanning axes of the laser scanning microscope, e.g. a portion of a neural process.
  • Fig. 10 illustrates such a rectangular raster scan area 1 needed to comprise the whole of a curved diagonal region of interest 2 comprising a neural process 3.
  • the amount of superfluous data is even larger if the curved region of interest is a 3D object, in which case cuboid raster scan volume is used to cover the whole region of interest (not illustrated).
  • regions of interests are understood to comprise points of interest, straight or curved lines of interest as well as areas and volumes of interest.
  • the invention further relates to a measuring system according to claim 12.
  • Fig. 1 is a schematic diagram of an exemplary laser scanning microscope that can be used in combination with the inventive system and the inventive method.
  • Fig. 2 is an enlarged view of a sample under the objective of the microscope illustrated in Fig. 1.
  • Fig. 3 is a schematic illustration of a focus spot and its point spread function (PSF).
  • PSF point spread function
  • Fig. 4 is a block diagram of a preferred embodiment of the inventive measuring system.
  • Fig. 5 is an illustration of a background image obtained from a biological specimen, wherein a scanning region is defined for scanning a region of interest.
  • Fig. 5a is a schematic illustration of how a region of interest and/or a scanning trajectory may be defined along a neural process.
  • Fig. 6 is a schematic illustration of the scanning trajectories and the cross- over trajectories.
  • Fig. 6a is a schematic illustration of the scanning trajectories defined along a strongly curved neural process.
  • Fig. 7 is a diagram presenting measurement data obtained along the scanning trajectories defined in Fig. 6.
  • Fig. 8 is a diagram of the backprojected measurement data of Fig. 7.
  • Fig. 9a is a diagram of the signal/noise ratio function obtained in response to stimulation.
  • Fig. 9b is a schematic illustration of the measurement data region series providing the values of the signal/noise ratio function of Fig. 9a.
  • Fig. 10 is a diagram illustrating the prior art raster scan area with respect to a region of interest along a neural process.
  • Fig. 1 is a schematic diagram illustrating an embodiment of a laser scanning microscope 10 which can be used in combination with the inventive measuring system and which is suitable for implementing the inventive method.
  • the microscope 10 comprises a laser source 12 providing a laser beam 13; beam deflecting means 14 for deflecting the laser beam 13 in X and Y directions; focusing means 15, in this case a microscope objective 16; drive means 18 for displacing the objective 16 along the optical axis (Z direction) of the objective 16; a sample stage 20 for holding or supporting a sample 22 (e.g. a biological specimen 22') under the objective 16; and a detector 24.
  • XY plane are understood to mean the plane perpendicular to the optical axis (Z axis) of the microscope 10, and X and Y directions are understood to mean two perpendicular axes lying in the XY plane.
  • the optical axis (corresponding to the Z direction) is not necessarily vertical, e.g. the optical axis of a microscope XY suitable for measuring thin tissue slices attached to a sample slide may lie at any desired angle to the vertical direction.
  • the illustrated embodiment represents an upright microscope, it will be apparent to the skilled person that an invert microscope could be applied as well.
  • a dichroic mirror 26 is arranged along the optical path of the laser beam
  • any suitable detector 24 can be used, e.g. a photo multiplier, to detect the emitted or reflected photons.
  • the deflecting means 14 can be any suitable electro-mechano-optic beam deflecting device, such as galvanometric scanning mirrors 14' (mirrors mounted on galvanometric scanners configured to deflect the laser beam 13 in X and Y directions for scanning within a given XY focal plane).
  • galvanometric scanning mirrors 14' mirrors mounted on galvanometric scanners configured to deflect the laser beam 13 in X and Y directions for scanning within a given XY focal plane.
  • Additional optical guide means such as lenses 28 or mirrors (e.g. spherical mirrors guiding the laser beam 13 onto and between the scanning mirrors 14') can be provided to create a desired optical path and to hinder divergence of the laser beam 13.
  • the microscope objective 16 of the illustrated microscope 10 is mounted on the objective drive means 18, which is preferably a piezo positioner 18' capable of providing very fast micro- and even nano-scale displacements, but optionally other types of suitable devices and opto-mechanical solution can be used as well to modify the position of the focal plane within the sample, e.g. electromagnetic positioning of the optical parts, mechanical step motor drives, resonant driving of optical elements mounted on springs, or a modified imaging system objective can be used, wherein only one small lens is moved within the objective, whereby the working distance of the objective (i.e. the position of the focal plane) can be changed without having to move the whole mass of the objective.
  • the objective drive means 18 is preferably a piezo positioner 18' capable of providing very fast micro- and even nano-scale displacements, but optionally other types of suitable devices and opto-mechanical solution can be used as well to modify the position of the focal plane within the sample, e.g. electromagnetic positioning of the optical parts, mechanical step motor drives, resonant driving of optical elements mounted
  • focusing along the Z direction can be achieved in any other known way as well, e.g. acousto-optic deflectors may be used, or the sample stage 20 might be mounted on a mechanical step motor drive and may be displaced relative to the microscope objective 16 (if the nature of the sample 22 does not require that the sample 22 be kept inert).
  • scanning at various focal depths may be achieved using the known two-photon laser excitation technology, however, any other known technology might be used as well, e.g. confocal microscope technology, could be applied in connection with the present invention.
  • the invention can be applied with conventional 2D scanning technologies, wherein scanning is performed on the surface of a sample 22.
  • a two-photon laser scanning microscope 10 uses laser light of lower energy of which two photons are needed to excite a fluorophore in a quantum event, resulting in the emission of a fluorescence photon, which is then detected by the detector 24.
  • the laser source 12 can be a femtosecond impulse laser, e.g. a Mode-locked Ti-sapphire laser providing the laser beam 13 having the required photon flux for the two-photon excitation, while keeping the average laser beam intensity sufficiently low.
  • the laser beam 13 may be made up of discrete laser impulses of MHz repetition rate and femtosecond impulse width.
  • the focal plane 29 of the objective 16 is shifted relative to the sample 22 (depicted as a biological specimen 22' of a neural tissue), thus the focus spot 30 (i.e. the focal volume of the focused laser beam 13) can be moved in the Z direction.
  • the focus spot 30 sweeps an effective focal layer 29' of a certain width d corresponding to the diameter of the effective focus spot, i.e. the focus spot 30 sweeps a quasi planar layer within which measuring points may be excited by the laser beam 13.
  • the width d of the effective focal layer 29' depends on factors such as the type of measurement to be performed, the intensity of the laser beam 13, the sample 22, the quantity and quality of any fluorophore markers, etc.
  • the shape of the focus spot 30 depends on the microscope optics, and in particular on the focusing means (e.g. the microscope objective 16 or acousto-optic deflectors); and can be best described by its point spread function (PSF) (see Fig. 3).
  • PSF point spread function
  • the focus spot 30 has a certain diameter d, when scanning along a scanning line the measured data relates to a band-like scanning region of d width comprising the actual scanning line.
  • the measuring system 50 comprises control means 52 (also indicated in Fig. 1) for controlling the beam deflecting means 14 and the drive means 18 of the objective 16 (or any other device serving to change the focus position relative to the sample 22 along the optical axis of the microscope 10).
  • the control means 52 can be a device such as a computer or a microcontroller or a software application installed on a computer, or a computer program on a computer readable media.
  • the control means 52 may provide drive signals directly for the deflecting means 14 and the objective drive means 18, or alternatively the control means 52 may serve to control a plurality of control units separately controlling components of the microscope 10, such as the objective drive means 18 and the deflecting means 14.
  • the control means 52 is preferably configured to obtain data from the separate control units (such as measurement data from the detector 24; position feedback information from the deflecting means 14 and the objective drive means 18) and for processing such data and for sending back control signals to the control units based on the results of the processing.
  • data from the separate control units such as measurement data from the detector 24; position feedback information from the deflecting means 14 and the objective drive means 18
  • the measuring system 50 preferably further comprises at least one data output interface 54, which may be a visual display of a software application that can be displayed on information presentation means 56 such as a monitor, display, printer or the like.
  • information presentation means 56 such as a monitor, display, printer or the like.
  • the data output interface 54 may itself comprise such information presentation means or the like.
  • the measuring system 50 preferably further comprises at least one data input interface 58, which may be a user interface allowing the user to input data, or it may serve for other software applications for inputting data to be used by the control system 52 or to be displayed by the data output interface.
  • the data input interface 58 may be a physical interface such as a keyboard, mouse, touch screen, etc., or it may be a software interface, e.g. an active field in a graphical interface where data may be inputted.
  • the data output interface 54 and the data input interface 58 may form a single interface means, e.g. a touch screen may be provided as data input interface 58 for inputting measurement commands, and at the same time serve as the data output interface 54, displaying e.g. measurement data.
  • a software may provide a graphical display area for presenting data as well as for inputting data e.g. via active data input fields.
  • the regions of interest correspond typically to axons or dendrite portions or spines, which may fall substantially within a single focal layer 29', thus very often it is sufficient to perform the measurements at a given objective position (or focal plane position, should depth focusing be performed in any other way than displacing the objective 16 and/or sample 22 relative to each other, e.g. by changing the settings of acousto- optic deflectors).
  • a background image 80 (Fig. 5) of the biological specimen 22' is obtained by any conventional imaging technique.
  • the background image 80 comprises a dendrite portion 82 and dendritic spines 84.
  • a regions of interest 86 may be defined e.g. by a user via the data input interface 58 or e.g. via an external or an internal image processing application automatically by identifying the dendrites 82 and spines 84 within the background image 80.
  • Such an external image processing application may use the data output interface 54 to obtain background image data and may use the data input interface 58 for returning the data of the calculated regions of interest 86 (e.g. dendrites 82 and spines 84).
  • the background image 80 is graphically displayed by the data output interface 54 and the user may chose the regions of interest 86 and/or a scanning trajectory 92a by selecting discrete guide points 90 in the background image 80 via the data input interface 58 in order to define a region to be measured (scanned). For example the user may select primary guide points 90a (illustrated with crosses) along the dendrite portion 82 and auxiliary guide points 90b (illustrated with dots) which need also be included in the measured region as illustrated in Fig. 5a.
  • a primary scanning trajectory 92a is calculated for scanning along the dendrite portion 82 defined by the selected primary guide points 90a.
  • the control means 52 preferably includes a calculating means and the primary scanning trajectory 92a may be calculated by the control means 52 of the measuring system 50 or it may be calculated by an external calculating means (not depicted) using any suitable interpolation algorithm.
  • the measuring system 50 is configured to take the physical properties of the scanner motors of the electro-mechano-optic deflecting means 14 and the provided measurement particulars (such as the desired scanning speed, the spatial position of the selected guide points 90, etc.) into account in the interpolation algorithm, in order not to exceed the maximum speed and maximum acceleration of the scanner motors.
  • the interpolation may be based on solving the differential equation of the motion of the scanner motors or at least on modelling the scanner motors' motion. If the maximum speed and acceleration of the motors are not exceeded the scanning trajectory 92a may be scanned repetitively without any substantial deviation between each scan.
  • the calculating means is configured to provide a plurality of spaced apart auxiliary scanning trajectories 92b running along the primary scanning trajectory 92a as illustrated in Fig. 6, in order to cover the region of interest 86 defined by the guide points 90.
  • the choice of the spacing 89 between the scanning trajectories 92 depends on factors such as the effective diameter d of the focus spot 30 and the required measurement data quality.
  • the spacing is preferably not greater than the effective diameter d of the focus spot 30, thereby when scanning along the scanning trajectories 92 the focus spot 30 sweeps overlapping bands of width d, thus no regions are left unmeasured within the region of interest 86.
  • each spacing 89 has a width of about d/2, whereby measurement data is obtained from each spacing 89 when scanning along both boundary scanning trajectories 92.
  • the auxiliary scanning trajectories 92b may be provided by translation of the primary scanning trajectory 92a (as illustrated in Fig. 6) or by generating auxiliary scanning trajectories 92b with constant spacing 89 from each other in case the primary scanning trajectory 92b is strongly curved (as illustrated in Fig. 6a), the apparent difference being that in the later case the length of the scanning trajectories 92 varies within the region of interest 86.
  • the user selected auxiliary guide points 92b may serve to define distant measuring points (e.g. end points of dendritic spines 84 belonging to the dendrite portion 82), which need to be included in the measurement as well.
  • the calculating means may then calculate the number and position of auxiliary scanning trajectories 92b necessary for covering all of the auxiliary guide points 90b as well.
  • the primary and auxiliary scanning trajectories 92 thus cover the whole region of interest 86 defined by the guide points 90.
  • the calculating means may calculate the auxiliary scanning trajectories 92 without auxiliary guide points 90b, e.g. the dendritic spines 84 belonging to the selected dendrite portion 82 may be identified based on shape recognition or other data analysing procedure.
  • the user may change or delete each selected guide point 90 via the data input interface 58 in order to influence the interpolated primary scanning trajectory 92a and/or the calculated auxiliary scanning trajectories 92b. For example the user may find that the defined region of interest 86 is too small or too large and may wish to add or delete or displace auxiliary guide points 90b accordingly.
  • the calculating means of the measuring system 52 is configured to provide a scanning sequence for scanning the scanning trajectories 92.
  • the scanning sequence is simply the order of the neighbouring scanning trajectories 92, e.g. starting from the topmost scanning trajectory 92 as illustrated in Fig. 6.
  • the calculating means of the measuring system 52 is further configured to provide cross-over trajectories 94 between two consecutive scanning trajectories 92 in the scanning sequence.
  • the cross-over trajectories 94 preferably connect opposite ends of consecutive scanning trajectories 92 in the scanning sequence.
  • the scanning sequence may be repeated over and over again, thereby allowing for different measurements, such activation of photo-sensitive molecules such as uncaging, photo-stimulation, FRET (Fluorescence resonance energy transfer), FRAP (Fluorescence recovery after photobleaching), or measurements such as SHG (Second harmonic generation), or OCT (Optical coherent tomography), FLIM (Fluorescence lifetime imaging), STED (Stimulated Emission Depletion), etc., and for calculating average measurement data of all the scanning cycles (one cycle corresponding to scanning along all the scanning trajectories 92).
  • photo-sensitive molecules such as uncaging, photo-stimulation, FRET (Fluorescence resonance energy transfer), FRAP (Fluorescence recovery after photobleaching), or measurements such as SHG (Second harmonic generation), or OCT (Optical coherent tomography), FLIM (Fluorescence lifetime imaging), STED (Stimulated Emission Depletion), etc.
  • the scanning beam 13 may be switched off via any sufficiently fast shutter device, or the laser illumination may be switched off temporarily using a sufficiently fast intensity modulator e.g. a Pockels-cell.
  • directing or moving the focus spot 30 is understood to comprise directing or moving a virtual focus spot as well.
  • the position of the focus spot 30 is determined by the settings of the microscope 10 (e.g. position of the scanning mirrors 14' and objective 16 or frequency settings of acousto-optic focusing means)
  • directing or moving the focus spot 30 amounts to changing the settings of the microscope 10 in a way that would bring the focus spot 30 of a laser beam 13 traversing the microscope optics to the desired new position.
  • the meaning of the term cross-over trajectory 94 comprises the possibility of a virtual trajectory as well along which the virtual focus spot would be moving had the laser beam 13 not been switched-off.
  • directing or moving the focus spot 30 is understood to comprise controlling a electro-mecha no-optic deflecting means 14 such that a laser beam 13 traversing the deflecting means 14 would be deflected relative to the optical axis of the microscope 10, even if the state of other components of the microscope 10 do not allow for the actual focusing of the laser beam 13, hence no focus spot 30 would be produced even if the laser beam 13 would be allowed to pass.
  • acousto- optic focusing means for performing depth focusing i.e. for changing the focal plane 29
  • the laser beam 13 might be spatially scattered while changing between two focal plane positions.
  • the diagram in Fig 7 illustrates measurement data obtained in a scanning sequence covering all the scanning trajectories 92 of Fig. 6.
  • the vertical axis of the diagram corresponds to the scanning position along a scanning trajectory 92; the horizontal axis corresponds to the scanning trajectories 92, i.e. the measurement data obtained along each scanning trajectory 92 is depicted in different rows of the diagram.
  • the scanning trajectories 92b are all of the same length resulting in the rectangular measurement data diagram of Fig. 7. If, however, the auxiliary scanning trajectories 92b are defined with equal spacing 89, then the length of the scanning trajectories may substantially vary (as illustrated in Fig. 6a) leading to a trapezoidal measurement data diagram (not illustrated).
  • the obtained measurement data can be transformed back to real-space coordinates; the transformation is referred to as backprojection.
  • Fig. 8 shows the real-space measurement data of the scanned region of interest 86 obtained by backprojection of the measurement data diagram of Fig. 7.
  • Various algorithms may be applied for performing the backprojection.
  • the real-space position of the deflecting means 14 may be obtained by the control means 52 from a position feed-back device, thereby each measurement data can be associated with the real-space position of the theoretical focal point roughly corresponding to the centre of the focus spot 30 of the laser beam 13. Any kind of interpolation algorithm may be used to obtain measurement values between the actual measurement data points.
  • the backprojected real-space measurement data maybe outputted by the data output interface (e.g. graphically presented as illustrated in Fig. 8) thereby allowing a user to estimate the quality of the measurement. If the quality is found unsatisfactory (e.g. the resulting image is too blurred, or the spines 84 are not distinguishable) the user may change the measurement particulars such as the scanning speed, the cross-over speed, the number of auxiliary scanning trajectories 92b, etc. In order to enhance accuracy of the measurement data, the measurement data may be corrected by background light intensity measured in a neutral region of the specimen 22", preferably in the proximity of the region of interest 86.
  • an initial measurement data may be obtained before stimulating the specimen 22, after which stimulation may be applied, such as photo-stimulation (e.g. uncaging, activation of photo-sensitive molecules).
  • stimulation such as photo-stimulation (e.g. uncaging, activation of photo-sensitive molecules).
  • the scanning trajectories 92 are scanned again, and the obtained measurement data is preferably spatially normalised by the initial measurement data.
  • Fig. 9a shows the signal/noise (S/N) ratio function obtained along a dendritic spine 84 (Fig. 9b) in response to stimulation.
  • the invention provides a new and inventive method allowing for a fast and efficient way to scan non-rectangular regions of interest 86, for example the region of interest 86 defined by the user as explained in connection with Fig. 5a.
  • the method comprises providing a primary scanning trajectory 92a for the region of interest 86.
  • 92b are provided along the primary scanning trajectory 92a within the region of interest 86, e.g. as explained in connection with Fig. 6 or 6a.
  • a scanning sequence is provided for scanning all the scanning trajectories 92. Practically, the neighbouring scanning trajectories 92 are scanned sequentially as explained in connection with Fig. 6.
  • cross-over trajectories 94 are provided between the scanning trajectories 92 of two consecutive scanning trajectories 92 in the scanning sequence.
  • the cross-over trajectories 94 preferably extend between opposite ends of neighbouring scanning trajectories 92 as illustrated in Fig. 6. If the scanning sequence needs to be repeated, a cross-over trajectory 94 is provided between the first and last scanning trajectory 92 in the sequence.
  • the measurements are carried out by deflecting the laser beam 13 via the electro-mechano-optic means 14 in order to move the focus spot 30 of the focused laser beam 14 along the scanning trajectory 92.
  • moving the focus spot 30 comprises moving a virtual focus spot.
  • the focus spot 30 is moved at a constant or at a varying scanning speed.
  • the scanning speed along the scanning trajectories 92 is typically substantially constant, however this is not a requirement for performing the inventive method, thus the scanning speed is described by its average value.
  • the scanning speed is generally selected so as to allow for satisfactory measurement data quality.
  • moving the focus spot 30 includes moving a virtual focus spot (i.e. appropriately controlling the angular speed of the scanning mirrors 14').
  • the focus spot 30 is moved at a generally varying cross-over speed: at an increasing speed along a first part of the cross-over trajectory 94 and at a decreasing speed along a second part of the cross-over trajectory 94.
  • the cross-over speed preferably reaches a maximum, which is higher than the average scanning speed.
  • the cross-over speed may also be constant, in which case the maximum cross-over speed corresponds to the cross-over speed itself. Due to the mechanical properties of the deflecting means 14 the actual trajectory 95 of the focus spot 30 may differ substantially from the provided cross-over trajectory 94.
  • the inventive method allows for decreasing the overall scanning time firstly by reducing the area to be scanned by scanning a non-rectangular region of interest 86, and secondly by moving the focus spot 30 along the cross-over trajectories 94 substantially faster than the scanning speed required along the scanning trajectories 92.
  • a back measurement technique is preferably applied in connection with the inventive method.
  • the back measurement comprises measuring a background image 80 of the specimen 22 such as the one illustrated in Fig. 6b.
  • a background image 80 of the specimen 22 such as the one illustrated in Fig. 6b.
  • at least one region of interest is selected on the background image 80 by the user or by the measuring system 50 (e.g. via a shape recognition software running on the measuring system 50) or by an external software or device.
  • the measurement particulars may also be provided by the user via the user data input interface 58, or they can be pre-given settings and instructions to be carried out by the measuring system 50.
  • the measurement particulars may include particulars relating to scanning speed of all, some or a specific scanning trajectory 92, crossover speed of all, some or a specific cross-over trajectory 94, overall scanning time, definition of the regions of interest 86, etc.
  • a first sequence of measurements is performed to obtain calibration data.
  • this first sequence of measurement time dependent position data of the deflector 14 is obtained (e.g. using known position feed-back devices) and the position of the focus spot 30 of the laser beam 13 is measured back to the XY scanning plane (i.e. the focus plane 29).
  • the measured back position of the focus spot 30 of the laser beam 13 may be displayed against the background image 80 via the information presentation means 56, thus allowing the user to verify accuracy of the actual trajectory.
  • the back measurement is advantageously used to correct any positioning error of the focal point along the scanning trajectory 92.
  • the deflector 14 is no longer able to trace the scanning trajectory 92 precisely, thus the actual trajectory 95 may be offset from the theoretical scanning trajectory 92 by a small but significant error. Since this error has been found to be relatively stable over the repetitive scanning cycles along the composite trajectory 99, the actual trajectory 95 may be visualized by the back measurement technique and the error may be corrected by the user by means of offsetting the measurement particulars in the same amount, or the correction may be automated using known convergence algorithms.
  • the method according to the invention preferably comprises the steps of:
  • the error function is reduced by an iterative convergence algorithm comprising:
  • a laser scanning microscope 10 having focusing means (such as the objective 16) for focusing a laser beam 13 and having electro-mechano-optic deflecting means 14 for deflecting the laser beam 13 is used for performing the actual measurements on the at least one region of interest 86 within the examined sample 22 (e.g. a biological specimen 22', such as neural processes).
  • the laser scanning microscope 10 itself is not necessarily part of the measuring system 50.
  • the measuring system 50 may be a software application on a computer readable medium.
  • the measuring system 50 comprises control means 52 being configured to control the electro-mechano-optic means 14 so as to
  • deflect the laser beam 13 for moving the focus spot 30 of the laser beam 13 along a cross-over trajectory 94 at a cross-over speed having a maximum, the maximum of the cross-over speed being higher than the average scanning speed.
  • the control means 52 may include the calculating means configured to calculate the auxiliary scanning trajectories 92b and the cross-over trajectories 94.
  • the cross-over trajectories 94 and the cross-over speed along the crossover trajectories may be calculated based on any of the methods proposed in the parallel European patent application "METHOD AND MEASURING SYSTEM FOR SCANNING MULTIPLE REGIONS OF INTEREST" of the applicant contents of which are hereby incorporated by reference.
  • inventive method is also suitable for scanning a region of interest in 3D, e.g. based on the principles described in the above referenced European patent application and Hungarian patent application P08 00433 the contents of both being incorporated herein by reference.

Abstract

The invention relates to a method for scanning a region of interest, such as a portion of a neural process, via a laser scanning microscope having focusing means for focusing a laser beam and having electro-mechano-optic deflecting means for deflecting the laser beam, the method comprising: providing a primary scanning trajectory (92a) for the at least one region of interest; providing a plurality of spaced apart (89) auxiliary scanning trajectories (92b) running along the primary scanning trajectory within the region of interest; providing a scanning sequence for scanning the scanning trajectories (192); providing cross-over (94) trajectories between the scanning trajectories of two consecutive scanning trajectories in the scanning sequence. The invention further relates to a measuring system for implementing the method according to the invention.

Description

Method and measuring system for scanning a region of interest
The present invention relates to a method for carrying out measurements on at least one region of interest within a sample via a laser scanning microscope having focusing means for focusing a laser beam and having electro-mechano- optic deflecting means for deflecting the laser beam.
The invention further relates to a measuring system for realising the inventive method.
Two-dimensional (2D) and three-dimensional (3D) laser scanning technologies have great importance in performing measurements on biological specimens (including scanning, imaging, detection, excitation, etc.) e.g. imaging biological structures or mapping fluorescent markers of cell surface receptors or performing measurements such as uncaging, photo-stimulation, FRET (Fluorescence resonance energy transfer), FLIM (Fluorescence lifetime imaging), etc. The present invention relates to, but is not limited to two-photon laser scanning microscope technology, e.g. the present invention may be implemented on a one-photon laser scanning microscope or on a confocal laser scanning microscope.
When applying either 2D or 3D scanning technologies scanning of a sample may be performed by moving the sample stage e.g. via stepping motors, or by changing the position of the focal spot of a laser beam traversing the microscope optics relative to the sample. The first solution - i.e. moving the sample stage - is complicated to implement when using submerge specimen chambers or when electrical recording is performed on the biological specimen with microelectrodes. Accordingly, in the case of measuring biological specimens it is often preferred to move the focus spot of the laser beam instead of moving the specimen. This can be achieved by deflecting the laser beam to scan different points within a given focal plane (XY plane) and - in case of 3D microscopy - by displacing the objective along its optical axis (Z axis), e.g. via a piezo-positioner, or by using acousto-optic deflectors to change the depth of the scanned focal plane. Several known electro-mechano-optic technologies exist for deflecting the laser beam in an XY plane, e.g. via deflecting mirrors mounted on galvanometric scanners.
As indicated above, when performing certain types of biological measurements very often only a fraction of the whole specimen is of interest. For example, when measuring neural processes typically only a few dendrite, axons, spines or the like are under examination and any additional scanning data obtained from the rest of the biological tissue does not contribute to the actual measurement. In order to enhance the efficiency of the measurement prior art laser scanning technologies aim at reducing the area (or volume) of the superfluous regions which are inevitably scanned together with the regions of interest (single points, straight or curved lines, areas, volumes of interest). Using prior art raster scanning technologies this can be achieved by selecting a minimal rectangular raster scan area (or cuboid volume) containing all the regions of interest. The whole of the minimal raster scan area (or volume) is scanned and superfluous data obtained from regions not belonging to the regions of interest are discarded. However the prior art rectangular raster scan area (or cuboid volume) may comprise extensive amount of superfluous data, for example in the case of scanning a curved region of interest or a longitudinal region of interest which is diagonal to the scanning axes of the laser scanning microscope, e.g. a portion of a neural process. Fig. 10 illustrates such a rectangular raster scan area 1 needed to comprise the whole of a curved diagonal region of interest 2 comprising a neural process 3. The amount of superfluous data is even larger if the curved region of interest is a 3D object, in which case cuboid raster scan volume is used to cover the whole region of interest (not illustrated).
It is an object of the invention to overcome the problems associated with the prior art laser scanning microscopes. In particular, it is an object of the invention to further enhance the prior art scanning technologies by providing a method and system for reducing the scanning time spent on other parts of a specimen than the regions of interest.
It is a further object of the present invention to combine the inventive scanning technology with existing 2D and 3D laser scanning technologies, for the purpose of reducing the overall scanning time needed for scanning regions of interest in 2 D or 3D respectively.
The above objects are achieved by a method according to claim 1.
In the context of the present invention regions of interests are understood to comprise points of interest, straight or curved lines of interest as well as areas and volumes of interest.
The invention further relates to a measuring system according to claim 12.
Further advantageous embodiments of the invention are defined in the attached dependent claims.
Further details of the invention will be apparent from the accompanying figures and exemplary embodiments.
Fig. 1 is a schematic diagram of an exemplary laser scanning microscope that can be used in combination with the inventive system and the inventive method.
Fig. 2 is an enlarged view of a sample under the objective of the microscope illustrated in Fig. 1.
Fig. 3 is a schematic illustration of a focus spot and its point spread function (PSF).
Fig. 4 is a block diagram of a preferred embodiment of the inventive measuring system. Fig. 5 is an illustration of a background image obtained from a biological specimen, wherein a scanning region is defined for scanning a region of interest.
Fig. 5a is a schematic illustration of how a region of interest and/or a scanning trajectory may be defined along a neural process.
Fig. 6 is a schematic illustration of the scanning trajectories and the cross- over trajectories.
Fig. 6a is a schematic illustration of the scanning trajectories defined along a strongly curved neural process. Fig. 7 is a diagram presenting measurement data obtained along the scanning trajectories defined in Fig. 6.
Fig. 8 is a diagram of the backprojected measurement data of Fig. 7.
Fig. 9a is a diagram of the signal/noise ratio function obtained in response to stimulation.
Fig. 9b is a schematic illustration of the measurement data region series providing the values of the signal/noise ratio function of Fig. 9a.
Fig. 10 is a diagram illustrating the prior art raster scan area with respect to a region of interest along a neural process. Fig. 1 is a schematic diagram illustrating an embodiment of a laser scanning microscope 10 which can be used in combination with the inventive measuring system and which is suitable for implementing the inventive method. The microscope 10 comprises a laser source 12 providing a laser beam 13; beam deflecting means 14 for deflecting the laser beam 13 in X and Y directions; focusing means 15, in this case a microscope objective 16; drive means 18 for displacing the objective 16 along the optical axis (Z direction) of the objective 16; a sample stage 20 for holding or supporting a sample 22 (e.g. a biological specimen 22') under the objective 16; and a detector 24.
In the context of the present invention XY plane are understood to mean the plane perpendicular to the optical axis (Z axis) of the microscope 10, and X and Y directions are understood to mean two perpendicular axes lying in the XY plane. Furthermore, it should be noted that the optical axis (corresponding to the Z direction) is not necessarily vertical, e.g. the optical axis of a microscope XY suitable for measuring thin tissue slices attached to a sample slide may lie at any desired angle to the vertical direction. Also, although the illustrated embodiment represents an upright microscope, it will be apparent to the skilled person that an invert microscope could be applied as well.
A dichroic mirror 26 is arranged along the optical path of the laser beam
13 to separate the laser beam 13 provided by the laser source 12 from the reflected beam 13' reflected from the sample or back fluoresced light beam 13" emitted by the sample, e.g. the fluoroscence photons emitted by excited fluorophores within the sample 22 under examination. Any suitable detector 24 can be used, e.g. a photo multiplier, to detect the emitted or reflected photons.
Also, more than one detectors 24 provided with appropriate wavelength filters may be arranged in a known way if emitted photons of different wavelengths are to be detected separately. The deflecting means 14 can be any suitable electro-mechano-optic beam deflecting device, such as galvanometric scanning mirrors 14' (mirrors mounted on galvanometric scanners configured to deflect the laser beam 13 in X and Y directions for scanning within a given XY focal plane).
Additional optical guide means such as lenses 28 or mirrors (e.g. spherical mirrors guiding the laser beam 13 onto and between the scanning mirrors 14') can be provided to create a desired optical path and to hinder divergence of the laser beam 13.
The microscope objective 16 of the illustrated microscope 10 is mounted on the objective drive means 18, which is preferably a piezo positioner 18' capable of providing very fast micro- and even nano-scale displacements, but optionally other types of suitable devices and opto-mechanical solution can be used as well to modify the position of the focal plane within the sample, e.g. electromagnetic positioning of the optical parts, mechanical step motor drives, resonant driving of optical elements mounted on springs, or a modified imaging system objective can be used, wherein only one small lens is moved within the objective, whereby the working distance of the objective (i.e. the position of the focal plane) can be changed without having to move the whole mass of the objective.
Furthermore focusing along the Z direction can be achieved in any other known way as well, e.g. acousto-optic deflectors may be used, or the sample stage 20 might be mounted on a mechanical step motor drive and may be displaced relative to the microscope objective 16 (if the nature of the sample 22 does not require that the sample 22 be kept inert).
Advantageously, scanning at various focal depths may be achieved using the known two-photon laser excitation technology, however, any other known technology might be used as well, e.g. confocal microscope technology, could be applied in connection with the present invention. Furthermore, the invention can be applied with conventional 2D scanning technologies, wherein scanning is performed on the surface of a sample 22.
A two-photon laser scanning microscope 10 uses laser light of lower energy of which two photons are needed to excite a fluorophore in a quantum event, resulting in the emission of a fluorescence photon, which is then detected by the detector 24.
The probability of a near simultaneous absorption of two photons is extremely low requiring a high flux of excitation photons, thus two-photon excitation practically only occurs in the focus spot of the laser beam, i.e. a small ellipsoidal volume having typically a size of approximately 300nm x 300nm x
1000nm.
For the purpose of two-photon laser excitation the laser source 12 can be a femtosecond impulse laser, e.g. a Mode-locked Ti-sapphire laser providing the laser beam 13 having the required photon flux for the two-photon excitation, while keeping the average laser beam intensity sufficiently low. Accordingly the laser beam 13 may be made up of discrete laser impulses of MHz repetition rate and femtosecond impulse width.
As illustrated in Fig. 2 when depth focusing is performed e.g. by displacing the objective 16 along its optical axis (i.e. in the Z direction) the focal plane 29 of the objective 16 is shifted relative to the sample 22 (depicted as a biological specimen 22' of a neural tissue), thus the focus spot 30 (i.e. the focal volume of the focused laser beam 13) can be moved in the Z direction.
When moving the focus spot 30 in the focal plane 29 the focus spot 30 sweeps an effective focal layer 29' of a certain width d corresponding to the diameter of the effective focus spot, i.e. the focus spot 30 sweeps a quasi planar layer within which measuring points may be excited by the laser beam 13. The width d of the effective focal layer 29' depends on factors such as the type of measurement to be performed, the intensity of the laser beam 13, the sample 22, the quantity and quality of any fluorophore markers, etc. Furthermore, the shape of the focus spot 30 depends on the microscope optics, and in particular on the focusing means (e.g. the microscope objective 16 or acousto-optic deflectors); and can be best described by its point spread function (PSF) (see Fig. 3).
Because the focus spot 30 has a certain diameter d, when scanning along a scanning line the measured data relates to a band-like scanning region of d width comprising the actual scanning line.
An exemplary measuring system 50 according to the invention is illustrated in the block diagram of Fig. 4. The measuring system 50 comprises control means 52 (also indicated in Fig. 1) for controlling the beam deflecting means 14 and the drive means 18 of the objective 16 (or any other device serving to change the focus position relative to the sample 22 along the optical axis of the microscope 10). The control means 52 can be a device such as a computer or a microcontroller or a software application installed on a computer, or a computer program on a computer readable media. The control means 52 may provide drive signals directly for the deflecting means 14 and the objective drive means 18, or alternatively the control means 52 may serve to control a plurality of control units separately controlling components of the microscope 10, such as the objective drive means 18 and the deflecting means 14. The control means 52 is preferably configured to obtain data from the separate control units (such as measurement data from the detector 24; position feedback information from the deflecting means 14 and the objective drive means 18) and for processing such data and for sending back control signals to the control units based on the results of the processing.
The measuring system 50 preferably further comprises at least one data output interface 54, which may be a visual display of a software application that can be displayed on information presentation means 56 such as a monitor, display, printer or the like. Alternatively the data output interface 54 may itself comprise such information presentation means or the like.
The measuring system 50 preferably further comprises at least one data input interface 58, which may be a user interface allowing the user to input data, or it may serve for other software applications for inputting data to be used by the control system 52 or to be displayed by the data output interface. Furthermore the data input interface 58 may be a physical interface such as a keyboard, mouse, touch screen, etc., or it may be a software interface, e.g. an active field in a graphical interface where data may be inputted.
The data output interface 54 and the data input interface 58 may form a single interface means, e.g. a touch screen may be provided as data input interface 58 for inputting measurement commands, and at the same time serve as the data output interface 54, displaying e.g. measurement data. Similarly a software may provide a graphical display area for presenting data as well as for inputting data e.g. via active data input fields.
In the case of measuring neural processes the regions of interest correspond typically to axons or dendrite portions or spines, which may fall substantially within a single focal layer 29', thus very often it is sufficient to perform the measurements at a given objective position (or focal plane position, should depth focusing be performed in any other way than displacing the objective 16 and/or sample 22 relative to each other, e.g. by changing the settings of acousto- optic deflectors).
In the following an exemplary embodiment of the method and measuring system 50 according to the invention is described in connection with a biological specimen 22' of neural processes, wherein a region of interest lies substantially in a single focal layer 29', thus no depth scanning needs to be performed apart from initially setting the focal plane 29 to a height at which the focal layer 29' substantially comprises the region of interest.
Preferably a background image 80 (Fig. 5) of the biological specimen 22' is obtained by any conventional imaging technique. As can be seen the background image 80 comprises a dendrite portion 82 and dendritic spines 84. A regions of interest 86 may be defined e.g. by a user via the data input interface 58 or e.g. via an external or an internal image processing application automatically by identifying the dendrites 82 and spines 84 within the background image 80.
Such an external image processing application may use the data output interface 54 to obtain background image data and may use the data input interface 58 for returning the data of the calculated regions of interest 86 (e.g. dendrites 82 and spines 84). In a preferred embodiment the background image 80 is graphically displayed by the data output interface 54 and the user may chose the regions of interest 86 and/or a scanning trajectory 92a by selecting discrete guide points 90 in the background image 80 via the data input interface 58 in order to define a region to be measured (scanned). For example the user may select primary guide points 90a (illustrated with crosses) along the dendrite portion 82 and auxiliary guide points 90b (illustrated with dots) which need also be included in the measured region as illustrated in Fig. 5a. A primary scanning trajectory 92a is calculated for scanning along the dendrite portion 82 defined by the selected primary guide points 90a. The control means 52 preferably includes a calculating means and the primary scanning trajectory 92a may be calculated by the control means 52 of the measuring system 50 or it may be calculated by an external calculating means (not depicted) using any suitable interpolation algorithm. Advantageously the measuring system 50 is configured to take the physical properties of the scanner motors of the electro-mechano-optic deflecting means 14 and the provided measurement particulars (such as the desired scanning speed, the spatial position of the selected guide points 90, etc.) into account in the interpolation algorithm, in order not to exceed the maximum speed and maximum acceleration of the scanner motors. The interpolation may be based on solving the differential equation of the motion of the scanner motors or at least on modelling the scanner motors' motion. If the maximum speed and acceleration of the motors are not exceeded the scanning trajectory 92a may be scanned repetitively without any substantial deviation between each scan.
The calculating means is configured to provide a plurality of spaced apart auxiliary scanning trajectories 92b running along the primary scanning trajectory 92a as illustrated in Fig. 6, in order to cover the region of interest 86 defined by the guide points 90. The choice of the spacing 89 between the scanning trajectories 92 depends on factors such as the effective diameter d of the focus spot 30 and the required measurement data quality. The spacing is preferably not greater than the effective diameter d of the focus spot 30, thereby when scanning along the scanning trajectories 92 the focus spot 30 sweeps overlapping bands of width d, thus no regions are left unmeasured within the region of interest 86. Most preferably each spacing 89 has a width of about d/2, whereby measurement data is obtained from each spacing 89 when scanning along both boundary scanning trajectories 92.
The auxiliary scanning trajectories 92b may be provided by translation of the primary scanning trajectory 92a (as illustrated in Fig. 6) or by generating auxiliary scanning trajectories 92b with constant spacing 89 from each other in case the primary scanning trajectory 92b is strongly curved (as illustrated in Fig. 6a), the apparent difference being that in the later case the length of the scanning trajectories 92 varies within the region of interest 86. The user selected auxiliary guide points 92b may serve to define distant measuring points (e.g. end points of dendritic spines 84 belonging to the dendrite portion 82), which need to be included in the measurement as well. The calculating means may then calculate the number and position of auxiliary scanning trajectories 92b necessary for covering all of the auxiliary guide points 90b as well. The primary and auxiliary scanning trajectories 92 thus cover the whole region of interest 86 defined by the guide points 90. Optionally, the calculating means may calculate the auxiliary scanning trajectories 92 without auxiliary guide points 90b, e.g. the dendritic spines 84 belonging to the selected dendrite portion 82 may be identified based on shape recognition or other data analysing procedure. In a preferred embodiment the user may change or delete each selected guide point 90 via the data input interface 58 in order to influence the interpolated primary scanning trajectory 92a and/or the calculated auxiliary scanning trajectories 92b. For example the user may find that the defined region of interest 86 is too small or too large and may wish to add or delete or displace auxiliary guide points 90b accordingly.
It is also possible to allow the user to draw the primary scanning trajectory 92a manually, and optionally configure the control means 52 to smoothen the user defined primary scanning trajectory 92a in order to obtain a scanning trajectory 92 free of any sharp turns or abrupt change in the scanning speed or direction. Furthermore, the user may select the region of interest 86 by drawing around it.
The calculating means of the measuring system 52 is configured to provide a scanning sequence for scanning the scanning trajectories 92. Generally the scanning sequence is simply the order of the neighbouring scanning trajectories 92, e.g. starting from the topmost scanning trajectory 92 as illustrated in Fig. 6.
The calculating means of the measuring system 52 is further configured to provide cross-over trajectories 94 between two consecutive scanning trajectories 92 in the scanning sequence.
Although it would be possible to provide the cross-over trajectories 94 between the neighbouring ends of the consecutive scanning trajectories 92, however, it has been found that when scanning at higher speeds the actual trajectory 95 of the focus spot 30 is offset from the scanning trajectory 92 in the curved regions of the scanning trajectory 92 as can be seen in Fig. 6b. Furthermore, it has also been found that the direction relative to the calculated scanning trajectory 92 in which the actual scanning trajectory 95 is offset in the curves depends on the scanning direction. This phenomena is very similar to the skidding of vehicles in sharp bends, and is due to the mechanical properties of the deflecting means 14 which is used for displacing the focus spot 30 within a given focal plane 29. Thus, in order to keep an even spacing 89 between the actual scanning trajectories 95 as well, it is proposed to scan all the scanning trajectories 92 starting from one side of the scanned region of interest 86 as illustrated in Fig. 6. Hence the cross-over trajectories 94 preferably connect opposite ends of consecutive scanning trajectories 92 in the scanning sequence.
The scanning sequence may be repeated over and over again, thereby allowing for different measurements, such activation of photo-sensitive molecules such as uncaging, photo-stimulation, FRET (Fluorescence resonance energy transfer), FRAP (Fluorescence recovery after photobleaching), or measurements such as SHG (Second harmonic generation), or OCT (Optical coherent tomography), FLIM (Fluorescence lifetime imaging), STED (Stimulated Emission Depletion), etc., and for calculating average measurement data of all the scanning cycles (one cycle corresponding to scanning along all the scanning trajectories 92).
When there is no biologically relevant structure along the cross-over trajectory 94 or no relevant information to collect, obtaining measurement data along the cross-over trajectory 94 has no influence on the actual measurement to be performed on the regions of interest 86. However, should the regions of interest 86 lie more spaced apart with biologically relevant structures interposed between them it might be preferred to switch off the scanning laser beam 13 while crossing over such biologically relevant structures along the cross-over trajectory 94. The scanning beam 13 may be switched off via any sufficiently fast shutter device, or the laser illumination may be switched off temporarily using a sufficiently fast intensity modulator e.g. a Pockels-cell.
In the context of the present invention directing or moving the focus spot 30 is understood to comprise directing or moving a virtual focus spot as well. As the position of the focus spot 30 is determined by the settings of the microscope 10 (e.g. position of the scanning mirrors 14' and objective 16 or frequency settings of acousto-optic focusing means), directing or moving the focus spot 30 (or a virtual focus spot) amounts to changing the settings of the microscope 10 in a way that would bring the focus spot 30 of a laser beam 13 traversing the microscope optics to the desired new position. Accordingly, the meaning of the term cross-over trajectory 94 comprises the possibility of a virtual trajectory as well along which the virtual focus spot would be moving had the laser beam 13 not been switched-off.
Furthermore, in the context of the present invention directing or moving the focus spot 30 is understood to comprise controlling a electro-mecha no-optic deflecting means 14 such that a laser beam 13 traversing the deflecting means 14 would be deflected relative to the optical axis of the microscope 10, even if the state of other components of the microscope 10 do not allow for the actual focusing of the laser beam 13, hence no focus spot 30 would be produced even if the laser beam 13 would be allowed to pass. For example when using acousto- optic focusing means for performing depth focusing (i.e. for changing the focal plane 29) the laser beam 13 might be spatially scattered while changing between two focal plane positions. However even in this situation the position of the scanning mirrors 14' within the deflecting means 14 allows for determining a virtual trajectory projected onto an XY plane perpendicular to the optical axis of the microscope 10. The diagram in Fig 7 illustrates measurement data obtained in a scanning sequence covering all the scanning trajectories 92 of Fig. 6. The vertical axis of the diagram corresponds to the scanning position along a scanning trajectory 92; the horizontal axis corresponds to the scanning trajectories 92, i.e. the measurement data obtained along each scanning trajectory 92 is depicted in different rows of the diagram. If the auxiliary scanning trajectories 92b are obtained by identical translation, then the scanning trajectories 92 are all of the same length resulting in the rectangular measurement data diagram of Fig. 7. If, however, the auxiliary scanning trajectories 92b are defined with equal spacing 89, then the length of the scanning trajectories may substantially vary (as illustrated in Fig. 6a) leading to a trapezoidal measurement data diagram (not illustrated).
Preferably the obtained measurement data can be transformed back to real-space coordinates; the transformation is referred to as backprojection. Fig. 8 shows the real-space measurement data of the scanned region of interest 86 obtained by backprojection of the measurement data diagram of Fig. 7. Various algorithms may be applied for performing the backprojection. For example the real-space position of the deflecting means 14 may be obtained by the control means 52 from a position feed-back device, thereby each measurement data can be associated with the real-space position of the theoretical focal point roughly corresponding to the centre of the focus spot 30 of the laser beam 13. Any kind of interpolation algorithm may be used to obtain measurement values between the actual measurement data points.
The backprojected real-space measurement data maybe outputted by the data output interface (e.g. graphically presented as illustrated in Fig. 8) thereby allowing a user to estimate the quality of the measurement. If the quality is found unsatisfactory (e.g. the resulting image is too blurred, or the spines 84 are not distinguishable) the user may change the measurement particulars such as the scanning speed, the cross-over speed, the number of auxiliary scanning trajectories 92b, etc. In order to enhance accuracy of the measurement data, the measurement data may be corrected by background light intensity measured in a neutral region of the specimen 22", preferably in the proximity of the region of interest 86. Furthermore, an initial measurement data may be obtained before stimulating the specimen 22, after which stimulation may be applied, such as photo-stimulation (e.g. uncaging, activation of photo-sensitive molecules). After stimulation, the scanning trajectories 92 are scanned again, and the obtained measurement data is preferably spatially normalised by the initial measurement data.
Fig. 9a shows the signal/noise (S/N) ratio function obtained along a dendritic spine 84 (Fig. 9b) in response to stimulation. The stimulation was applied at t=1000 ms resulting in a sudden increase in the signal measured by the detector 24. The invention provides a new and inventive method allowing for a fast and efficient way to scan non-rectangular regions of interest 86, for example the region of interest 86 defined by the user as explained in connection with Fig. 5a. In a first step the method comprises providing a primary scanning trajectory 92a for the region of interest 86. In a second step plurality of spaced apart auxiliary scanning trajectories
92b are provided along the primary scanning trajectory 92a within the region of interest 86, e.g. as explained in connection with Fig. 6 or 6a.
In the following step a scanning sequence is provided for scanning all the scanning trajectories 92. Practically, the neighbouring scanning trajectories 92 are scanned sequentially as explained in connection with Fig. 6.
In the following step cross-over trajectories 94 are provided between the scanning trajectories 92 of two consecutive scanning trajectories 92 in the scanning sequence. The cross-over trajectories 94 preferably extend between opposite ends of neighbouring scanning trajectories 92 as illustrated in Fig. 6. If the scanning sequence needs to be repeated, a cross-over trajectory 94 is provided between the first and last scanning trajectory 92 in the sequence.
The measurements are carried out by deflecting the laser beam 13 via the electro-mechano-optic means 14 in order to move the focus spot 30 of the focused laser beam 14 along the scanning trajectory 92. As explained above, moving the focus spot 30 comprises moving a virtual focus spot. The focus spot 30 is moved at a constant or at a varying scanning speed. The scanning speed along the scanning trajectories 92 is typically substantially constant, however this is not a requirement for performing the inventive method, thus the scanning speed is described by its average value. The scanning speed is generally selected so as to allow for satisfactory measurement data quality.
After having scanned the first scanning trajectory 92 (which is generally not the primary scanning trajectory 92a but the outermost auxiliary scanning trajectory 92b) the laser beam 13 is deflected via the electro-mechano-optic means 14 such that the focus spot 30 of the laser beam 13 is directed along the cross-over trajectory 94 connecting the previously measured scanning trajectory 92 with the subsequent scanning trajectory 92 in the measuring sequence. As explained above, moving the focus spot 30 includes moving a virtual focus spot (i.e. appropriately controlling the angular speed of the scanning mirrors 14'). The focus spot 30 is moved at a generally varying cross-over speed: at an increasing speed along a first part of the cross-over trajectory 94 and at a decreasing speed along a second part of the cross-over trajectory 94. However, the cross-over speed preferably reaches a maximum, which is higher than the average scanning speed. The cross-over speed may also be constant, in which case the maximum cross-over speed corresponds to the cross-over speed itself. Due to the mechanical properties of the deflecting means 14 the actual trajectory 95 of the focus spot 30 may differ substantially from the provided cross-over trajectory 94. The inventive method allows for decreasing the overall scanning time firstly by reducing the area to be scanned by scanning a non-rectangular region of interest 86, and secondly by moving the focus spot 30 along the cross-over trajectories 94 substantially faster than the scanning speed required along the scanning trajectories 92. A back measurement technique is preferably applied in connection with the inventive method. The back measurement comprises measuring a background image 80 of the specimen 22 such as the one illustrated in Fig. 6b. As explained above at least one region of interest is selected on the background image 80 by the user or by the measuring system 50 (e.g. via a shape recognition software running on the measuring system 50) or by an external software or device. The measurement particulars may also be provided by the user via the user data input interface 58, or they can be pre-given settings and instructions to be carried out by the measuring system 50. The measurement particulars may include particulars relating to scanning speed of all, some or a specific scanning trajectory 92, crossover speed of all, some or a specific cross-over trajectory 94, overall scanning time, definition of the regions of interest 86, etc. A first sequence of measurements is performed to obtain calibration data. During this first sequence of measurement time dependent position data of the deflector 14 is obtained (e.g. using known position feed-back devices) and the position of the focus spot 30 of the laser beam 13 is measured back to the XY scanning plane (i.e. the focus plane 29). The measured back position of the focus spot 30 of the laser beam 13 may be displayed against the background image 80 via the information presentation means 56, thus allowing the user to verify accuracy of the actual trajectory.
The back measurement is advantageously used to correct any positioning error of the focal point along the scanning trajectory 92. At very high scanning speeds the deflector 14 is no longer able to trace the scanning trajectory 92 precisely, thus the actual trajectory 95 may be offset from the theoretical scanning trajectory 92 by a small but significant error. Since this error has been found to be relatively stable over the repetitive scanning cycles along the composite trajectory 99, the actual trajectory 95 may be visualized by the back measurement technique and the error may be corrected by the user by means of offsetting the measurement particulars in the same amount, or the correction may be automated using known convergence algorithms.
The method according to the invention preferably comprises the steps of:
- obtaining a background image of the sample;
- defining the at least one region of interest on the background image; - obtaining the measured back position of the focus spot of the laser beam or the time dependent position data of the deflecting means on the background image of the sample
- searching for the "stabile" trajectories of the scanning which are characterized by a "spatiotemporal" accuracy limit as a function of repetition number of the trajectory
- calculating an error function obtained from the difference between the stabile trajectory and the originally defined trajectory - reducing this error function under a certain accuracy limit by changing the position signal of the deflecting means in one or more steps;
- optionally showing the result on the background image.
This manual or automatic process is done at the beginning of the measurement.
The error function is reduced by an iterative convergence algorithm comprising:
- calculating the difference in every point between the stabile trajectory and the originally obtained actual trajectory; - adding an additional compensatory signal to the previous position signal of the deflecting means using a given convergence algorithm (the calculus of compensatory signal is based on the difference between the stabile trajectory and the originally obtained trajectory);
- obtaining the newly measured back position of the focus spot of the laser beam or the time dependent position data of the deflecting means and obtaining the new stabile trajectory ;
- returning to the first step or stop the iteration if the error is smaller then a given accuracy limit;
- optionally showing the result on the background image. Alternatively, the user may manually correct the stabile trajectory in one or more steps changing the curve while visualising the newly obtained position data on the background image in order to approximate the originally planned trajectory. The inventive method may advantageously be carried out with the help of the measuring system 50 according to the invention, as explained above. A laser scanning microscope 10 having focusing means (such as the objective 16) for focusing a laser beam 13 and having electro-mechano-optic deflecting means 14 for deflecting the laser beam 13 is used for performing the actual measurements on the at least one region of interest 86 within the examined sample 22 (e.g. a biological specimen 22', such as neural processes). It should be emphasised that the laser scanning microscope 10 itself is not necessarily part of the measuring system 50. For example the measuring system 50 may be a software application on a computer readable medium. In a preferred embodiment the measuring system 50 comprises control means 52 being configured to control the electro-mechano-optic means 14 so as to
• deflect the laser beam 13 for moving the focus spot 30 of the focused laser beam 13 along a scanning trajectory 92 with an average scanning speed; and
• deflect the laser beam 13 for moving the focus spot 30 of the laser beam 13 along a cross-over trajectory 94 at a cross-over speed having a maximum, the maximum of the cross-over speed being higher than the average scanning speed.
The control means 52 may include the calculating means configured to calculate the auxiliary scanning trajectories 92b and the cross-over trajectories 94. The cross-over trajectories 94 and the cross-over speed along the crossover trajectories may be calculated based on any of the methods proposed in the parallel European patent application "METHOD AND MEASURING SYSTEM FOR SCANNING MULTIPLE REGIONS OF INTEREST" of the applicant contents of which are hereby incorporated by reference.
The inventive method is also suitable for scanning a region of interest in 3D, e.g. based on the principles described in the above referenced European patent application and Hungarian patent application P08 00433 the contents of both being incorporated herein by reference.
The above-described embodiments are intended only as illustrating examples and are not to be considered as limiting the invention. Various modifications will be apparent to a person skilled in the art without departing from the scope of protection determined by the attached claims.

Claims

1. Method for scanning a region of interest, such as a portion of a neural process, via a laser scanning microscope having focusing means for focusing a laser beam and having electro-mechano-optic deflecting means for deflecting the laser beam, the method comprising:
- providing a primary scanning trajectory for the at least one region of interest; - providing a plurality of spaced apart auxiliary scanning trajectories running along the primary scanning trajectory within the region of interest;
- providing a scanning sequence for scanning the scanning trajectories;
- providing cross-over trajectories between the scanning trajectories of two consecutive scanning trajectories in the scanning sequence.
2. The method according to claim 1 , comprising providing the cross-over trajectories such as to connect opposite ends of neighbouring scanning trajectories.
3. The method according to claims 1 or 2, comprising:
- deflecting the laser beam via the electro-mechano-optic means for moving a focus spot of the focused laser beam along a scanning trajectory at an average scanning speed; and
- deflecting the laser beam via the electro-mechano-optic means for moving the focus spot of the laser beam along a cross-over trajectory at a crossover speed having a maximum, the maximum of the cross-over speed being higher than the average scanning speed.
4. The method according to any of claims 1 to 3, comprising transforming the measurement data obtained along the scanning trajectories in order to obtain real-space measurement data.
5. The method according to any of claims 1 to 4, comprising
- measuring background light intensity, preferably in the proximity of one or more regions of interest; - performing the sequence of measurements and obtaining a set of data;
- correcting the set of measurement data with the one or more measured background light intensity.
6. The method according to any of claims 1 to 5, comprising - performing a first sequence of measurements and obtaining a first set of data;
- performing a second sequence of measurements and obtaining a second set of data; and
- spatially normalising the second set of data with the first set of data.
7. The method according to claim 6, wherein the second sequence of measurements includes photo-stimulation such as uncaging, activation of photosensitive molecules, FRAP or includes chemical, electrical or electro-chemical stimulation.
8. The method according to any of claims 1 to 7, comprising
- obtaining a background image of the sample
- defining the at least one region of interest on the background image;
- providing measurement particulars; - performing a first sequence of measurements;
- obtaining time dependent position data of the deflecting means or measuring back the position of the focus spot of the laser beam in an XY plane.
9. The method according to claim 8, comprising - indicating the measured back position of the focus spot of the laser beam or the time dependent position data of the deflecting means on the background image 80 of the sample; - allowing for redefining the measurement particulars in view of the back measurement.
10. The method according to any of claims 1 to 9, comprising - obtaining a background image 80 of the sample;
- selecting the at least one region of interest on the background image 80, by selecting guide points; and
- interpolating a scanning trajectory over the region of interest based on the discrete guide points.
11. The method according to claim 10, comprising
- taking the physical properties of the scanner motor of the electro- mechano-optic deflecting means and the provided measurement particulars into account in the interpolation algorithm, in order not to exceed the maximum speed and maximum acceleration of the scanner motor.
12. Measuring system for carrying out measurements on at least one region of interest within a sample via a laser scanning microscope having focusing means for focusing a laser beam and having electro-mechano-optic deflecting means for deflecting the laser beam, the system comprising:
- means for providing a primary scanning trajectory for the at least one region of interest;
- calculating means configured:
• to provide a plurality of spaced apart auxiliary scanning trajectories running along the primary scanning trajectory in order to cover the region of interest;
• to provide a scanning sequence for scanning the scanning trajectories;
• to provide cross-over trajectories between the scanning trajectories of two consecutive scanning trajectories in the scanning sequence.
13. The measuring system according to claim 12, comprising control means being configured to control the electro-mecha no-optic means so as to
• deflect the laser beam for moving a focus spot of the focused laser beam along a scanning trajectory at an average scanning speed; and
• deflect the laser beam for moving the focus spot of the laser beam along a cross-over trajectory at a cross-over speed having a maximum, the maximum of the cross-over speed being higher than the average scanning speed.
14. The measuring system according to claim 14 or 15, comprising
- data output interface for outputting measurement data, and
- data input interface allowing for selecting guide points within the at least one region of interest within the sample; - the calculating means being further configured to calculate a first scanning trajectory based on the selected guide points.
15. The method according to any of claims 1 to 11 , comprising
- obtaining a background image of the sample;
- defining the at least one region of interest on the background image; - obtaining the measured back position of the focus spot of the laser beam or the time dependent position data of the deflecting means on the background image of the sample;
- searching for the "stabile" trajectories of the scanning which are characterized by a "spatiotemporal" accuracy limit as a function of repetition number of the trajectory
- calculating an error function obtained from the difference between the stabile trajectory and the originally defined trajectory
- reducing this error function under a certain accuracy limit by changing the position signal of the deflecting means in one or more steps; - optionally showing the result on the background image.
16. The method according to claim 15, wherein the error function is reduced by an iterative convergence algorithm comprising: calculating the difference in every point between the stabile trajectory and the originally obtained actual trajectory; adding an additional compensatory signal to the previous position signal of the deflecting means using a given convergence algorithm; obtaining the newly measured back position of the focus spot of the laser beam or the time dependent position data of the deflecting means and obtaining the new stabile trajectory; returning to the first step or stop the iteration if the error is smaller then a given accuracy limit; optionally showing the result on the background image.
PCT/HU2009/000095 2008-11-17 2009-11-17 Method and measuring system for scanning a region of interest WO2010055362A2 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
HU0800690A HU0800690D0 (en) 2008-11-17 2008-11-17 Method for scanning folded frame
HUP0800690 2008-11-17
HU0800782A HU0800782D0 (en) 2008-12-31 2008-12-31 Method and measuring system for scanning a region of interest
HUP0800782 2008-12-31

Publications (2)

Publication Number Publication Date
WO2010055362A2 true WO2010055362A2 (en) 2010-05-20
WO2010055362A3 WO2010055362A3 (en) 2010-07-08

Family

ID=89988691

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/HU2009/000095 WO2010055362A2 (en) 2008-11-17 2009-11-17 Method and measuring system for scanning a region of interest

Country Status (1)

Country Link
WO (1) WO2010055362A2 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013098567A1 (en) 2011-12-28 2013-07-04 Femtonics Kft. Method for the 3-dimensional measurement of a sample with a measuring system comprising a laser scanning microscope and such measuring system
EP3004848A1 (en) * 2013-06-03 2016-04-13 Stichting VU-VUmc Method and system for imaging a molecular strand
CN106596410A (en) * 2016-11-22 2017-04-26 西安工程大学 Method for quickly detecting property of active sludge through light deflection
WO2018042214A3 (en) * 2016-09-02 2018-04-19 Femtonics Kft Method for scanning along a 3-dimensional line and method for scanning a region of interest by scanning a plurality of 3-dimensional lines
US20220042790A1 (en) * 2018-07-03 2022-02-10 Amo Development, Llc Water-immersed high precision laser focus spot size measurement apparatus

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE10043986B4 (en) * 2000-09-05 2020-01-16 Leica Microsystems Cms Gmbh Procedure for examining a sample and confocal scanning microscope
DE102004034956A1 (en) * 2004-07-16 2006-02-02 Carl Zeiss Jena Gmbh Method for detecting at least one sample area with a light scanning microscope with linear scanning
WO2006123641A1 (en) * 2005-05-16 2006-11-23 Olympus Corporation Scanning type observing device

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013098567A1 (en) 2011-12-28 2013-07-04 Femtonics Kft. Method for the 3-dimensional measurement of a sample with a measuring system comprising a laser scanning microscope and such measuring system
EP3004848B1 (en) * 2013-06-03 2021-10-27 LUMICKS DSM Holding B.V. Method and system for imaging a molecular strand
EP3004848A1 (en) * 2013-06-03 2016-04-13 Stichting VU-VUmc Method and system for imaging a molecular strand
EP3957983A1 (en) * 2013-06-03 2022-02-23 LUMICKS DSM Holding B.V. Method and system for imaging a molecular strand
CN109952525B (en) * 2016-09-02 2022-02-08 菲托尼克斯公司 Method for scanning a region of interest by scanning a plurality of three-dimensional lines
CN109952525A (en) * 2016-09-02 2019-06-28 菲托尼克斯公司 Method for being scanned along three-dimensional line and the method for scanning interested region by scanning multiple three-dimensional lines
WO2018042214A3 (en) * 2016-09-02 2018-04-19 Femtonics Kft Method for scanning along a 3-dimensional line and method for scanning a region of interest by scanning a plurality of 3-dimensional lines
EP4235292A3 (en) * 2016-09-02 2023-09-06 Femtonics Kft. Method for scanning along a 3-dimensional line and method for scanning a region of interest by scanning a plurality of 3-dimensional lines
JP7418715B2 (en) 2016-09-02 2024-01-22 フェムトニクス・カーエフテー Scanning method along three-dimensional lines and method of scanning region of interest by scanning multiple three-dimensional lines
CN106596410A (en) * 2016-11-22 2017-04-26 西安工程大学 Method for quickly detecting property of active sludge through light deflection
US20220042790A1 (en) * 2018-07-03 2022-02-10 Amo Development, Llc Water-immersed high precision laser focus spot size measurement apparatus
US11561087B2 (en) * 2018-07-03 2023-01-24 Amo Development, Llc Water-immersed high precision laser focus spot size measurement apparatus
US11852460B2 (en) 2018-07-03 2023-12-26 Amo Development, Llc Water-immersed high precision laser focus spot size measurement apparatus

Also Published As

Publication number Publication date
WO2010055362A3 (en) 2010-07-08

Similar Documents

Publication Publication Date Title
US9030547B2 (en) Method and measuring system for scanning multiple regions of interest
US7355702B2 (en) Confocal observation system
US9740166B2 (en) Fluorescence receiving apparatus and fluorescence receiving method
JP4756819B2 (en) Scanning microscope system
US9715095B2 (en) Microscope and method for SPIM microscopy
EP2307921B1 (en) Laser scanning microscope for scanning along a 3d trajectory
US20150286042A1 (en) Microscope and method for spim microscopy
US9229207B2 (en) Laser scanning microscope with focus-detecting unit
WO2010055362A2 (en) Method and measuring system for scanning a region of interest
JP7064796B2 (en) Image reconstruction method, device and microimaging device
EP2573613A1 (en) Information processing apparatus, information processing method, program, and information processing system
WO2010055363A1 (en) Laser scanning microscope
JP2015513065A (en) Method and system for three-dimensional measurement of sample by measurement system equipped with laser scanning microscope
JP2003195174A (en) Scanning laser microscope system
US7238934B2 (en) Microscope apparatus and method for controlling microscope apparatus
EP1806575B1 (en) Examination apparatus
EP2146234A1 (en) Laser scanning microscope
US20220003979A1 (en) Light sheet microscope with line focus
EP3941328B1 (en) Ultra-compact microsystems-based single axis confocal endomicroscope
JP4803974B2 (en) Confocal observation system, light irradiation method, and light irradiation program
JP5212547B2 (en) Microscope device and control method
JP6550673B2 (en) Photoluminescence lifetime measuring apparatus and measuring method
WO2022202565A1 (en) Microscope, observation method, and program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 09801247

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 09801247

Country of ref document: EP

Kind code of ref document: A2