EP2742338B1 - Method for presenting and evaluation of images of microtiter plate properties - Google Patents

Method for presenting and evaluation of images of microtiter plate properties Download PDF

Info

Publication number
EP2742338B1
EP2742338B1 EP12821372.5A EP12821372A EP2742338B1 EP 2742338 B1 EP2742338 B1 EP 2742338B1 EP 12821372 A EP12821372 A EP 12821372A EP 2742338 B1 EP2742338 B1 EP 2742338B1
Authority
EP
European Patent Office
Prior art keywords
microplate
well
positions
plate bottom
well surface
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP12821372.5A
Other languages
German (de)
French (fr)
Other versions
EP2742338A4 (en
EP2742338A1 (en
Inventor
Loren Alan STAUFFER
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Molecular Devices LLC
Original Assignee
Molecular Devices LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Molecular Devices LLC filed Critical Molecular Devices LLC
Publication of EP2742338A1 publication Critical patent/EP2742338A1/en
Publication of EP2742338A4 publication Critical patent/EP2742338A4/en
Application granted granted Critical
Publication of EP2742338B1 publication Critical patent/EP2742338B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/0004Microscopes specially adapted for specific applications
    • G02B21/002Scanning microscopes
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/24Base structure
    • G02B21/241Devices for focusing
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01BMEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00Measuring arrangements characterised by the use of optical techniques
    • G01B11/02Measuring arrangements characterised by the use of optical techniques for measuring length, width or thickness
    • G01B11/06Measuring arrangements characterised by the use of optical techniques for measuring length, width or thickness for measuring thickness ; e.g. of sheet material
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/01Arrangements or apparatus for facilitating the optical investigation
    • G01N21/13Moving of cuvettes or solid samples to or from the investigating station
    • G01N2021/135Sample holder displaceable
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/17Systems in which incident light is modified in accordance with the properties of the material investigated
    • G01N21/25Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
    • G01N21/251Colorimeters; Construction thereof
    • G01N21/253Colorimeters; Construction thereof for batch operation, i.e. multisample apparatus

Definitions

  • This invention relates to imaging systems used in examining physical, chemical or biological characteristics of samples in multiple well sample plates, and more particularly to systems and methods for acquiring images for evaluation and presentation of sample plate properties.
  • Materials and objects typically analyzed visually using an imaging system may be analyzed using image acquisition systems capable of handling multiple samples using some level of automation.
  • image acquisition systems typically include a light source (such as for example, a white light or a fluorescent light source), a multiple sample plate, and an objective lens optically coupled to an image recording medium (or for example, a camera) to capture an image of the sample that may be substantially magnified.
  • the multiple sample plate may rest or fit onto a sample-positioning mechanism that moves the plate in the optical path of the light source and objective lens.
  • Examples of automated multiple-sample imaging systems include for example, microscopes and analyzers, such as analyzers that may use imaging for analysis of images captured in performing assays.
  • One example of a type of automated imaging system is a high content screening system in which the high-resolution imaging and analysis of samples is automated.
  • the analysis may entail recognizing and segmenting objects, arid extracting the features of interest for high content image analysis typically requires the use of high content visualization tools.
  • One particular example of a high content screening system is the Leica HCS A made by Leica Microsystems.
  • Image acquisition systems may acquire images of samples placed in multiple sample plates implemented as microwell or microplates having a plurality of wells or cavities. It is noted that alternative terms, such as for example, "microwell,” “microtiter,” and “microplate,” are used herein as reference to the plates.
  • the samples are placed in wells arranged in a matrix within the microwell plate.
  • the Leica HCS A uses a software component called the High Content Screening LAS AF ("Leica Application Suite Advanced Fluorescence") MATRIX M3 automation software.
  • the Leica HCS A system also includes a confocal microscope system.
  • the LAS AF MATRIX M3 automation software includes an autofocus function that permits acquisition of images, which may be used to generate a focus map.
  • a focus map is an automatically generated three-dimensional image showing true sample topology.
  • the focus map is used for Z-positioning of the object of interest during the scan. According to the size and planarity of the samples, the optimal number and positions of the autofocus points is defined.
  • the CellReporterTM is manufactured by Molecular Devices, LLC.
  • the CellReporterTM includes high-quality, quantitative, data image analysis software to image, analyze, and report on individual cells.
  • the CellReporterTM creates a 3D topological map, or surface map, of the plate-cell attachment surface. Prior to imaging the samples, each plate is scanned while images obtained thereafter are displaying the overall base of the plate.
  • the surface map is used to represent the whole plate contours. In this particular application, the surface map is used to maintain focus across each plate during imaging to eliminate the need to focus at each well before taking an image.
  • the conventional microplate structure comprises a specimen plate portion with wells or cavities, and a bottom plate portion.
  • the well bottom, or surface at the bottom of each well, is the plate-cell attachment surface of each well.
  • the surface of the microplate opposite the well bottoms is the plate bottom.
  • the distance between plate bottom and the well bottoms defines the thickness of the microplate.
  • the thickness of the microplate may range between about 10 ⁇ m and about 1500 ⁇ m depending on the manufacturer.
  • microplates play an important role when used in the system to record images with high resolution (X, Y resolution to 100nm, Z resolution to 300nm) and high magnification (with 4x magnification and higher).
  • Certain problems in obtaining in-focus images of the wells with samples therein are caused by out-of-focus light measurements.
  • the problem in obtaining un-focused images in the well with samples is caused by incorrect determination of the focal plane.
  • the incorrect assignment of the focal plane may be caused by variations in the microtiter plates' properties, for example flatness or the thickness of the plate.
  • the microplates may comprise between 6 and 1536 sample wells or cavities. Such plates are manufactured by a number of companies using a variety of materials. The flatness of the well bottoms and of the plate bottom varies substantially, not only from one manufacturer to another but even from different manufacturing lots. The thickness of the microplate typically varies substantially along the surface of the plate, which may result in incorrect assignment of the focal plane during images.
  • microplate well flatness The importance of the microplate well flatness in obtaining high quality images has been noted by researchers and manufacturers. For example, researchers at GE Healthcare BioSciences Corporation evaluated the 96 and 384-well MartiPlateTM by performing the same assays on a number of microplates so that the biological evaluations on the different plates under test could be compared. Imaging background measurements were made on all plates by measuring fluorescence intensity background areas of selected images; average background gray levels were measured on a selected number of wells in each plate. Plate flatness data was obtained taking the .xdce file output from each image stack, which contained information on the autofocus position for each image. The information on the autofocus position for each image was used to produce a surface plot for each type of plate.
  • the flatness of the plate well was assessed by taking measurements from several fields of view across three wells in each plate type. For each plate type, positions were abstracted from the .xdce file and plotted as a bar graph. The variation of the autofocus position within a given well reveals its flatness deviation.
  • This method may be used to label microplates with flatness characteristics, which enables a user to select microplates with desired flatness characteristics.
  • a number of manufacturers are not conducting this type of experiment with their microplates, or characterizing their microplates in terms of flatness.
  • Flatness characterization has not been made a part of their process for microplate evaluation, and accordingly, flatness characteristics are not part of the manufacturing specifications of the microplates.
  • the different manufacturing processes of the microplates make it difficult or impossible to provide the flatness characteristics for the plate bottom and for the bottom of the wells within the plate.
  • US 2008/0297774 A1 suggests a method and an apparatus for optically determining the thickness of a well bottom in a multi-well plate.
  • a focal plane sensor comprises a broad band wavelength light source which transmits via a lens a focus light beam towards the bottom of the multi-well plate.
  • the focal length of the lens is strongly dependent on the wavelength such that the focal points of different wavelengths are at different distance from the lens.
  • the focal plane sensor detects the reflection of the light from the bottom side of the multi-well plate, wherein a high reflection is achieved at optical density junctions, e.g.
  • the reflected light is analyzed by a spectrometer, wherein in dependency of the wavelength an intensity increase is detected at the junctions. As the focal length for the different wavelengths is calibrated, from their intensity peaks the junctions can be detected and thus the thickness of the well bottom can be determined. Also determining a mean thickness value is provided.
  • micro- plate properties especially with respect to the variation of the thickness along the microplate bottom portion via automated measurements, and to display these properties for acquiring "in focus" microscopy images.
  • the present disclosure provides methods, processes, systems, apparatus, instruments, and/or devices, as described by way of example in implementations set forth below.
  • a method for determining selected properties of a microplate comprising a plurality of sample wells. Each sample is open at a top portion of the microplate and includes a well surface receding into the microplate to a well bottom. The well surface is disposed on the top portion opposite a bottom portion of the microplate.
  • An example method includes the step of moving the microplate to position an initial x-y position on the microplate in an optical path generated by a laser.
  • An objective lens is positioned to be substantially centered in the optical path in a z-direction to an initial z-position.
  • a focusing function is performed to determine a well surface z-position for a current x-y position based on an intensity indicative of a best focus of the well surface.
  • the focusing function is performed again to determine a plate bottom z-position for the current x-y position based on an intensity indicative of a best focus of the plate bottom.
  • the microplate is then repeatedly moved to position a plurality of x-y positions with- in a selected area of the microplate in the optical path.
  • the selected area encompasses the well bottom of one of the wells on the microplate.
  • the well surface z-position and the plate bottom z-position corresponding to each of the plurality of x-y positions are determined by repeating the steps of positioning the objective lens, performing the focusing function to determine the well surface z-position, and performing the focusing function for the plate bottom z-position.
  • the well surface z-positions and the plate bottom z-positions at the plurality of x-y positions are then used to generate data to determine the selected properties of the microplate.
  • the method is further specified in claim 1.
  • a system for determining selected properties of a microplate comprising a plurality of sample wells.
  • An example system comprises a focusing laser for generating a focusing light along an optical path.
  • An objective lens is substantially centered at the optical path and mounted on an objective lens support.
  • the objective lens support includes z motor components configured to move the objective lens along a z-axis.
  • An x-y stage supports the microplate and includes x-y motor components configured to move the x-y stage to selected positions on an x-y plane.
  • a light sensor is positioned in the optical path to receive light reflected from the microplate. The light sensor senses an intensity level of the light reflected from the microplate.
  • the system includes a controller to control the focusing laser, the z motor components, and the x-y motor components.
  • the controller also detects the intensity level measured by the light sensor.
  • the controller comprises a computer-implemented microplate evaluation system, which is configured for:
  • Examples of the systems, methods, and apparatuses may be used in any suitably equipped imaging system such as a high content screening system, or a system that performs assays for diagnostic purposes using imaging to display the results.
  • the description below refers to an imaging system in which advantageous use may be made of the examples described below. It is to be understood that the reference to microplates, imaging systems, or any other example structure is without limitation.
  • microplate refers to any multiple sample plate having a plurality of wells or cavities. The samples to be analyzed are placed in wells arranged in a matrix within the microplate. It is noted that alternative terms, such as for example, “microwell” or “microtiter” may also be used to refer to a microplate. It is to be understood that the terms “microwell,” “microtiter,” and “microplate” may be used interchangeably in the description below.
  • sample refers to any material, object or composition that may be placed in a well of a microplate.
  • Figure 1 is a block diagram of an example of a system 100 for determining microplate properties.
  • the system 100 includes an x-y stage 102 , an objective lens 104 , an objective lens support 106 , a focusing laser 110 , a light sensor (or, laser focus detector) 112, and a controller 116 .
  • the system 100 may be implemented in an imaging system that may further include an imaging camera 120 and a display 122 .
  • the system 100 may operate as a standalone system for evaluating microplates and storing microplate data corresponding to individual microplates for use in analyzing data collected for samples using the individual microplates.
  • the system 100 may be integrated in an imaging system as components built-in as part of the imaging system.
  • the system 100 may also be implemented as a feature provided in a kit for later integration with the imaging system, such as for example, after manufacture of the imaging system.
  • a kit may include a storage medium containing a computer program that implements the functions of the system 100 and components that may not already be part of the imaging system.
  • the focusing laser 110 and the light sensor 112 may be components of such a kit.
  • the x-y stage 102 may be, or may be similar to, an x-y stage typically implemented on systems used to analyze samples. As shown in Figure 1 , a microplate 130 is mounted on the x-y stage 102 for evaluation of selected properties.
  • the x-y stage 102 includes x-y motor components (not shown in Figure 1 ) that moves the x-y stage 102 under control of the controller 116 to selected positions corresponding to points on the microplate 130 .
  • the x-y stage 102 moves along a plane corresponding to the substantially planar configuration of the microplate to x-y positions that may be identified using a calibration function.
  • the objective lens 104 may be any lens suitable for focusing the light from the focusing laser 110 on a point on an object on the x-y stage 102 .
  • the objective lens 104 may be positioned so that an optical path 115 created when the focusing laser 110 is emitting light passes substantially at the center of the objective lens 104.
  • the objective lens 104 In terms of an x-y plane coplanar with the x-y plane defining the space in which the x-y stage 102 moves, the objective lens 104 has a substantially fixed location.
  • the objective lens 104 is configured as shown in Figure 1 to move in a z-axis direction perpendicular to the x-y plane.
  • the objective lens 104 may be mounted or fixed to the objective lens support 106 , which may include z-motion components 108 to move the objective lens 104 in the z-axis direction.
  • the objective lens support 106 and the z-motion components 108 may include a z-reference indicator for homing and calibration of the motion of the objective lens 104 along the z-axis.
  • the optical path 115 may be directed using optics, such as mirrors, lenses, or splitters.
  • Figure 1 shows an optical path guide 114 that may be implemented with a mirror and a beam splitter to guide the optical path 115 of a focusing light 117 towards the objective lens 104, and to guide the optical path 115 of a reflected light 119 from the microplate 130 to the light sensor 112 .
  • the optical path guide 114 may be implemented using conventional optics components to guide the optical path 115 in conventional ways to ensure the focusing light 117 and the reflected light 119 are properly directed as needed in specific implementations.
  • the focusing laser 110 may be any laser suitable for generating a light that may be focused in a region near the x-y stage 102 .
  • the specific laser used for the focusing laser 110 may depend on the light sensor 112 used.
  • the focusing laser 110 should also have a suitable power rating.
  • the light sensor 112 may be any suitable photosensitive device having a suitably linear electrical response to light intensity levels. Examples of photosensitive devices that may be used include, without limitation, a phototransistor, a photodiode, a photomultiplier tube, and a photoresistor.
  • the light sensor 112 may be implemented using an imaging device, which may include a camera, or components that capture images using for example a CCD ("charge-coupled device") or a CMOS camera.
  • the light sensor 12 may be implemented in an example implementation using the imaging device used by the imaging system to analyze samples. For example, if the system 100 is implemented to operate using components of an imaging system, of which the system 100 may be an enhancement feature function, the system 100 may use the imaging camera of the imaging system to detect the laser focus.
  • the example system 100 shown in Figure 1 depicts an imaging camera 120 that is a component of the imaging system in which the system 100 may be implemented.
  • the imaging camera 120 may be mounted so as to receive the reflected light 119 of the optical path 115 during evaluation of a microplate 130 if the imaging camera 120 is also being used for the functions described herein for the light sensor 112 .
  • the imaging camera 120 is used for the primary purpose of the imaging system such as in analyzing samples in the wells of the microplate 130 .
  • the optical path guide 114 may be switched, or inserted into the optical path 115 , to direct the optical path 115 for evaluation functions.
  • the system 100 in Figure 1 includes the controller 116 , which controls the z-motion components 108 , the motion of the x-y stage 102 , the focusing laser 110 , the light sensor 112 , and the imaging camera 120 .
  • the controller 116 may control the motion of the x-y stage 102 , and the z-motion components 108 by controlling corresponding motors.
  • the controller 116 may control the focusing laser 110 by turning the laser 110 on or off.
  • the controller 116 may communicate with the light sensor 112 to receive intensity levels.
  • the controller 116 may also interface with a display 122 as well as with other user interface devices to allow the operator to view displayed results, and to provide user input.
  • the controller 116 may be any suitable processing device or system that includes memory and other computing resources.
  • the controller 116 includes memory or a memory interface to access a computer-implemented microplate evaluation system.
  • the microplate evaluation system may be implemented, for example, as software having functions, such as for example, a function for obtaining microplate data from which selected microplate properties may be determined.
  • the computer-implemented microplate evaluation system may perform methods such a method for determining microplate properties illustrated by the flowchart in Figure 3 .
  • the computer-implemented microplate evaluation system analyzes microplates by constructing an image of the microplate 130 from points of best focus along the z-axis at points on an x-y plane that are within a specified area of the microplate 130 .
  • the specified area may be defined to include several, or all, of the wells of the microplate as shown at 132 .
  • the specified area may also be defined to include a single well as shown at 134 in Figure 1 .
  • the area of interest is mainly the bottom part of the wells on the microplate.
  • the general structure of the microplate 130 may be considered in specifying the area containing the points of primary interest.
  • Figure 2 is a cross-sectional view of a portion of a microplate illustrating the structure of a conventional microplate 200.
  • the microplate 200 includes a top surface 202 and a bottom surface 210 .
  • the top surface 202 (indicated as being the "top" surface for convenience) provides openings to a first well 220 , a second well 222 , and a third well 224 .
  • Each well 220 , 222 , 224 includes a well bottom 204 and well sides 206 .
  • a sample 218 may be placed in the first well 220 , for example.
  • the sample 218 sits on the well bottom 204 .
  • the variations in the flatness of the well bottoms 204 may affect the focus during imaging of the sample 218 for analysis of the sample 218 .
  • the image constructed of the microplate structure provides data that may be used to analyze the flatness of the well bottoms 204 and other physical properties of the microplate 200 .
  • the properties of the microplate 200 may be determined by evaluating the changes in thickness of the area formed by the well bottoms 204 between the surface of the well bottoms 204 and the bottom surface 210 of the microplate 200 .
  • Figure 2 shows a first thickness, t 1 , at one point of the surface of the well bottom 204 of the first well 220 , and a second thickness, t 2 , at another point of the same well 220 .
  • best focus points for the surface of the well bottom 204 and the bottom surface 210 may be identified at x-y points within an area identified by a line indicating a cross-section 134'-134" encompassing the surface of the well bottom 204 of the first well 220 in Figure 2 . Similar areas may be specified to encompass the wells 222 , 224 .
  • Figure 3 is a flowchart illustrating operation of an example method 300 for determining microplate properties.
  • the method 300 illustrated in Figure 3 may be used to evaluate a single well, or to obtain data that may be processed to identify the placements of multiple wells based on the geometry of the microplate 200 as shown in Figure 2 .
  • the data may be analyzed to determine where each well is located in terms of the x-y plane of the microplate 200 .
  • the wells may be identified by determining where the thicknesses (such as t 1 and t 2 ) change dramatically and identifying the locations of dramatic changes as being the side walls 206 of the wells (see Figure 2 ).
  • the method 300 illustrated by the flowchart of Figure 3 is described as follows in the context of the system 100 shown in Figure 1 .
  • Component actors in the steps of the method 300 refer to the components in Figure 1 unless stated otherwise.
  • the method 300 in Figure 3 is started by placing the microplate 130 on the x-y stage 102 as indicated at step 302 .
  • the x-y stage 130 is then moved so that a selected point on the microplate 130 identified by an x, y position is in the optical path 115 .
  • a search is performed along the z-axis to identify the z-position of the plate bottom surface 210 as shown at step 306 .
  • the search is performed by first moving the objective lens 106 along the z-axis at step 314 .
  • an intensity level is retrieved from the light sensor 112 at step 316 .
  • a focus score is determined to identify the z-position that provides the best focus of the plate bottom 210 at step 318 .
  • the z-position of best focus of the plate bottom 210 is identified as the plate bottom z-position at step 308 for the current x,y position.
  • a search is started to determine the z-position of the surface of the well bottom 204 (in Figure 2 ).
  • the search is performed by first moving the objective lens 106 along the z-axis at step 320 .
  • an intensity level is retrieved from the light sensor 112 at step 322 .
  • a focus score is determined to identify the z-position that provides the best focus of the surface of the well bottom at step 324 .
  • the z-position of best focus of the surface of the well bottom 204 is identified as the well bottom z-position at step 312 .
  • the data for any reports or for images of the well that is being collected is updated to indicate the z-position of the plate bottom and the z-position of the well bottom surface for the microplate at the given x,y position.
  • a check is made to determine if there are any other x,y positions in the specified area that require determination of the z-positions of the plate bottom and the well bottom. If there are, the next x,y position is determined and the x-y stage 102 is moved to the next x,y position at step 304 . If the specified area has been covered, the report for the specified area is updated and stored at step 330 .
  • Step 330 may involve creating a database record of z-positions at the x-y positions in the specified area for an identified well. Depending on the area (encompassing a single well, or multiple wells), the database record may contain data for multiple wells. The method may be complete at step 332 for a single well, then repeated for a next well in the microplate. Or, the data may be processed to identify individual wells based on the general geometry of the microplate.
  • autofocusing functions may be integrated into the operation of the imaging system.
  • the autofocusing functions, or components of the autofocusing functions may be used to perform, for example, one or more of steps 306 , 314 , 316 , 318 , 308 , 310 , 320 , 322 , 324 , and 312 in Figure 3 .
  • the method 300 illustrated by the flowchart in Figure 3 performs steps of recording sensor information (steps 316 and 322 , for example), and the steps are described as determining an intensity level.
  • the intensity level indicates the amount of light reflected from the surface of the plate at either the well bottom surface or the plate bottom when the objective lens is at the appropriate distance from the plate.
  • the best focus position is also deemed to be the point at which the intensity level measured is at a maximum out of a group of measurements.
  • the term 'intensity level' in the context of the steps performed in the method 300 illustrated in Figure 3 may be a value read off the sensor, or a value that has been processed to either transform the measured value to an actual intensity value, or to scale or normalize the value to simplify the processing.
  • the term 'intensity level' may refer, for purposes of this description, to any parameter indicative of the amount of light reflected from the plate.
  • the microplate data recorded using the method 300 in Figure 3 is used to automatically measure and display microwell plate properties.
  • the microplate data may be presented and displayed in a 3D format.
  • the 3-D representation of the microplate data may be especially useful in evaluating the variation of the thickness of the wells and the plate bottom.
  • the 3-D representation may provide a visual display of the contours of the microplate as a whole in addition to the contours of the individual well bottoms.
  • the microplate properties may be automatically measured and displayed for multiple locations within individual wells.
  • the measured information may provide the user with feedback on the currently imaged microplate during data acquisition in a real-time format.
  • the imaging system may be performing an analysis of samples.
  • the system for determining microplate properties may be used in real-time to retrieve microplate data, and provide the user with information on the microplate properties in real-time during imaging data acquisition.
  • the microplate data for the microplate being used in imaging data acquisition may be retrieved from memory that stores microplate data for previously screened microplates.
  • the system may include user interface tools and functions that permit manipulation of the displayed plate bottom and well bottom images. For example, such manipulation may allow the user to rotate, or zoom-in and zoom-out or to perform other operations.
  • the images displaying the data for the plate bottom and well bottom may be presented as 3-D or 2-D color-coded diagram.
  • the microplate data that characterizes the plate bottom and well bottom may be collected and stored to conduct automated analysis and to produce reports requested by users.
  • the reports may be used, for example, in a certification system when the manufacturer of an acquisition device certifies that microplate models are compatible with the device, or when certification is required to validate acquisition device accuracy when using a known microplate.
  • Multiple microplates of the same model from the same manufacturer may be analyzed to determine the extent of the variation of the properties of microplates within a manufacturing lot or between different manufacturing lots.
  • the measured data from a plurality of microplates may also be used to automatically adjust acquisition settings during analysis of samples.
  • the results obtained from comparing the properties of multiple microplates may provide information relating to whether particular microplates "failed” or "passed” some desired certification standard. The results may also provide information regarding the overall quality of the microplates.
  • the microplate properties for multiple microplates may also provide a historical record that may be used to adjust acquisition settings for performing imaging of samples. The historical record may be stored as images acquired from various microplates.
  • the analysis of the numerical data may include for example, selecting minimum, maximum and average thickness of each well.
  • the information relating to the numerical values obtained by analysis of the numerical data of the microplate properties may be used to set positions and ranges to optimize the focus speed for the acquisition imaging systems used for analyzing samples.
  • the information relating to the optimization of the acquisition settings may include, for example, indications that the setting ranges are too narrow, which may further indicate that the optimization of the acquisition settings is not allowing the next attempt to obtain focus.
  • the information may indicate if the setting ranges are too broad, which may extend the time to obtain each focus.
  • the provided information may also indicate whether or not the plate is damaged or, for some other reason does not match the required criteria.
  • the 3-D visual display of the properties, and more particularly, the thickness of the plate bottom and well bottom allows for inspection and comparison of the results of such inspection with requested microplate's property.
  • a procedure may be used to obtain feedback on setting quality during analysis of a sample. Feedback procedures may be invoked and the data may be analyzed using the user interface.
  • the microplate properties data may be stored for multiple microplates to monitor quality and equipment compatibility control. For example, when the thickness property of the microplate is accurately defined and displayed, the process of selecting the microplates for testing on the instrumentation becomes more transparent and efficient for user.
  • FIG 4 is a diagram illustrating an example of a 3-D visualization 400 a section of a microplate.
  • the 3-D visualization 400 in Figure 4 depicts multiple top regions 402 indicating a plate top surface and corresponding bottom regions 404 indicating a plate bottom surface. It is noted that the 3-D visualization 400 is displayed with the z-axis to a different scale than the x and y axes in order to better visualize the well and plate contours.
  • the top and bottom regions 402 and 404 are shown positioned over corresponding specified areas of measurement 406 .
  • the specified areas of measurement 406 contain dots 408 indicating x-y positions for which a plate bottom z-position and a well surface z-position have been determined.
  • Rulers for x and y axis positions may be displayed to provide the user with indication of location on the plate.
  • the data for the 3-D visualization 400 in Figure 4 may be collected in real-time as the 3-D visualization 400 is displayed.
  • the 3-D visualization 400 may indicate the progress of the imaging by depicting the filling of the specified areas 406 with dots 408 .
  • the 3-D visualization 400 may be displayed as data is collected during a screening of the microplate performed to evaluate the microplate.
  • the 3-D visualization 400 may also be displayed during image acquisition for analyzing samples.
  • Figure 5 is a diagram illustrating the 3-D visualization of Figure 4 after it has been rotated to show a side view 500 of the microplate.
  • a 3-D visualization such as the 3-D visualization 400 in Figure 4 and the side view 500 in Figure 5 may include status information indicating the status of each focus attempt used to create the 3-D visualization.
  • the status information may be conveyed using combinations of colors, shapes or other unique visual characteristics that may be configured to indicate the success or failure of each focus attempt, as well as of the type of focus performed.
  • the microplate data may be used to generate visualization of the data in other forms.
  • the microplate data may be used to plot a heat map to display microplate-based image analysis data that may be used to display:
  • Figure 6 is a diagram illustrating an example of a surface map 600 generated using model data for representing the well bottom positions in a 3-D image.
  • the model data represents hypothetical values for well bottom surface positions and plate bottom positions for a model microplate.
  • the surface map 600 in Figure 6 shows the well bottom positions across the microplate, which illustrates the thickness variation along the bottom of the microplate.
  • the surface map 600 in Figure 6 was created using the Microsoft ExcelTM application, however, any suitable spreadsheet or imaging application that constructs images from data representing positions on a surface may also be used.

Description

    RELATED APPLICATION
  • This application claims priority to provisional application Ser. No. 61/522,089 titled Method for Presenting and Evaluation of Images of Microtiter Plate Properties filed on August 10, 2011,.
  • TECHNICAL FIELD
  • This invention relates to imaging systems used in examining physical, chemical or biological characteristics of samples in multiple well sample plates, and more particularly to systems and methods for acquiring images for evaluation and presentation of sample plate properties.
  • BACKGROUND
  • Materials and objects typically analyzed visually using an imaging system may be analyzed using image acquisition systems capable of handling multiple samples using some level of automation. Such image acquisition systems typically include a light source (such as for example, a white light or a fluorescent light source), a multiple sample plate, and an objective lens optically coupled to an image recording medium (or for example, a camera) to capture an image of the sample that may be substantially magnified. The multiple sample plate may rest or fit onto a sample-positioning mechanism that moves the plate in the optical path of the light source and objective lens. Examples of automated multiple-sample imaging systems include for example, microscopes and analyzers, such as analyzers that may use imaging for analysis of images captured in performing assays. One example of a type of automated imaging system is a high content screening system in which the high-resolution imaging and analysis of samples is automated. The analysis may entail recognizing and segmenting objects, arid extracting the features of interest for high content image analysis typically requires the use of high content visualization tools. One particular example of a high content screening system is the Leica HCS A made by Leica Microsystems.
  • Image acquisition systems may acquire images of samples placed in multiple sample plates implemented as microwell or microplates having a plurality of wells or cavities. It is noted that alternative terms, such as for example, "microwell," "microtiter," and "microplate," are used herein as reference to the plates. The samples are placed in wells arranged in a matrix within the microwell plate.
  • The Leica HCS A uses a software component called the High Content Screening LAS AF ("Leica Application Suite Advanced Fluorescence") MATRIX M3 automation software. The Leica HCS A system also includes a confocal microscope system. The LAS AF MATRIX M3 automation software includes an autofocus function that permits acquisition of images, which may be used to generate a focus map. A focus map is an automatically generated three-dimensional image showing true sample topology. The focus map is used for Z-positioning of the object of interest during the scan. According to the size and planarity of the samples, the optimal number and positions of the autofocus points is defined.
  • Another example of an imaging system is the CellReporter™, which is manufactured by Molecular Devices, LLC. The CellReporter™ includes high-quality, quantitative, data image analysis software to image, analyze, and report on individual cells. In capturing images for analysis, the CellReporterTM creates a 3D topological map, or surface map, of the plate-cell attachment surface. Prior to imaging the samples, each plate is scanned while images obtained thereafter are displaying the overall base of the plate. The surface map is used to represent the whole plate contours. In this particular application, the surface map is used to maintain focus across each plate during imaging to eliminate the need to focus at each well before taking an image.
  • The conventional microplate structure comprises a specimen plate portion with wells or cavities, and a bottom plate portion. The well bottom, or surface at the bottom of each well, is the plate-cell attachment surface of each well. The surface of the microplate opposite the well bottoms is the plate bottom. The distance between plate bottom and the well bottoms defines the thickness of the microplate. The thickness of the microplate may range between about 10 µm and about 1500 µm depending on the manufacturer.
  • The structure and specification of the microplates play an important role when used in the system to record images with high resolution (X, Y resolution to 100nm, Z resolution to 300nm) and high magnification (with 4x magnification and higher). Certain problems in obtaining in-focus images of the wells with samples therein are caused by out-of-focus light measurements. In particular, the problem in obtaining un-focused images in the well with samples is caused by incorrect determination of the focal plane. The incorrect assignment of the focal plane may be caused by variations in the microtiter plates' properties, for example flatness or the thickness of the plate.
  • The microplates may comprise between 6 and 1536 sample wells or cavities. Such plates are manufactured by a number of companies using a variety of materials. The flatness of the well bottoms and of the plate bottom varies substantially, not only from one manufacturer to another but even from different manufacturing lots. The thickness of the microplate typically varies substantially along the surface of the plate, which may result in incorrect assignment of the focal plane during images.
  • The importance of the microplate well flatness in obtaining high quality images has been noted by researchers and manufacturers. For example, researchers at GE Healthcare BioSciences Corporation evaluated the 96 and 384-well MartiPlate™ by performing the same assays on a number of microplates so that the biological evaluations on the different plates under test could be compared. Imaging background measurements were made on all plates by measuring fluorescence intensity background areas of selected images; average background gray levels were measured on a selected number of wells in each plate. Plate flatness data was obtained taking the .xdce file output from each image stack, which contained information on the autofocus position for each image. The information on the autofocus position for each image was used to produce a surface plot for each type of plate. The flatness of the plate well was assessed by taking measurements from several fields of view across three wells in each plate type. For each plate type, positions were abstracted from the .xdce file and plotted as a bar graph. The variation of the autofocus position within a given well reveals its flatness deviation.
  • This method may be used to label microplates with flatness characteristics, which enables a user to select microplates with desired flatness characteristics. However, a number of manufacturers are not conducting this type of experiment with their microplates, or characterizing their microplates in terms of flatness. Flatness characterization has not been made a part of their process for microplate evaluation, and accordingly, flatness characteristics are not part of the manufacturing specifications of the microplates. Moreover, the different manufacturing processes of the microplates make it difficult or impossible to provide the flatness characteristics for the plate bottom and for the bottom of the wells within the plate.
  • US 2008/0297774 A1 suggests a method and an apparatus for optically determining the thickness of a well bottom in a multi-well plate. Besides the light source and camera for detecting samples in the wells of the multi-well plate, a focal plane sensor is provided. The focal plane sensor comprises a broad band wavelength light source which transmits via a lens a focus light beam towards the bottom of the multi-well plate. The focal length of the lens is strongly dependent on the wavelength such that the focal points of different wavelengths are at different distance from the lens. The focal plane sensor detects the reflection of the light from the bottom side of the multi-well plate, wherein a high reflection is achieved at optical density junctions, e.g. at the junctions from air to the well material at the plate bottom and from the well bottom to the interior of the well. The reflected light is analyzed by a spectrometer, wherein in dependency of the wavelength an intensity increase is detected at the junctions. As the focal length for the different wavelengths is calibrated, from their intensity peaks the junctions can be detected and thus the thickness of the well bottom can be determined. Also determining a mean thickness value is provided.
  • Therefore, there is a need for automated measurement and visualization of microwell plate information on the flatness of the bottom of the plate and the bottom of the wells for obtaining high quality images in high content screening of samples.
  • There is also a need for improved and efficient evaluation and presentation of micro- plate properties, especially with respect to the variation of the thickness along the microplate bottom portion via automated measurements, and to display these properties for acquiring "in focus" microscopy images.
  • SUMMARY
  • The invention is defined in claims 1 and 9, respectively. Particular embodiments are set out in the dependent claims.
  • To address the foregoing problems, in whole or in part, and/or other problems that may have been observed by persons skilled in the art, the present disclosure provides methods, processes, systems, apparatus, instruments, and/or devices, as described by way of example in implementations set forth below.
  • According to one implementation, a method is provided for determining selected properties of a microplate comprising a plurality of sample wells. Each sample is open at a top portion of the microplate and includes a well surface receding into the microplate to a well bottom. The well surface is disposed on the top portion opposite a bottom portion of the microplate. An example method includes the step of moving the microplate to position an initial x-y position on the microplate in an optical path generated by a laser. An objective lens is positioned to be substantially centered in the optical path in a z-direction to an initial z-position. A focusing function is performed to determine a well surface z-position for a current x-y position based on an intensity indicative of a best focus of the well surface. The focusing function is performed again to determine a plate bottom z-position for the current x-y position based on an intensity indicative of a best focus of the plate bottom.
  • The microplate is then repeatedly moved to position a plurality of x-y positions with- in a selected area of the microplate in the optical path. The selected area encompasses the well bottom of one of the wells on the microplate. At each of the plurality of x-y positions, the well surface z-position and the plate bottom z-position corresponding to each of the plurality of x-y positions are determined by repeating the steps of positioning the objective lens, performing the focusing function to determine the well surface z-position, and performing the focusing function for the plate bottom z-position. The well surface z-positions and the plate bottom z-positions at the plurality of x-y positions are then used to generate data to determine the selected properties of the microplate. The method is further specified in claim 1.
  • According to another implementation, a system is provided for determining selected properties of a microplate comprising a plurality of sample wells. An example system comprises a focusing laser for generating a focusing light along an optical path. An objective lens is substantially centered at the optical path and mounted on an objective lens support. The objective lens support includes z motor components configured to move the objective lens along a z-axis. An x-y stage supports the microplate and includes x-y motor components configured to move the x-y stage to selected positions on an x-y plane. A light sensor is positioned in the optical path to receive light reflected from the microplate. The light sensor senses an intensity level of the light reflected from the microplate.
  • The system includes a controller to control the focusing laser, the z motor components, and the x-y motor components. The controller also detects the intensity level measured by the light sensor. The controller comprises a computer-implemented microplate evaluation system, which is configured for:
    1. a. performing a focusing function to determine a well surface z-position for a plurality of x-y positions in a selected area of the microplate encompassing one of the wells based on an intensity indicative of a best focus of the well surface;
    2. b. performing the focusing function to determine a plate bottom z-position for the plurality of x-y positions in the selected area based on an intensity indicative of a best focus of the plate bottom;
    3. c. moving the microplate to position the plurality of x-y positions within the selected area of the microplate in the optical path to perform the focusing functions; and
    4. d. using the well surface z-positions and the plate bottom z-positions at the plurality of x-y positions to generate microplate data to determine the selected properties of the microplate.
    The system is further specified in claim 9. BRIEF DESCRIPTION OF THE DRAWINGS
  • The invention can be better understood by referring to the following figures. The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention. In the figures, like reference numerals designate corresponding parts throughout the different views.
    • Figure 1 is a block diagram of an example of a system for determining microplate properties.
    • Figure 2 is a schematic diagram illustrating the structure of a conventional microplate.
    • Figure 3 is a flowchart illustrating operation of an example method for determining microplate properties.
    • Figure 4 is a diagram illustrating an example of a 3-D visualization of multiple points within a well in a microplate.
    • Figure 5 is a diagram illustrating the 3-D visualization of Figure 4 after it has been rotated to show a side view of the well.
    • Figure 6 is a diagram illustrating an example of a surface map generated using model data for representing the well bottom positions in a 3-D image.
    DETAILED DESCRIPTION
  • Disclosed herein are systems, methods, and apparatuses for evaluating and determining properties of microplates. Examples of the systems, methods, and apparatuses may be used in any suitably equipped imaging system such as a high content screening system, or a system that performs assays for diagnostic purposes using imaging to display the results. The description below refers to an imaging system in which advantageous use may be made of the examples described below. It is to be understood that the reference to microplates, imaging systems, or any other example structure is without limitation.
  • It is noted that the term "microplate" as used below refers to any multiple sample plate having a plurality of wells or cavities. The samples to be analyzed are placed in wells arranged in a matrix within the microplate. It is noted that alternative terms, such as for example, "microwell" or "microtiter" may also be used to refer to a microplate. It is to be understood that the terms "microwell," "microtiter," and "microplate" may be used interchangeably in the description below.
  • It is to be understood that the term "sample" refers to any material, object or composition that may be placed in a well of a microplate.
  • Figure 1 is a block diagram of an example of a system 100 for determining microplate properties. The system 100 includes an x-y stage 102, an objective lens 104, an objective lens support 106, a focusing laser 110, a light sensor (or, laser focus detector) 112, and a controller 116. The system 100 may be implemented in an imaging system that may further include an imaging camera 120 and a display 122. The system 100 may operate as a standalone system for evaluating microplates and storing microplate data corresponding to individual microplates for use in analyzing data collected for samples using the individual microplates. The system 100 may be integrated in an imaging system as components built-in as part of the imaging system. The system 100 may also be implemented as a feature provided in a kit for later integration with the imaging system, such as for example, after manufacture of the imaging system. Such a kit may include a storage medium containing a computer program that implements the functions of the system 100 and components that may not already be part of the imaging system. For example, the focusing laser 110 and the light sensor 112 may be components of such a kit.
  • The x-y stage 102 may be, or may be similar to, an x-y stage typically implemented on systems used to analyze samples. As shown in Figure 1, a microplate 130 is mounted on the x-y stage 102 for evaluation of selected properties. The x-y stage 102 includes x-y motor components (not shown in Figure 1) that moves the x-y stage 102 under control of the controller 116 to selected positions corresponding to points on the microplate 130. The x-y stage 102 moves along a plane corresponding to the substantially planar configuration of the microplate to x-y positions that may be identified using a calibration function. Such calibration functions that associate numerical x positions (along an x-axis of the plane of the motion of the x-y stage 102) and numerical y positions (along the y-axis) to physical points on the microplate are well known to those of ordinary skill in the art, and therefore require no further description.
  • The objective lens 104 may be any lens suitable for focusing the light from the focusing laser 110 on a point on an object on the x-y stage 102. The objective lens 104 may be positioned so that an optical path 115 created when the focusing laser 110 is emitting light passes substantially at the center of the objective lens 104. In terms of an x-y plane coplanar with the x-y plane defining the space in which the x-y stage 102 moves, the objective lens 104 has a substantially fixed location. The objective lens 104 is configured as shown in Figure 1 to move in a z-axis direction perpendicular to the x-y plane. The objective lens 104 may be mounted or fixed to the objective lens support 106, which may include z-motion components 108 to move the objective lens 104 in the z-axis direction. The objective lens support 106 and the z-motion components 108 may include a z-reference indicator for homing and calibration of the motion of the objective lens 104 along the z-axis.
  • The optical path 115 may be directed using optics, such as mirrors, lenses, or splitters. Figure 1 shows an optical path guide 114 that may be implemented with a mirror and a beam splitter to guide the optical path 115 of a focusing light 117 towards the objective lens 104, and to guide the optical path 115 of a reflected light 119 from the microplate 130 to the light sensor 112. The optical path guide 114 may be implemented using conventional optics components to guide the optical path 115 in conventional ways to ensure the focusing light 117 and the reflected light 119 are properly directed as needed in specific implementations.
  • The focusing laser 110 may be any laser suitable for generating a light that may be focused in a region near the x-y stage 102. The specific laser used for the focusing laser 110 may depend on the light sensor 112 used. The focusing laser 110 should also have a suitable power rating.
  • The light sensor 112 may be any suitable photosensitive device having a suitably linear electrical response to light intensity levels. Examples of photosensitive devices that may be used include, without limitation, a phototransistor, a photodiode, a photomultiplier tube, and a photoresistor. In an alternative implementation, the light sensor 112 may be implemented using an imaging device, which may include a camera, or components that capture images using for example a CCD ("charge-coupled device") or a CMOS camera. The light sensor 12 may be implemented in an example implementation using the imaging device used by the imaging system to analyze samples. For example, if the system 100 is implemented to operate using components of an imaging system, of which the system 100 may be an enhancement feature function, the system 100 may use the imaging camera of the imaging system to detect the laser focus.
  • The example system 100 shown in Figure 1 depicts an imaging camera 120 that is a component of the imaging system in which the system 100 may be implemented. The imaging camera 120 may be mounted so as to receive the reflected light 119 of the optical path 115 during evaluation of a microplate 130 if the imaging camera 120 is also being used for the functions described herein for the light sensor 112. Alternatively, the imaging camera 120 is used for the primary purpose of the imaging system such as in analyzing samples in the wells of the microplate 130. If the system 100 includes the light sensor 112 for detecting the laser focus during evaluation of the microplate 130, the optical path guide 114 may be switched, or inserted into the optical path 115, to direct the optical path 115 for evaluation functions.
  • The system 100 in Figure 1 includes the controller 116, which controls the z-motion components 108, the motion of the x-y stage 102, the focusing laser 110, the light sensor 112, and the imaging camera 120. The controller 116 may control the motion of the x-y stage 102, and the z-motion components 108 by controlling corresponding motors. The controller 116 may control the focusing laser 110 by turning the laser 110 on or off. The controller 116 may communicate with the light sensor 112 to receive intensity levels. The controller 116 may also interface with a display 122 as well as with other user interface devices to allow the operator to view displayed results, and to provide user input.
  • The controller 116 may be any suitable processing device or system that includes memory and other computing resources. In an example implementation, the controller 116 includes memory or a memory interface to access a computer-implemented microplate evaluation system. The microplate evaluation system may be implemented, for example, as software having functions, such as for example, a function for obtaining microplate data from which selected microplate properties may be determined. In an example implementation, the computer-implemented microplate evaluation system may perform methods such a method for determining microplate properties illustrated by the flowchart in Figure 3.
  • The computer-implemented microplate evaluation system analyzes microplates by constructing an image of the microplate 130 from points of best focus along the z-axis at points on an x-y plane that are within a specified area of the microplate 130. The specified area may be defined to include several, or all, of the wells of the microplate as shown at 132. The specified area may also be defined to include a single well as shown at 134 in Figure 1. The area of interest is mainly the bottom part of the wells on the microplate. The general structure of the microplate 130 may be considered in specifying the area containing the points of primary interest.
  • Figure 2 is a cross-sectional view of a portion of a microplate illustrating the structure of a conventional microplate 200. In general, the microplate 200 includes a top surface 202 and a bottom surface 210. The top surface 202 (indicated as being the "top" surface for convenience) provides openings to a first well 220, a second well 222, and a third well 224. Each well 220, 222, 224 includes a well bottom 204 and well sides 206. When the microplate 200 is in use, a sample 218 may be placed in the first well 220, for example. The sample 218 sits on the well bottom 204.
  • As noted above, the variations in the flatness of the well bottoms 204 may affect the focus during imaging of the sample 218 for analysis of the sample 218. In an example method for determining microplate properties, the image constructed of the microplate structure provides data that may be used to analyze the flatness of the well bottoms 204 and other physical properties of the microplate 200. The properties of the microplate 200 may be determined by evaluating the changes in thickness of the area formed by the well bottoms 204 between the surface of the well bottoms 204 and the bottom surface 210 of the microplate 200. For example, Figure 2 shows a first thickness, t1 , at one point of the surface of the well bottom 204 of the first well 220, and a second thickness, t2 , at another point of the same well 220. To determine the selected microplate properties at the first well 220, best focus points for the surface of the well bottom 204 and the bottom surface 210 may be identified at x-y points within an area identified by a line indicating a cross-section 134'-134" encompassing the surface of the well bottom 204 of the first well 220 in Figure 2. Similar areas may be specified to encompass the wells 222, 224.
  • Figure 3 is a flowchart illustrating operation of an example method 300 for determining microplate properties. The method 300 illustrated in Figure 3 may be used to evaluate a single well, or to obtain data that may be processed to identify the placements of multiple wells based on the geometry of the microplate 200 as shown in Figure 2. Once the method 300 is performed for an area that encompasses more than one well, the data may be analyzed to determine where each well is located in terms of the x-y plane of the microplate 200. The wells may be identified by determining where the thicknesses (such as t1 and t2 ) change dramatically and identifying the locations of dramatic changes as being the side walls 206 of the wells (see Figure 2).
  • The method 300 illustrated by the flowchart of Figure 3 is described as follows in the context of the system 100 shown in Figure 1. Component actors in the steps of the method 300 refer to the components in Figure 1 unless stated otherwise. With the focusing laser 110 in the 'on' state, the method 300 in Figure 3 is started by placing the microplate 130 on the x-y stage 102 as indicated at step 302. At step 304, the x-y stage 130 is then moved so that a selected point on the microplate 130 identified by an x, y position is in the optical path 115. With the x-y stage 102 at the selected x,y position, a search is performed along the z-axis to identify the z-position of the plate bottom surface 210 as shown at step 306. The search is performed by first moving the objective lens 106 along the z-axis at step 314. At each z-position, an intensity level is retrieved from the light sensor 112 at step 316. As intensity levels are collected, a focus score is determined to identify the z-position that provides the best focus of the plate bottom 210 at step 318. The z-position of best focus of the plate bottom 210 is identified as the plate bottom z-position at step 308 for the current x,y position.
  • At step 310, a search is started to determine the z-position of the surface of the well bottom 204 (in Figure 2). The search is performed by first moving the objective lens 106 along the z-axis at step 320. At each z-position, an intensity level is retrieved from the light sensor 112 at step 322. As intensity levels are collected, a focus score is determined to identify the z-position that provides the best focus of the surface of the well bottom at step 324. The z-position of best focus of the surface of the well bottom 204 is identified as the well bottom z-position at step 312.
  • At step 326, the data for any reports or for images of the well that is being collected is updated to indicate the z-position of the plate bottom and the z-position of the well bottom surface for the microplate at the given x,y position. At step 328, a check is made to determine if there are any other x,y positions in the specified area that require determination of the z-positions of the plate bottom and the well bottom. If there are, the next x,y position is determined and the x-y stage 102 is moved to the next x,y position at step 304. If the specified area has been covered, the report for the specified area is updated and stored at step 330. Step 330 may involve creating a database record of z-positions at the x-y positions in the specified area for an identified well. Depending on the area (encompassing a single well, or multiple wells), the database record may contain data for multiple wells. The method may be complete at step 332 for a single well, then repeated for a next well in the microplate. Or, the data may be processed to identify individual wells based on the general geometry of the microplate.
  • It is noted that in some imaging systems, such as, for example, confocal imaging systems, autofocusing functions may be integrated into the operation of the imaging system. The autofocusing functions, or components of the autofocusing functions, may be used to perform, for example, one or more of steps 306, 314, 316, 318, 308, 310, 320, 322, 324, and 312 in Figure 3.
  • The method 300 illustrated by the flowchart in Figure 3 performs steps of recording sensor information ( steps 316 and 322, for example), and the steps are described as determining an intensity level. In the examples described herein, the intensity level indicates the amount of light reflected from the surface of the plate at either the well bottom surface or the plate bottom when the objective lens is at the appropriate distance from the plate. The best focus position is also deemed to be the point at which the intensity level measured is at a maximum out of a group of measurements. It is to be understood by those of ordinary skill in the art that the term 'intensity level' in the context of the steps performed in the method 300 illustrated in Figure 3 may be a value read off the sensor, or a value that has been processed to either transform the measured value to an actual intensity value, or to scale or normalize the value to simplify the processing. The term 'intensity level' may refer, for purposes of this description, to any parameter indicative of the amount of light reflected from the plate.
  • The microplate data recorded using the method 300 in Figure 3 is used to automatically measure and display microwell plate properties. The microplate data may be presented and displayed in a 3D format. The 3-D representation of the microplate data may be especially useful in evaluating the variation of the thickness of the wells and the plate bottom. The 3-D representation may provide a visual display of the contours of the microplate as a whole in addition to the contours of the individual well bottoms.
  • The microplate properties may be automatically measured and displayed for multiple locations within individual wells. The measured information may provide the user with feedback on the currently imaged microplate during data acquisition in a real-time format. For example, the imaging system may be performing an analysis of samples. The system for determining microplate properties may be used in real-time to retrieve microplate data, and provide the user with information on the microplate properties in real-time during imaging data acquisition. Alternatively, the microplate data for the microplate being used in imaging data acquisition may be retrieved from memory that stores microplate data for previously screened microplates. The system may include user interface tools and functions that permit manipulation of the displayed plate bottom and well bottom images. For example, such manipulation may allow the user to rotate, or zoom-in and zoom-out or to perform other operations. The images displaying the data for the plate bottom and well bottom may be presented as 3-D or 2-D color-coded diagram.
  • The microplate data that characterizes the plate bottom and well bottom may be collected and stored to conduct automated analysis and to produce reports requested by users. The reports may be used, for example, in a certification system when the manufacturer of an acquisition device certifies that microplate models are compatible with the device, or when certification is required to validate acquisition device accuracy when using a known microplate. Multiple microplates of the same model from the same manufacturer may be analyzed to determine the extent of the variation of the properties of microplates within a manufacturing lot or between different manufacturing lots. The measured data from a plurality of microplates may also be used to automatically adjust acquisition settings during analysis of samples.
  • The results obtained from comparing the properties of multiple microplates may provide information relating to whether particular microplates "failed" or "passed" some desired certification standard. The results may also provide information regarding the overall quality of the microplates. The microplate properties for multiple microplates may also provide a historical record that may be used to adjust acquisition settings for performing imaging of samples. The historical record may be stored as images acquired from various microplates. The analysis of the numerical data may include for example, selecting minimum, maximum and average thickness of each well. The information relating to the numerical values obtained by analysis of the numerical data of the microplate properties may be used to set positions and ranges to optimize the focus speed for the acquisition imaging systems used for analyzing samples.
  • The information relating to the optimization of the acquisition settings may include, for example, indications that the setting ranges are too narrow, which may further indicate that the optimization of the acquisition settings is not allowing the next attempt to obtain focus. In addition, the information may indicate if the setting ranges are too broad, which may extend the time to obtain each focus. The provided information may also indicate whether or not the plate is damaged or, for some other reason does not match the required criteria. The 3-D visual display of the properties, and more particularly, the thickness of the plate bottom and well bottom, allows for inspection and comparison of the results of such inspection with requested microplate's property.
  • After the acquisition settings are optimized, a procedure may be used to obtain feedback on setting quality during analysis of a sample. Feedback procedures may be invoked and the data may be analyzed using the user interface.
  • The microplate properties data may be stored for multiple microplates to monitor quality and equipment compatibility control. For example, when the thickness property of the microplate is accurately defined and displayed, the process of selecting the microplates for testing on the instrumentation becomes more transparent and efficient for user.
  • Figure 4 is a diagram illustrating an example of a 3-D visualization 400 a section of a microplate. The 3-D visualization 400 in Figure 4 depicts multiple top regions 402 indicating a plate top surface and corresponding bottom regions 404 indicating a plate bottom surface. It is noted that the 3-D visualization 400 is displayed with the z-axis to a different scale than the x and y axes in order to better visualize the well and plate contours. The top and bottom regions 402 and 404 are shown positioned over corresponding specified areas of measurement 406. The specified areas of measurement 406 contain dots 408 indicating x-y positions for which a plate bottom z-position and a well surface z-position have been determined. Rulers for x and y axis positions may be displayed to provide the user with indication of location on the plate. The data for the 3-D visualization 400 in Figure 4 may be collected in real-time as the 3-D visualization 400 is displayed. When generated in real-time, the 3-D visualization 400 may indicate the progress of the imaging by depicting the filling of the specified areas 406 with dots 408. The 3-D visualization 400 may be displayed as data is collected during a screening of the microplate performed to evaluate the microplate. The 3-D visualization 400 may also be displayed during image acquisition for analyzing samples.
  • Figure 5 is a diagram illustrating the 3-D visualization of Figure 4 after it has been rotated to show a side view 500 of the microplate.
  • A 3-D visualization such as the 3-D visualization 400 in Figure 4 and the side view 500 in Figure 5 may include status information indicating the status of each focus attempt used to create the 3-D visualization. The status information may be conveyed using combinations of colors, shapes or other unique visual characteristics that may be configured to indicate the success or failure of each focus attempt, as well as of the type of focus performed.
  • The microplate data may be used to generate visualization of the data in other forms. For example, the microplate data may be used to plot a heat map to display microplate-based image analysis data that may be used to display:
    1. 1. Z-axis variation within a well,
    2. 2. the moving from site-to-site,
    3. 3. the variation among adjacent wells moving from well-to-well, or
    4. 4. the variations in bottom thickness.
  • Figure 6 is a diagram illustrating an example of a surface map 600 generated using model data for representing the well bottom positions in a 3-D image. The model data represents hypothetical values for well bottom surface positions and plate bottom positions for a model microplate. The surface map 600 in Figure 6 shows the well bottom positions across the microplate, which illustrates the thickness variation along the bottom of the microplate. The surface map 600 in Figure 6 was created using the Microsoft Excel™ application, however, any suitable spreadsheet or imaging application that constructs images from data representing positions on a surface may also be used.
  • It will be understood that various aspects or details of the invention may be changed without departing from the scope of the invention. Furthermore, the foregoing description is for the purpose of illustration only, and not for the purpose of limitation-the invention being defined by the claims.

Claims (14)

  1. A method for determining selected properties of a microplate (130, 200) comprising a plurality of sample wells (220, 222, 224), each sample well being open at a top portion of the microplate and including a well surface (202, 204), the well surface being disposed on the top portion opposite a plate bottom (210) of the microplate, the method comprising:
    moving (304) the microplate (130, 200) to position an initial x-y position on the microplate in an optical path (115) generated by a laser (110);
    positioning an objective lens (104) substantially centered in the optical path (115) in a z-direction to an initial z-position;
    performing (314-308) a focusing function to determine a well surface (202, 204) z-position for a current x-y position based on an intensity indicative of a best focus of the well surface;
    performing (320-312) the focusing function to determine a plate bottom (210) z-position for the current x-y position based on an intensity indicative of a best focus of the plate bottom (210);
    moving (328-304) the microplate (130, 200) to position a plurality of x-y positions within a selected area of the microplate in the optical path (115) where the selected area encompasses a well bottom (204) of one of the wells (220, 222, 224) on the microplate (130, 200);
    at each of the plurality of x-y positions, determining the well surface (202, 204) z-position and the plate bottom (210) z-position corresponding to each of the plurality of x-y positions by repeating the steps of positioning the objective lens (104), performing the focusing function to determine the well surface z-position, and performing the focusing function for the plate bottom z-position; and
    using the well surface (202, 204) z-positions and the plate bottom (210) z-positions at the plurality of x-y positions to:
    generate an image in a 3-D format to depict the well bottom (204) and plate bottom (210); and
    display the image on a display to illustrate a flatness of the well bottom (204), the flatness of the well bottom (204) being one of the selected properties of the microplate (130, 200).
  2. The method of claim 1, where the step of using the well surface z-positions and the plate bottom z-positions at the plurality of x-y positions includes determining a plate thickness between the well bottom (204) and the plate bottom (210) in the area encompassing a selected well, the plate thickness being one of the selected properties of the microplate (130, 200).
  3. The method of claim 1 or 2, further comprising:
    moving the microplate (130, 200) to position an initial x-y position in a next area encompassing another well (220, 222, 224) on the microplate in the optical path (115);
    determining the well surface z-position and the plate bottom z-position for each x-y position in the next area; and
    repeating the steps of moving the microplate (130, 200) to an area encompassing another well (220, 222, 224) and of determining the well surface z-position and the plate bottom z-position for each x-y position in the area until data for determining the selected properties of the microplate has been generated for a selected plurality of wells on the microplate.
  4. The method of claim 3, where the step of using the well surface z-positions and the plate bottom z-positions at the plurality of x-y positions includes:
    generating a surface map or a heat map comprising well images representing well bottom positions for the wells (220, 222, 224) within the specific areas for which well surface (202, 204) and plate bottom (210) focusing was performed; and
    displaying the surface map or the heat map on a display to display well bottom positions on the surface map.
  5. The method of claim 3, where the step of using the well surface z-positions and the plate bottom z-positions at the plurality of x-y positions includes:
    generating a 3-D image comprising well images representing well bottom positions for the wells within the specific areas for which well surface and plate bottom focusing was performed;
    displaying the 3-D image on a display; and
    receiving user interface commands to either zoom in, zoom out, or rotate the 3-D image about an axis.
  6. The method of any of the previous claims, further comprising indicating a success or failure of the steps of performing the focus functions.
  7. The method of any of the previous claims, where the step of performing the focus function to determine the well surface z-position for a current x-y position includes:
    measuring an intensity of light reflected from the microplate (130, 200) at a sensor (112, 120) in the optical path (115) as the objective lens (104) is moved to a plurality of z-positions; and
    determining a best focus z-position corresponding to the intensity indicative of a best focus of the well surface (202, 204).
  8. The method of any of the previous claims, where the step of performing the focus function to determine the plate bottom z-position for a current x-y position includes:
    measuring an intensity of light reflected from the microplate (130, 200) at a sensor (112, 120) in the optical path as the objective lens (104) is moved to a plurality of z-positions; and
    determining a best focus z-position corresponding to the intensity indicating a best focus of the plate bottom (210).
  9. A system (100) for determining selected properties of a microplate (130, 200) comprising a plurality of sample wells (220, 222, 224), each sample well being open at a top portion of the microplate and including a well surface (202, 204), the well surface being disposed on the top portion opposite a plate bottom (210) of the microplate, the system comprising:
    a focusing laser (110) for generating a focusing light (117) along an optical path (115);
    an objective lens (104) substantially centered at the optical path (115) and mounted on an objective lens support (106) that includes z motor components (108) configured to move the objective lens along a z-axis;
    an x-y stage (102) configured to support the microplate (130, 200), the x-y stage comprising x-y motor components configured to move the x-y stage to selected positions on an x-y plane;
    a light sensor (112, 120) positioned in the optical path (115) to receive light reflected from the microplate (130, 200) and to sense an intensity level of the light reflected from the microplate; and
    a controller (116) configured to control the focusing laser (110), the z motor components (108), and the x-y motor components, and to detect the intensity level measured by the light sensor (112, 120), the controller further comprising a computer-implemented microplate evaluation system configured for:
    performing a focusing function to determine a well surface z-position for a plurality of x-y positions in a selected area of the microplate (130, 200) encompassing one of the wells (220, 222, 224) based on an intensity indicative of a best focus of the well surface (202, 204);
    performing the focusing function to determine a plate bottom z-position for the plurality of x-y positions in the selected area based on an intensity indicative of a best focus of the plate bottom (210);
    moving the microplate (130, 200) to position the plurality of x-y positions within the selected area of the microplate in the optical path (115) to perform the focusing functions; and
    using the well surface z-positions and the plate bottom z-positions at the plurality of x-y positions to:
    generate an image in a 3-D format to depict the well bottom (204) and plate bottom (210); and
    display the image on a display to illustrate a flatness of the well bottom (204), the flatness of the well bottom (204) being one of the selected properties of the microplate (130, 200).
  10. The system of claim 9, where the light sensor (112) is a photosensitive device that generates an electrical signal having a parameter that changes in response to the intensity of the light received by the photosensitive device, wherein in particular the photosensitive device is any of a phototransistor, a photodiode, a photomultiplier tube, and a photoresistor.
  11. The system of claim 9, where the light sensor (120) is an imaging device.
  12. The system of claim 9, 10 or 11,
    where the laser (110), the light sensor (112, 120), x-y stage (102), objective lens (104), and controller (116) are components integrated into an imaging system; or
    where the controller (116), the x-y stage (102), and the objective lens (104) are components of an imaging system, and the computer-implemented microplate evaluation system is configured to be added after manufacture of the imaging system.
  13. The system of any of the claims 9 to 12, where the controller (116) is implemented in a personal computer communicating with the focusing laser (110), the z motor components (108), the x-y motor components, and the light sensor (112, 120) via a communication interface, wherein in particular the personal computer comprises: a display; and a database configured to store microplate profiles each comprising microplate data corresponding to a specific microplate (130, 200).
  14. The system of any of claims 9 to 13, further comprising:
    a communications interface configured to operatively place the controller (116) in communication with a computing environment configured to process the microplate data; and/or
    a display (122) to display images generated using the microplate data, where the images are formatted to depict selected properties of the microplate (130, 200).
EP12821372.5A 2011-08-10 2012-08-09 Method for presenting and evaluation of images of microtiter plate properties Active EP2742338B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201161522089P 2011-08-10 2011-08-10
PCT/US2012/050067 WO2013023017A1 (en) 2011-08-10 2012-08-09 Method for presenting and evaluation of images of microtiter plate properties

Publications (3)

Publication Number Publication Date
EP2742338A1 EP2742338A1 (en) 2014-06-18
EP2742338A4 EP2742338A4 (en) 2015-06-03
EP2742338B1 true EP2742338B1 (en) 2019-04-24

Family

ID=47668942

Family Applications (1)

Application Number Title Priority Date Filing Date
EP12821372.5A Active EP2742338B1 (en) 2011-08-10 2012-08-09 Method for presenting and evaluation of images of microtiter plate properties

Country Status (4)

Country Link
US (1) US9360659B2 (en)
EP (1) EP2742338B1 (en)
JP (1) JP6032767B2 (en)
WO (1) WO2013023017A1 (en)

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6242202B2 (en) * 2012-12-13 2017-12-06 オリンパス株式会社 Optical microscope system and method for observing cells using optical microscope
US10274715B2 (en) * 2014-08-06 2019-04-30 Cellomics, Inc. Image-based laser autofocus system
GB2534402A (en) * 2015-01-22 2016-07-27 Idea Biomedical Ltd Auto-focussing method and device
EP3101107B1 (en) 2015-06-05 2019-04-24 The Procter and Gamble Company Compacted liquid laundry detergent composition
EP3101100B1 (en) 2015-06-05 2018-02-07 The Procter and Gamble Company Compacted liquid laundry detergent composition
EP3101102B2 (en) 2015-06-05 2023-12-13 The Procter & Gamble Company Compacted liquid laundry detergent composition
JP6667411B2 (en) * 2016-09-30 2020-03-18 富士フイルム株式会社 Observation device and method, and observation device control program
JP6812562B2 (en) * 2017-08-30 2021-01-13 富士フイルム株式会社 Observation device and method and observation device control program
EP3690513B1 (en) * 2017-09-27 2023-06-07 FUJIFILM Corporation Observation device, observation method, and observation program
JP6785984B2 (en) * 2017-09-29 2020-11-18 富士フイルム株式会社 Observation device, observation method and observation program
JP6812944B2 (en) * 2017-10-18 2021-01-13 株式会社島津製作所 Information management device for mass spectrometer
WO2019244275A1 (en) * 2018-06-20 2019-12-26 株式会社日立ハイテクノロジーズ Observation device
US11523046B2 (en) * 2019-06-03 2022-12-06 Molecular Devices, Llc System and method to correct for variation of in-focus plane across a field of view of a microscope objective
JP2024507127A (en) * 2021-02-15 2024-02-16 アムジエン・インコーポレーテツド System and method for automatic in-line plunger depth measurement

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5757502A (en) * 1996-10-02 1998-05-26 Vlsi Technology, Inc. Method and a system for film thickness sample assisted surface profilometry
GB2337815A (en) * 1998-05-28 1999-12-01 Visionix Ltd Thickness meter for thin transparent objects
US6130745A (en) * 1999-01-07 2000-10-10 Biometric Imaging, Inc. Optical autofocus for use with microtiter plates
DE19916749B4 (en) * 1999-04-14 2004-02-12 Carl Zeiss Jena Gmbh Procedure for examining samples
GB0317679D0 (en) * 2003-07-29 2003-09-03 Amersham Biosciences Uk Ltd Analysing biological entities
US20060166305A1 (en) * 2005-01-27 2006-07-27 Genetix Limited Animal cell confluence detection method and apparatus
JPWO2007010803A1 (en) * 2005-07-15 2009-01-29 オリンパス株式会社 Light measuring device
JP2009529671A (en) * 2006-03-10 2009-08-20 コーニング インコーポレイテッド Reference microplate and method of manufacturing and using reference microplate
EP2023127B1 (en) * 2006-05-31 2017-12-20 Olympus Corporation Biological specimen imaging method and biological specimen imaging apparatus
US7646482B2 (en) 2007-05-31 2010-01-12 Genetix Limited Methods and apparatus for optical analysis of samples in biological sample containers
JP4985980B2 (en) * 2008-02-28 2012-07-25 横河電機株式会社 Well plate and fluorescence imaging system using it

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
JP2014521978A (en) 2014-08-28
EP2742338A4 (en) 2015-06-03
WO2013023017A1 (en) 2013-02-14
EP2742338A1 (en) 2014-06-18
JP6032767B2 (en) 2016-11-30
US20140210981A1 (en) 2014-07-31
US9360659B2 (en) 2016-06-07

Similar Documents

Publication Publication Date Title
EP2742338B1 (en) Method for presenting and evaluation of images of microtiter plate properties
US9025839B2 (en) System and method to determine slide quality of a digitized microscope slide
EP1264205B1 (en) A microscope suitable for high-throughput screening having an autofocusing apparatus
US8878923B2 (en) System and method for enhanced predictive autofocusing
US7646482B2 (en) Methods and apparatus for optical analysis of samples in biological sample containers
JP2017194700A (en) Fully automatic rapid microscope slide scanner
US20210215923A1 (en) Microscope system
US7535556B2 (en) Methods and apparatus for optical analysis of samples in biological sample containers
CN105509641A (en) Method for programming three-dimensional workpiece scan path for metrology system
US8937654B2 (en) Machine vision inspection system comprising two cameras having a rotational offset
JP2012008415A (en) Microscope system
US20140232844A1 (en) Method and apparatus for defining a z-range in a sample, in which a z-stack of the sample is to be recorded by means of a microscope
KR20140103000A (en) Fast and quantitative raman analysis method and apparatus thereof for large-area multiple bio-targets
JP3137634U (en) Macro micro navigation system
KR101535494B1 (en) Method and apparatus for high speed raman analysis using unit area patching
US20240061229A1 (en) Microscopy system and method using an immersion liquid
EP4235568A1 (en) Analysis method and analysis apparatus
CA3218518A1 (en) Microscopy system and method using an immersion liquid
WO2023107734A1 (en) Surface sensing in automated sample analysis
JP2006158207A (en) Method for detecting microorganism and device for metering microorganism
De Mey et al. ORRECTED

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20140310

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAX Request for extension of the european patent (deleted)
RA4 Supplementary search report drawn up and despatched (corrected)

Effective date: 20150504

RIC1 Information provided on ipc code assigned before grant

Ipc: G01B 11/24 20060101ALI20150424BHEP

Ipc: G01N 35/00 20060101ALI20150424BHEP

Ipc: G01B 11/06 20060101ALI20150424BHEP

Ipc: G01N 21/13 20060101AFI20150424BHEP

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: MOLECULAR DEVICES, LLC

RIC1 Information provided on ipc code assigned before grant

Ipc: G01B 11/24 20060101ALI20180924BHEP

Ipc: G01N 35/00 20060101ALI20180924BHEP

Ipc: G01N 21/25 20060101ALI20180924BHEP

Ipc: G01B 11/06 20060101ALI20180924BHEP

Ipc: G02B 21/00 20060101ALI20180924BHEP

Ipc: G01N 21/13 20060101AFI20180924BHEP

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20181113

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1124710

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190515

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602012059416

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20190424

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190424

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190424

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190424

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190424

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190424

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190424

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190824

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190424

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190724

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190725

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190424

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190424

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190424

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190724

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1124710

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190424

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190824

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602012059416

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190424

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190424

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190424

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190424

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190424

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190424

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190424

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190424

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190424

26N No opposition filed

Effective date: 20200127

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190831

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190424

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190831

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190424

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190809

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20190831

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190809

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190831

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190424

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190424

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20120809

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190424

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230530

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20230620

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20230615

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20230613

Year of fee payment: 12