AU2013273789A1 - Thickness estimation for Microscopy - Google Patents

Abstract

- 30 THICKNESS ESTIMATION FOR MICROSCOPY A method of determining at least one thickness parameter of a specimen from a stack of images of the specimen obtained from a microscope at a series of depths is disclosed. The method selects (340), within a region of interest, sets (850) of patches at a plurality of corresponding transverse locations in each image within the stack of images. A depth value is determined (350) at each transverse location from a set of contrast metrics computed for the set of patches at the transverse location, and the method determines (380) at least one thickness parameter for the specimen at the region of interest from at least a distribution (700) of the determined depth values. 8130727_1 P092066_speci lodge Data storage I Computer Fig. 1

Description

- 1 THICKNESS ESTIMATION FOR MICROSCOPY TECHNICAL FIELD [0001] The current invention relates to thickness estimation for specimens viewed using an imaging device, in particular for the case of a translucent specimen viewed using a microscope. Thickness estimation can improve the efficiency and accuracy of the capture of images of a specimen and subsequent post-processing. BACKGROUND [0002] Virtual microscopy is a technology that gives physicians the ability to navigate and observe a biological specimen at different simulated magnifications and through different three dimensional (3D) views as though they were controlling a microscope. It can be achieved using a display device, such as a computer monitor or tablet, with access to a database of microscope images of the specimen. There are a number of advantages of virtual microscopy over traditional microscopy. With virtual microscopy, the specimen is not required at the time of viewing, thereby facilitating archiving, telemedicine and education. Virtual microscopy can also enable the processing of the specimen images to change the depth of field and to reveal pathological features that would be otherwise difficult to observe by eye, for example as part of a computer aided diagnosis system. [0003] Capture of images for virtual microscopy is generally performed using a high throughput slide scanner. The specimen is loaded mechanically onto a stage of the microscope and moved under the microscope objective as images of different parts of the specimen are captured using a sensor. Depth and thickness information for the specimen being imaged are required in order to perform an efficient capture. [0004] Adjacent images have an overlap region so that the multiple images of the same specimen can be combined into a 3D volume in a computer system attached to the microscope. Mosaicing and other software algorithms are used to register both the neighbouring images at the same depth and at different depths so that there are no defects between adjoining images to give a seamless 3D view. Virtual Microscopy is different from other image mosaicking tasks in a number of important ways. Firstly, the specimen is typically moved by the stage under the 8130727_1 P092066_speci lodge -2 optics, rather than the optics being moved to capture different parts of the subject as would take place in a panorama. The stage movement can be controlled very accurately and the specimen may be fixed in a substrate on the stage. [0005] The microscope is used in a controlled environment, for example mounted on a vibration isolation platform in a laboratory with a calibrated illumination set up, so that the optical tolerances of the system (alignment and orientation of optical components and the stage) are very tight. Therefore, the coarse alignment of the tiles of captured images for mosaicking can be fairly accurate, the lighting even, and the transform between the tiles well represented by a rigid transform. On the other hand, the scale of certain important features of a specimen can be of the order of several pixels and the features can be densely arranged over the captured tile images. This means that the required stitching accuracy for virtual microscopy is very high. Additionally, given that the microscope can be loaded automatically and operated in batch mode, the processing throughput requirements are also high. [0006] An accurate thickness estimate is also useful after the capture of images of a specimen in a microscope. For example, in the field of stereology, a thickness estimate may be used to form unbiased estimates of geometric properties of the specimen. In this case the specimen would typically be a biological sample such as a histology slide consisting of a tissue fixed in a substrate and stained to highlight specific features. [0007] Manual depth and thickness estimation for a specimen may be performed. The operator of the microscope can make a judgment of the position of top and bottom surfaces while modifying the position of the stage along the optical axis. Being manual, this estimation method is not suitable for use in an automated slide scanner. Additionally, the accuracy will depend on the skill of the manual operator. [0008] Automated depth estimation in a microscope may be performed based on a focus function which measures the local contrast or sharpness in the field of view. The axial location at which the focus function is greatest defines the position of best focus, which may be considered to represent the depth of a thin specimen. Many focus functions have been described in the literature, including the normalised variance, Brenner and Tenenbaum. Specimen thickness estimation may also be performed using a focus function. In this case, a heuristic rule may be 8130727_1 P092066_speci lodge -3 used to estimate where in the focus curve the specimen may be considered to end, for example at the point of inflexion or at a particular percentage of the maximum height of the curve. These estmates of the end of the specimen may then be used to define the top or bottom surface of the specimen. The bias and accuracy of the method is dependent, among other things, on the selection of the heuristic rule. [0009] Other depth estimation methods are known based on active illumination and aperture masks; however, these techniques require additional components that are not required in a standard microscope set up, and may not generate thickness information. [0010] A need therefore exists for efficient and accurate methods of measuring the thickness of a specimen in a microscope. SUMMARY [0011] According to one aspect of the present disclosure, there is provided a method of determining at least one thickness parameter of a specimen from a stack of images of the specimen obtained from a microscope at a series of depths, the method comprising: selecting, within a region of interest, sets of patches at a plurality of corresponding transverse locations in each image within the stack of images; determining a depth value at each transverse location from a set of contrast metrics computed for the set of patches at the transverse location; and determining at least one thickness parameter for the specimen at the region of interest from at least a distribution of the determined depth values. [0012] The method may further comprise forming the distribution from depth values determined at different transverse locations across the specimen. Desirably the forming of the distribution comprises determining in the distribution outlier depth values that exceed a predetermined corresponding thickness parameter and removing the outlier depth values from the distribution before the determining of the at least one revised thickness parameter. Preferably the predetermined corresponding thickness parameter is determined at an image level larger than the patches. 8130727_1 P092066_speci lodge -4 [0013] According to another aspect there is disclosed a method of determining at least one thickness parameter of a specimen from a stack of images of the specimen captured at a series of depths, the method comprising: selecting, within a region of interest, a patch at a corresponding location in each image within the stack of images to form a set of patches; determining at least one initial thickness parameter of the specimen at the selected region of interest based on a set of contrast metrics computed for the set of patches; selecting sets of sub-patches at a plurality of corresponding transverse locations within each patch in the set of patches; determining a depth value at each transverse location from a set of contrast metrics computed for the sub-patches at the transverse location; and determining at least one revised thickness parameter for the specimen at the region of interest from a distribution of the determined depth values and the initial thickness parameter. [0014] Preferbaly the thickness parameter is selected from the group consisting of a thickness of the specimen, a location of a top surface of the specimen, and a location of a bottom surface of the specimen. [0015] Advantageously the method may further comprise determining outlier depth values that exceed a corresponding thickness parameter and removing the outlier depth values from the distribution before the determining of the at least one revised thickness parameter. Preferably the corresponding thickness parameter comprises the initial thickness parameter. Also the corresponding thickness parameter may comprise a value of an upper or lower percentile of determined depth values. The method may further comprise selecting the patch within the region of interest for each image from an aligned stack of patches within the region. Desirably the determining of the at least one initial thickness parameter comprises forming estimates of a first and second thickness parameters at the current region of interest based on the set of contrast metrics computed for the set of patches. [0016] Preferably the determining of the at least one revised thickness parameter comprises analysing the distribution of depth values for the region to obtain current estimates for a top surface location and a bottom surface location of the specimen, the analysing comprising at least one of: 8130727_1 P092066_speci lodge -5 (i) selecting a top surface location from a smallest one of the depth values; (ii) selecting a bottom surface location from a largest one of the depth values; and (iii) evaluating a percentile depth value of the distribution as one of the top or bottom surface locations. [0017] Here, the determining of the at least one revised thickness parameter may comprise forming a correction factor using at least the initial thickness parameter and the current estimates and modifying the current estimates using the correction factor. Advantageously the initial thickness parameter comprises an initial thickness estimate, and a current thickness estimate comprises a difference of the top surface location and the bottom surface location, and the correction factor is an average of the difference between the initial thickness estimate and the current thickness estimate across the regions of the specimen, and the revised thickness parameter is a revised thickness formed by subtracting the correction factor from the initial thickness estimate. [0018] The method may further comprise setting a size of the sub-patches according to a feature size within the specimen. The specimen most typically comprises a stained histology slide. [0019] Other aspects are also disclosed. BRIEF DESCRIPTION OF THE DRAWINGS [0020] At least one embodiment of the present invention will now be described with reference to the following drawings, in which: [0021] Fig. 1 shows a high-level system diagram for a general microscope capture system; [0022] Fig. 2 is a schematic flow diagram that illustrates a general overview of a computer implemented method that can be used to estimate the thickness of a specimen according to the present disclosure; [0023] Fig. 3 is a schematic flow diagram that illustrates the generation of thickness parameters in the method of Fig. 2; 8130727_1 P092066_speci lodge -6 [0024] Fig. 4 is a schematic flow diagram that illustrates a prior art method of estimating the depth of best focus and thickness parameters of a specimen based on a stack of image patches or sub-patches; [0025] Fig. 5 illustrates contrast metric data for a stack of aligned patches or sub-patches through a specimen that highlights the estimated depth of best focus and of top and bottom surfaces; [0026] Fig. 6 is a schematic flow diagram that illustrates a method of analysing a set of best focus depths corresponding to a region to estimate the thickness of a specimen; [0027] Fig. 7A illustrates a set of best focus depths corresponding to a region that may be used to estimate the thickness of a specimen; [0028] Fig. 7B illustrates the distribution of thickness estimates for a specimen of fixed thickness based on different techniques; [0029] Figs. 8A to 8C illustrate the estimation of thickness of a specimen; and [0030] Figs. 9A and 9B collectively form a schematic block diagram representation of the computer of Fig. 1 within which the various methods described may be implemented. DETAILED DESCRIPTION INCLUDING BEST MODE Context [0031] Fig. 1 shows a high-level system diagram for a general microscope capture system 100. A specimen 102 is physically positioned on a movable stage 110 that is under the lens and within the field of view of a microscope 101. The stage of the microscope 102 may move as multiple images of the calibration target 102 are captured by a camera 103 mounted to the microscope 101. The camera 103 takes one or more images 104 at each location of the stage 110. Multiple images can be taken with different optical settings or using different types of illumination. The captured images 104 are passed to a computer system 105 which can either start processing the images immediately or store them in temporary storage 106 for later processing. As part of its processing the computer 105 generates a thickness estimate for one or more regions of interest of 8130727_1 P092066_speci lodge -7 the specimen 102. The thickness estimate may be used to determine the parameters of subsequent image captures by the microscope 101. As shown, movement of the stage 110 may be controlled by computer 105 via a connection 111. [0032] The transverse optical resolution of the microscope 101 may be estimated based on the optical configuration of the microscope 101 and is related to the point spread function of the microscope. A standard approximation to this resolution is given by: 0.61 2 ( Dr=NA where NA is the numerical aperture, and A is the wavelength of light. For air immersion, an NA of 0.7 and a wavelength of 500 nm, the estimated resolution is 0.4 micron. [0033] The depth of field of the microscope may also be estimated based on the optical configuration of the microscope. A standard approximation to this depth of field is given by the following relationship: Dz = An (2) where again NA is the numerical aperture and A is the wavelength of light in the microscope. The parameter n is the refractive index in the medium (n=1.0 for air immersion or may be higher if the lens is immersed, for example in oil). For air immersion, an NA of 0.7 and a wavelength of 500 nm, the estimated depth of field is 1 micron. The depth of field is an important parameter when considering depth or thickness estimation from a stack of images. The depth of field can be used to select a suitable axial separation of captured images in a stack of images to be used for depth or thickness estimation. It is inefficient to use an axial separation that is much smaller than the depth of field. Axially dependent features around this scale or larger may be observable as differences between adjacent capture layers images. [0034] The specimen being observed may be a biological specimen such as a histology slide consisting of a tissue fixed in a substrate and stained to highlight specific features. Such a slide may include a variety of biological features on a wide range of scales. The most common scale of features in a given slide may depend on the specific tissue sample and stain used to create the histology slide. These features may be the most useful for depth or thickness estimation using focus finding techniques. Overview 8130727_1 P092066_speci lodge -8 [0035] Figs. 9A and 9B depict a general-purpose computer system 900, upon which the various arrangements described can be practiced. [0036] As seen in Fig. 9A, the computer system 900 includes: the computer module 105 of Fig. 1, input devices such as a keyboard 902, a mouse pointer device 903, a scanner 926, the camera 103, and a microphone 980; and output devices including the microscope 101 (for stage control), the display device 107 and loudspeakers 917. An external Modulator-Demodulator (Modem) transceiver device 916 may be used by the computer module 105 for communicating to and from a communications network 920 via a connection 921. The communications network 920 may be a wide-area network (WAN), such as the Internet, a cellular telecommunications network, or a private WAN. Where the connection 921 is a telephone line, the modem 916 may be a traditional "dial-up" modem. Alternatively, where the connection 921 is a high capacity (e.g., cable) connection, the modem 916 may be a broadband modem. A wireless modem may also be used for wireless connection to the communications network 920. [0037] The computer module 105 typically includes at least one processor unit 905, and a memory unit 906. For example, the memory unit 906 may have semiconductor random access memory (RAM) and semiconductor read only memory (ROM). The computer module 105 also includes an number of input/output (1/0) interfaces including: an audio-video interface 907 that couples to the video display 107, loudspeakers 917 and microphone 980; an 1/0 interface 913 that couples to the keyboard 902, mouse 903, scanner 926, camera 103 and optionally a joystick or other human interface device (not illustrated); and an interface 908 for the external modem 916 and the microscope 101. In some implementations, the modem 916 may be incorporated within the computer module 105, for example within the interface 908. The computer module 105 also has a local network interface 911, which permits coupling of the computer system 900 via a connection 923 to a local-area communications network 922, known as a Local Area Network (LAN). As illustrated in Fig. 9A, the local communications network 922 may also couple to the wide network 920 via a connection 924, which would typically include a so-called "firewall" device or device of similar functionality. The local network interface 911 may comprise an Ethernet circuit card, a BluetoothTM wireless arrangement or an IEEE 802.11 wireless arrangement; however, numerous other types of interfaces may be practiced for the interface 911. 8130727_1 P092066_speci lodge -9 [0038] The 1/0 interfaces 908 and 913 may afford either or both of serial and parallel connectivity, the former typically being implemented according to the Universal Serial Bus (USB) standards and having corresponding USB connectors (not illustrated). Storage devices 909 are provided and typically include a hard disk drive (HDD) 910. Other storage devices such as a floppy disk drive and a magnetic tape drive (not illustrated) may also be used. An optical disk drive 912 is typically provided to act as a non-volatile source of data. Portable memory devices, such optical disks (e.g., CD-ROM, DVD, Blu ray Disc TM), USB-RAM, portable, external hard drives, and floppy disks, for example, may be used as appropriate sources of data to the system 900. Such storage devices may in part perform the role of the data storage 106 described above. Further, the data storage 106 may also be performed using networked-based storage available upon server devices (not illustrated) but able to be coupled to the networks 920 and 922. [0039] The components 905 to 913 of the computer module 105 typically communicate via an interconnected bus 904 and in a manner that results in a conventional mode of operation of the computer system 900 known to those in the relevant art. For example, the processor 905 is coupled to the system bus 904 using a connection 918. Likewise, the memory 906 and optical disk drive 912 are coupled to the system bus 904 by connections 919. Examples of computers on which the described arrangements can be practised include IBM-PC's and compatibles, Sun Sparcstations, Apple MacTM or a like computer systems. [0040] The methods of microscopy thickness estimation may be implemented using the computer system 900 wherein the processes of Figs. 2 to 8, to be described, may be implemented as one or more software application programs 933 executable within the computer system 900. In particular, the steps of the methods of thickness estimation are effected by instructions 931 (see Fig. 9B) in the software 933 that are carried out within the computer system 900. The software instructions 931 may be formed as one or more code modules, each for performing one or more particular tasks. The software may also be divided into two separate parts, in which a first part and the corresponding code modules performs the thickness estimation methods and a second part and the corresponding code modules manage a user interface between the first part and the user. 8130727_1 P092066_speci lodge -10 [0041] The software may be stored in a computer readable medium, including the storage devices described below, for example. The software is loaded into the computer system 900 from the computer readable medium, and then executed by the computer system 900. A computer readable medium having such software or computer program recorded on the computer readable medium is a computer program product. The use of the computer program product in the computer system 900 preferably effects an advantageous apparatus for thickness estimation, particularly in microscopy applications. [0042] The software 933 is typically stored in the HDD 910 or the memory 906. The software is loaded into the computer system 900 from a computer readable medium, and executed by the computer system 900. Thus, for example, the software 933 may be stored on an optically readable disk storage medium (e.g., CD-ROM) 925 that is read by the optical disk drive 912. A computer readable medium having such software or computer program recorded on it is a computer program product. The use of the computer program product in the computer system 900 preferably effects an apparatus for thickness estimation. [0043] In some instances, the application programs 933 may be supplied to the user encoded on one or more CD-ROMs 925 and read via the corresponding drive 912, or alternatively may be read by the user from the networks 920 or 922. Still further, the software can also be loaded into the computer system 900 from other computer readable media. Computer readable storage media refers to any non-transitory tangible storage medium that provides recorded instructions and/or data to the computer system 900 for execution and/or processing. Examples of such storage media include floppy disks, magnetic tape, CD-ROM, DVD, Blu-ray DiscTM, a hard disk drive, a ROM or integrated circuit, USB memory, a magneto-optical disk, or a computer readable card such as a PCMCIA card and the like, whether or not such devices are internal or external of the computer module 105. Examples of transitory or non-tangible computer readable transmission media that may also participate in the provision of software, application programs, instructions and/or data to the computer module 105 include radio or infra-red transmission channels as well as a network connection to another computer or networked device, and the Internet or Intranets including e-mail transmissions and information recorded on Websites and the like. 8130727_1 P092066_speci lodge - 11 [0044] The second part of the application programs 933 and the corresponding code modules mentioned above may be executed to implement one or more graphical user interfaces (GUIs) to be rendered or otherwise represented upon the display 107. Through manipulation of typically the keyboard 902 and the mouse 903, a user of the computer system 900 and the application may manipulate the interface in a functionally adaptable manner to provide controlling commands and/or input to the applications associated with the GUI(s). Other forms of functionally adaptable user interfaces may also be implemented, such as an audio interface utilizing speech prompts output via the loudspeakers 917 and user voice commands input via the microphone 980. [0045] Fig. 9B is a detailed schematic block diagram of the processor 905 and a "memory" 934. The memory 934 represents a logical aggregation of all the memory modules (including the HDD 909 and semiconductor memory 906) that can be accessed by the computer module 105 in Fig. 9A. [0046] When the computer module 105 is initially powered up, a power-on self-test (POST) program 950 executes. The POST program 950 is typically stored in a ROM 949 of the semiconductor memory 906 of Fig. 9A. A hardware device such as the ROM 949 storing software is sometimes referred to as firmware. The POST program 950 examines hardware within the computer module 105 to ensure proper functioning and typically checks the processor 905, the memory 934 (909, 906), and a basic input-output systems software (BIOS) module 951, also typically stored in the ROM 949, for correct operation. Once the POST program 950 has run successfully, the BIOS 951 activates the hard disk drive 910 of Fig. 9A. Activation of the hard disk drive 910 causes a bootstrap loader program 952 that is resident on the hard disk drive 910 to execute via the processor 905. This loads an operating system 953 into the RAM memory 906, upon which the operating system 953 commences operation. The operating system 953 is a system level application, executable by the processor 905, to fulfil various high level functions, including processor management, memory management, device management, storage management, software application interface, and generic user interface. [0047] The operating system 953 manages the memory 934 (909, 906) to ensure that each process or application running on the computer module 105 has sufficient memory in which to execute without colliding with memory allocated to another process. Furthermore, the different 8130727_1 P092066_speci lodge -12 types of memory available in the system 900 of Fig. 9A must be used properly so that each process can run effectively. Accordingly, the aggregated memory 934 is not intended to illustrate how particular segments of memory are allocated (unless otherwise stated), but rather to provide a general view of the memory accessible by the computer system 900 and how such is used. [0048] As shown in Fig. 9B, the processor 905 includes a number of functional modules including a control unit 939, an arithmetic logic unit (ALU) 940, and a local or internal memory 948, sometimes called a cache memory. The cache memory 948 typically includes a number of storage registers 944 - 946 in a register section. One or more internal busses 941 functionally interconnect these functional modules. The processor 905 typically also has one or more interfaces 942 for communicating with external devices via the system bus 904, using a connection 918. The memory 934 is coupled to the bus 904 using a connection 919. [0049] The application program 933 includes a sequence of instructions 931 that may include conditional branch and loop instructions. The program 933 may also include data 932 which is used in execution of the program 933. The instructions 931 and the data 932 are stored in memory locations 928, 929, 930 and 935, 936, 937, respectively. Depending upon the relative size of the instructions 931 and the memory locations 928-930, a particular instruction may be stored in a single memory location as depicted by the instruction shown in the memory location 930. Alternately, an instruction may be segmented into a number of parts each of which is stored in a separate memory location, as depicted by the instruction segments shown in the memory locations 928 and 929. [0050] In general, the processor 905 is given a set of instructions which are executed therein. The processor 905 waits for a subsequent input, to which the processor 905 reacts to by executing another set of instructions. Each input may be provided from one or more of a number of sources, including data generated by one or more of the input devices 902, 903, data received from an external source across one of the networks 920, 902, data retrieved from one of the storage devices 906, 909 or data retrieved from a storage medium 925 inserted into the corresponding reader 912, all depicted in Fig. 9A. The execution of a set of the instructions may in some cases result in output of data. Execution may also involve storing data or variables to the memory 934. 8130727_1 P092066_speci lodge -13 [0051] The disclosed thickness estimation arrangements use input variables 954, which are stored in the memory 934 in corresponding memory locations 955, 956, 957. The arrangements produce output variables 961, which are stored in the memory 934 in corresponding memory locations 962, 963, 964. Intermediate variables 958 may be stored in memory locations 959, 960, 966 and 967. [0052] Referring to the processor 905 of Fig. 9B, the registers 944, 945, 946, the arithmetic logic unit (ALU) 940, and the control unit 939 work together to perform sequences of micro operations needed to perform "fetch, decode, and execute" cycles for every instruction in the instruction set making up the program 933. Each fetch, decode, and execute cycle comprises: (i) a fetch operation, which fetches or reads an instruction 931 from a memory location 928, 929, 930; (ii) a decode operation in which the control unit 939 determines which instruction has been fetched; and (iii) an execute operation in which the control unit 939 and/or the ALU 940 execute the instruction. [0053] Thereafter, a further fetch, decode, and execute cycle for the next instruction may be executed. Similarly, a store cycle may be performed by which the control unit 939 stores or writes a value to a memory location 932. [0054] Each step or sub-process in the processes of Figs. 2 -8 is associated with one or more segments of the program 933 and is performed by the register section 944, 945, 947, the ALU 940, and the control unit 939 in the processor 905 working together to perform the fetch, decode, and execute cycles for every instruction in the instruction set for the noted segments of the program 933. [0055] The method of thickness estimation may alternatively be implemented in dedicated hardware such as one or more integrated circuits performing one or more the functions or sub functions to be described. Such dedicated hardware may include graphic processors, digital signal processors, ASICs, FPGAs or one or more microprocessors and associated memories. Thickness Estimation 8130727_1 P092066_speci lodge -14 [0056] A general overview of a computerised process 200 that can be used to perform thickness estimation of a specimen in a microscope system 100 is shown in Fig. 2. The process 200 is preferably expressed as software typically stored in the HDD 910 and executed by the processor 905 in concert with the memory 906. At step 210 a specimen is loaded onto the microscope stage 110. Next, at step 220 an approximate location is determined at which the specimen would be in the field of view and roughly in focus. In the case where the stage 110 is implemented using a slide scanner running in batch mode, this may be achieved based on the expected location and depth of specimens within a batch. If such information is not available then known techniques may be employed such as focus-finding for depth estimation combined with sampling the transverse space. [0057] Next at step 230 one or more stacks of images are captured using the microscope 101 and camera 103. For example a single image stack may be captured consisting of a set of images at a set of depths and at approximately the same transverse location (i.e. with a common field of view, or at least one region of interest in common across all the layers). The set of depths should span the range of focus from one side to the other side of the best focus of the current view of the specimen, and should be larger than the depth of field of the microscope 101. For example, for a typical microscope set up with a magnification of 20, an air immersion lens with an NA of 0.7, the depth of field is roughly 1.0 micron. If the specimen is a histology slide that is nominally fabricated to have a thickness of around 5 microns, the stack might consist of 10-20 capture layers over a range 10-20 microns centred near the best focus of the current view of the specimen. [0058] Multiple stacks of images may be taken at different environmental conditions (e.g. temperature), wavelengths of light (for example by illuminating at a specific wavelength). Also, in the case that the microscope 101 includes multiple sensors for simultaneous capture, stacks of images may be captured for each sensor. In the latter case, the capture field of view of the multiple sensors may be offset in the transverse or axial directions or both depending on the optical design of the microscope 101. [0059] After the image stacks have been captured, each stack is analysed at a set of selected regions in a loop structure starting at step 240 and returning at step 270. The set of regions may be evenly spaced in the transverse dimension across each stack or may take some other 8130727_1 P092066_speci lodge -15 distribution across the field of view. Each region is selected such that it is possible to find a patch within the region that is accurately aligned in the transverse space with patches from all other regions in the stack. For example, if the tolerances of the microscope capture are well known, then the possible random transverse shift and/or geometric distortion between image regions can be determined, and each region should be larger than the desired patch by the maximum expected transverse shift or geometric distortion. A suitable patch is square shaped and with a size that is considerably larger than the common small features in the specimen. For example, for a histology slide with features roughly of the order of a few microns in size, the patch size should be around 100 microns in size, which may correspond to roughly 400 pixels if the transverse resolution is around 0.25 microns. [0060] Next at step 250, one or more estimates of thickness parameters for the selected region are generated. Within this description the term 'thickness parameter' may mean the thickness of the specimen or the location of the top or bottom surface of the specimen. At step 250, the thickness parameters may be estimated based on the methods disclosed herein and to be described. The position of best focus and/or the centre depth of the stack (which may be different) may also be generated in step 250. Step 250 will be described in further detail with respect to Fig. 3 below. [0061] Next at step 260, the thickness parameters generated at step 250 are stored in the memory 106, such as the HDD 910. Optionally thickness parameters may be collected over a number of regions in one or more specimens. This collected data may be used to form a refined thickness estimate at step 390 described below. Following this, processing continues to step 270 to determine if there are further transverse regions to process whereupon the method 200 returns to step 240, otherwise the method 200 continues to step 280. [0062] Step 280 optionally captures more images using the microscope 101 and camera 103. For example if the thickness and depth estimates obtained at step 250 are to be used to define the parameters of a capture stack for the purpose of virtual microscopy then more images may be required to capture the entire stack. Finally at step 290 the images at steps 230 and/or 280 may be processed using the stored thickness parameters, for example they may be stitched to form a 2D or 3D virtual slide, processed to form stereological estimates, or analysed by Computer Aided Diagnosis tools. 8130727_1 P092066_speci lodge -16 Thickness Parameter Determination [0063] A method 300, used at step 250 to generate thickness parameters for a selected region will now be described in further detail below with reference to Fig. 3. [0064] First at step 310 a stack of aligned patches is generated from a stack of the captured images 104. Step 310 initially involves aligning a stack of image regions, then after this alignment, one patch is generated from each aligned region in the stack of images. Alternatively the patches may be aligned after generating them from the unaligned region. Many techniques are known for aligning a set of image regions. These may be based on pairwise alignment or alignment as a group. The techniques may be based on a variety of transforms (affine, projective, rigid), deformable registration or non-parametric registration and may compare all of the pixels, or a subset of the pixels, at full or subsample resolution. For example, the alignment may be performed by defining an affine transform for each region that transforms it to a common reference frame. The affine transform may be estimated based shift estimates for small patches (50 or 100 pixels in size) located within the regions and from adjacent layers. The small patches may be located at the corners of the region, or may be selected based on the image content in the regions. Further, techniques exist for selecting an optimal set of patches for affine transform estimation. [0065] Once the selected regions have been aligned, a stack of aligned patch images is created, one patch from each region in the stack. A common region in the aligned stack of regions may be found as the intersection of all regions in the common reference frame. A suitable patch is a square region of around 100 microns in the common reference frame. In order to generate the patch images, a suitable interpolation method may be required. Various techniques are known for this purpose such as bi-cubic or Fourier based interpolation. [0066] It is noted that in the case that the microscope 101 is very accurately configured, alignment of regions may not be required as it may be acceptable to simply select a set of patches directly from the captured images at fixed locations. [0067] The concepts of a region and aligned patches are illustrated in Figs. 8A and 8B. Fig. 8A shows a stack 810 of images (801 to 805) of a specimen captured at a set of depths around the best focus. Fig. 8A includes a transverse view of a single stack image 820 (803 in the stack), in 8130727_1 P092066_speci lodge -17 which a region 830 associated with a particular biological structure is shown by a dashed line. A magnified copy of region 830 is shown in Fig. 8B, within which a patch 840 is indicated. In a similar fashion to the stack 810 of images 801-805, a corresponding stack 850 or set of aligned patches 821 to 825 from the selected regions of captured images 801 to 805 is also illustrated in Fig. 8B. It is noted that although the patches are aligned, their images are not identical due to depth variation of the specimen being imaged. [0068] Following the generation of an aligned stack of patches in step 310, processing of the method 300 then moves to step 320 which may be optionally used to estimate one or more thickness parameters for the stack of patches using a prior art method 400, which will be described in further detail below with reference to Fig. 4. [0069] Next, at step 330, a loop structure is then used to process a set of sub-patches within a region in turn. The sets of sub-patches may be evenly spaced in the transverse dimension across the stack of patches or may take some other distribution. The sub-patches are much smaller than the patches, and may be set to a size to match some common small feature sizes in the specimen. For example, the sub-patches may be square regions of 5 microns in width. A large number of sub-patches should be selected over the region - for example there may be a total of 100 sub patches evenly distributed on a grid over the patch. Fig. 8C shows a magnified sub-patch 860 selected within a patch 840 from a captured image 803 of a stack of captured images 810. [0070] Next, at step 340, a stack of aligned sub-patches is generated, each of the sub-patches being taken at the same location from a corresponding patch of the stack of patches 850. A stack 880 of sub-patches 861-865 is also illustrated in Fig. 8C. As was the case for the stack of aligned patches 850, it is noted that the images in the stack are not identical due to depth variation of the specimen being imaged. Furthermore, given that the smaller sub-patch emphasises smaller features, and that smaller features typically vary more rapidly in depth, the variation in image content over the stack 880 of sub-patches is typically more pronounced than the variation over patches. [0071] Next, at step 350, the stack of sub-patches is processed to determine a depth of best focus. The processing of step 350 is preferably performed according to the method 400 that will be described in further detail below with reference to Fig. 4. Following this, processing 8130727_1 P092066_speci lodge -18 continues to step 360 which determines if there are further sub-patches to process, whereupon the method 300 returns to step 330, otherwise the method 300 continues to step 370. [0072] At step 370, outlier rejection and removal is performed on the best focus data collected for the sub-patches. The focus data is a set of depth estimates generated at step 350. Many techniques exist that are suitable for this purpose, for example the upper and lower percentile of depth estimates may be rejected (the top one or two percent might be suitable). Alternatively, a technique may be used based on the thickness parameter estimation that may have been performed for a stack of patches at step 320. For example, any estimates lying above a top surface estimate or below a bottom surface estimate based on the patch may be rejected. [0073] Step 370 is illustrated in Fig. 7A on which various squares 725, 730 and 760 represent a distribution 700 of depth estimates for one region of the specimen shown along a horizontal axis representing depth, each square essentially relating to an estimate derived from a single stack of images from a sub-patch in the region. The true local depth, being that between the top and bottom surfaces in the current particular region of the specimen, are indicated at the depths 705 and 710 respectively. The depth estimate 715 represents the predetermined top surface estimated at step 320 according to the prior art process 400, and the depth estimate 730 is rejected as the estimate 730 is an outlier above this surface, therefore exceeding that limit. Similarly, the depth estimate 720 represents the predetermined bottom surface estimated at step 320 according to the prior art process 400, and the depth estimate 760 is rejected as it is an outlier below this surface, therefore exceeding that limit. Accordingly, estimates 730 and 760 are rejected as outliers, being beyond the bounds 715 and 720 respectively for the estimated top and bottom surfaces. Those estimates 730 and 760 are therefore removed before further consideration of the distribution 700. [0074] Following outlier rejection, step 380 computes thickness parameters for the region of the specimen based on the distribution of the remaining best focus data according to a method 600 which will be described in further detail below with reference to Fig. 6. The method 300 then continues to step 390 which optionally generates a revised or refined thickness estimate for the region of the specimen based on data collected and stored at step 320. [0075] The revised, and preferably refined, thickness estimate generated at step 390 is computed by modifying by subtracting a correction factor from the estimate of thickness obtained at step 8130727_1 P092066_speci lodge -19 320. A suitable correction factor is the average value of the difference between the thickness estimates previously obtained at step 320 and 380 over an ensemble of regions that have been previously stored at step 260. The ensemble of regions should ideally be selected from a similar specimen. For example, the ensemble of regions may be all from histology slides of the same tissue sample type and stain captured using an identical optical configuration. Step 390 effectively operates to correct for systematic errors which typically lead to overestimation of the region, and therefore, the specimen thickness. [0076] In a variation, step 390 may be placed outside the loop formed by steps 240 to 270 (for example before step 280). The statistical analysis of step 390 may be done on the set of regions that have been analysed after collecting data from all regions in the current specimen (i.e. separate step in the process 200) or on data previously calculated on other specimens. An ensemble of regions which change by using the previous point to update the ensemble may be used in the refining process and can produce the same results. Estimates of Depth [0077] A method 400 used at each of steps 320 and 350 to estimate the depth of best focus and thickness parameters of a specimen based on a single stack of image patches (step 320) or sub patches (step 350) will now be described with reference to Fig. 4. [0078] First, a loop structure is employed starting at step 410 that selects each patch or sub-patch in the stack in turn. Next, at step 420 a contrast metric for the current patch/sub-patch image is computed by the processor 905. This may be performed using known techniques. One suitable method applies a window function prior to computing the contrast. The Tukey window function is suitable for the purpose and is given by: w(i, j) = t(i,W)t(j, H) (3) where 8130727_1 P092066_speci lodge - 20 1 1+ Cos (T -2 1 0 ! n ! a( - 1 2K a~1 a(N-1)2 1 Ia N -1 +a1)]0 t(n, N) 1 2 < n < (N - 1) (1 - ) 1 + Cos ( T(2 - 2+ 1 (N -1) (1- a)< n! ; (N - 1) where w is the weighting of the pixel at coordinate (ij), W is the patch width and H is the patch height, and a is a fractional parameter defining the spread of the window function, for which a suitable parameter setting is 0.5. The patch is assumed to be centred at i=j=0. Many focus functions described in the literature are suitable for computing the contrast metric, including the normalised variance. The normalised variance is defined as follows: FNV ) H-1 2 where lij is the intensity of the pixel at location (ij) in the patch, W and H are the width and height of the patch, and p is the mean intensity over the patch. [0079] After the contrast metric has been computed at step 420, processing continues to a testing step 430 which then returns the method 400 to step 410 if there are more patches/sub-patches in the stack. Otherwise, when all patches/sub-patches have been considered, processing continues to step 440 which analyses the functional dependence of the contrast metric on the depth, where the depth corresponding to each patch/sub-patch image is taken as the z-axis microscope stage location during the capture of the image from which the patch/sub-patch was created. This depth may for example be one corresponding to a depth to which the stage 110 is driven via the control 111. Here, a specific measured value of depth is not important; rather, what is important is that the change in driven depth between image captures across the stack is consistent. One suitable technique for determining the functional dependence is based on fitting the contrast metric data as a function of the depth. A suitable fit function is an offset modified Gaussian function F(z): F(z) = poe-P1Iz-P2P4 + |P31 (4) where z is the nominal stage depth of the microscope when capturing the image corresponding to a patch and the parameters pi are the parameters of the fit to the normalised contrast data values. A nonlinear fitting method may be used to create the function fit, for example the parameters of the fit may be found by minimising the mean square error between the normalised contrast values and the fit function using a downhill simplex algorithm. 8130727_1 P092066_speci lodge - 21 [0080] Fig. 5 illustrates a graph having a plot 510 for fitting of contrast metrics to a focus function for a specimen consisting of small square features. The dots in the graph represent the calculated normalised metric values at a discrete set of depths from -8 to +8 microns around a nominal central point near the best focus. The plot 510 represents an offset modified Gaussian function fit to the data represented by the dots. Image patches 501 to 505 represent the patches used to calculate the normalised contrast values around the depths -5, -3, 0, 3 and 5 tm. The dashed line 520 represents the depth of best focus based on the data, while the dashed lines 530 and 540 represent the top and bottom surfaces of the specimen as estimated from the data. These may be estimated using known prior-art techniques. In this image, the depth values happen to be centred around a value of zero, although this centre value is arbitrary. [0081] Based on the modified-Gaussian fit of Equation (5), an estimate of the best focus depth for the patch or sub-patch may be estimated at step 450 as the value of the parameter P2. Next for the implementation of step 320, being the first use of the method 400, at step 460, an estimate of thickness parameter(s) is made. The top and bottom surfaces may be estimated as depths shifted from the best focus depth estimated in step 450 by a distance of +1/pi (i.e. the bottom and top surfaces are at P2 ± 1/p1, respectively). An estimate of the thickness is 2/pi, which is the difference between the bottom and top surface locations. [0082] For the implementation of step 350, step 460 is bypassed, as all that is required at step 350 is the estimate of best focus depth for the patch/sub-patch. Further, whilst the process 400 is described with reference to step 320 for a stack of patches and with reference to step 350 for a stack of sub-patches, in a further implementation, step 350 may be performed simply for "patches" where a coarse (initial) estimate is not available. Also, a "sub-patch" may be considered a "patch" is some implementations. [0083] A number of alternative methods of analysing the contrast metrics data to determine depth of best focus and thickness parameters are known and may alternatively be used for steps 450 and 460. The end result of steps 450 and 460 for stacks of patches or sub-patches across a region is a set of depth estimates which may be considered a set of best focus depths. Analysis of Depths 8130727_1 P092066_speci lodge - 22 [0084] A method 600 used at step 380 for analysing a set of best focus depths corresponding to a region to estimate the thickness of a specimen will now be described with reference to Fig. 6. [0085] First, at step 610, the set of best focus depths (being drawn from an underlying distribution of depths) across the region is sorted into ascending order. Then, at step 620 the top surface location for the region is estimated as the smallest value of best focus depth. For the example illustrated in Fig. 7A, the top surface estimate 750 corresponds to the location of the smallest depth of best focus 740 (note that depth 730 was rejected as an outlier). Then, at step 630 the bottom surface location for the region is estimated as the largest value of best focus depth. In Fig. 7A, the bottom surface estimate 745 corresponds to the location of the largest depth of best focus 735 (note that depth 760 was rejected as an outlier). Finally, at step 640, the estimated thickness for the region is determined by taking the difference between the bottom and top surface location estimates found at step 630 and 620 respectively. [0086] Alternative methods of determining the thickness parameters for method 600 may be based on statistical analysis of the set of best focus depths. For example, the sorted set of best focus depths may be considered a distribution and the top and bottom surface locations may be evaluated from, for example, the 10% and 90% percentile locations in the distribution. Further, depending on a shape of the distribution of best focus depths, any selection of the top or bottom surface locations may be biased based upon any assessed bias in the set of best focus depths. For example, where the distribution shows a predominance of values toward a top surface location, then the actual smallest value of best focus depth may be used for the top surface location, and a percentile value may be used for evaluating the bottom surface location. Example(s) / User Case(s) [0087] The result of step 380 are thickness parameters, including top surface location, bottom surface location, and thickness, for a region of the image of the specimen. Step 390 then refines those parameters. Repeating the loop of steps 240-270 across multiple regions of the specimen gives a distribution of those parameters for specimen. Fig. 7B illustrates distributions of thickness estimates for a test specimen of fixed (true) thickness 795. A curve 765 shows a distribution of thickness estimates based on prior-art techniques, such as those used at step 320. The distribution 765 is relatively narrow, however its accuracy is limited due to the fact that it is centred around a thickness 770 that may be well away from the true thickness 795. This may be 8130727_1 P092066_speci lodge - 23 due to the effect of the point spread function acting on the specimen texture. A distribution curve 775, on the other hand, shows thickness estimates based on the technique described herein and particularly that resulting from step 380 being iterated across multiple regions of the specimen. The estimates of the distribution 775 are centred on a thickness 780 that the present inventors have found to be statistically closer to the true thickness 795, so that the overall accuracy is improved compared to the distribution 765. However the accuracy of the distribution 775 is limited by the relatively large width of the distribution 775. The refined thickness estimates arising from processing according to both steps 380 and 390 have the combined advantage of both the distributions 765 and 775. Particularly, the operation of step 390 provides for identifying the estimated thickness 780 from the distribution 775 and using the estimated thickness 780 to correct the prior art distribution 765 to effectively re-position that distribution at the estimated thickness 780, as depicted by the dashed distribution 790. The distribution of the refined thickness estimates is represented by the dashed line 790 has the same narrow distribution as the distribution 765 centred at the thickness 780 which the present inventors have found to be statistically close to the true thickness 795, and a clear improvement over the thickness 770 of the prior art approaches. INDUSTRIAL APPLICABILITY [0088] The arrangements described are applicable to the computer and data processing industries and particularly for processing of images for the measurement of biological samples using microscopy. [0089] The foregoing describes only some embodiments of the present invention, and modifications and/or changes can be made thereto without departing from the scope and spirit of the invention, the embodiments being illustrative and not restrictive. [0090] For example, the above description recites a hierarchy of imaging levels including specimen, region, patch and sub-patch, each of which may be associated with a corresponding stack of imaging depths. It will therefore be appreciated, that the relationship between a patch and a region, as described, is practically no different of that between a sub-patch and a patch. As described, step 320 operates upon patches to obtain a coarse estimation of thickness parameters and step 350 subsequently operates upon sub-patches to obtain a distribution of parameters for later refinement. Similar processing may be performed between other imaging levels such as 8130727_1 P092066_speci lodge - 24 specimen and region, or region and patch, provided the former is a larger imaging level than the later. (Australia Only) In the context of this specification, the word "comprising" means "including principally but not necessarily solely" or "having" or "including", and not "consisting only of'. Variations of the word "comprising", such as "comprise" and "comprises" have correspondingly varied meanings. 8130727_1 P092066_speci lodge

Claims (20)

1. A method of determining at least one thickness parameter of a specimen from a stack of images of the specimen obtained from a microscope at a series of depths , said method comprising: selecting, within a region of interest, sets of patches at a plurality of corresponding transverse locations in each image within the stack of images; determining a depth value at each transverse location from a set of contrast metrics computed for the set of patches at the transverse location; and determining at least one thickness parameter for the specimen at the region of interest from at least a distribution of the determined depth values.

2. A method according to claim 1, further comprising forming the distribution from depth values determined at different transverse locations across the specimen.

3. A method according to claim 2, wherein the forming of the distribution comprises determining outlier depth values that exceed a predetermined corresponding thickness parameter and removing the outlier depth values from the distribution before the determining of the at least one revised thickness parameter.

4. A method according to claim 3, wherein the predetermined corresponding thickness parameter is determined at an image level larger than the patches.

5. A method of determining at least one thickness parameter of a specimen from a stack of images of the specimen captured at a series of depths, said method comprising: selecting, within a region of interest, a patch at a corresponding location in each image within the stack of images to form a set of patches; determining at least one initial thickness parameter of the specimen at the selected region of interest based on a set of contrast metrics computed for the set of patches; selecting sets of sub-patches at a plurality of corresponding transverse locations within each patch in the set of patches; determining a depth value at each transverse location from a set of contrast metrics computed for the sub-patches at the transverse location; and 8130727_1 P092066_speci lodge - 26 determining at least one revised thickness parameter for the specimen at the region of interest from a distribution of the determined depth values and the initial thickness parameter.

6. A method according to claim 5 wherein the thickness parameter is selected from the group consisting of a thickness of the specimen, a location of a top surface of the specimen, and a location of a bottom surface of the specimen.

7. A method according to claim 5, further comprising determining outlier depth values that exceed a corresponding thickness parameter and removing the outlier depth values from the distribution before the determining of the at least one revised thickness parameter.

8. A method according to claim 7, wherein the corresponding thickness parameter comprises the initial thickness parameter.

9. A method according to claim 7, wherein the corresponding thickness parameter comprises a value of an upper or lower percentile of determined depth values.

10. A method according to claim 5, further comprising selecting the patch within the region of interest for each image from an aligned stack of patches within the region.

11. A method according to claim 5, wherein the determining of the at least one initial thickness parameter comprises forming estimates of a first and second thickness parameters at the current region of interest based on the set of contrast metrics computed for the set of patches.

12. A method according to claim 5, wherein the determining of the at least one revised thickness parameter comprises analysing the distribution of depth values for the region to obtain current estimates for a top surface location and a bottom surface location of the specimen, the analysing comprising at least one of: (i) selecting a top surface location from a smallest one of the depth values; (ii) selecting a bottom surface location from a largest one of the depth values; and (iii) evaluating a percentile depth value of the distribution as one of the top or bottom surface locations.

13. A method according to claim 12, wherein the determining of the at least one revised thickness parameter comprises forming a correction factor using at least the initial thickness 8130727_1 P092066_speci lodge - 27 parameter and the current estimates and modifying the current estimates using the correction factor.

14. A method according to claim 13 wherein the initial thickness parameter comprises an initial thickness estimate, and a current thickness estimate comprises a difference of the top surface location and the bottom surface location, and the correction factor is an average of the difference between the initial thickness estimate and the current thickness estimate across the regions of the specimen, and the revised thickness parameter is a revised thickness formed by subtracting the correction factor from the initial thickness estimate.

15. A method according to claim 5, further comprising setting a size of the sub-patches according to a feature size within the specimen.

16. A method according to claim 5, wherein the specimen comprises a stained histology slide.

17. A computer readable storage medium having a program recorded thereon, the program being excutable by computerised apparatus to determine at least one thickness parameter of a specimen from a stack of images of the specimen obtained from a microscope at a series of depths, said program comprising: code for selecting, within a region of interest, sets of patches at a plurality of corresponding transverse locations in each image within the stack of images; code for determining a depth value at each transverse location from a set of contrast metrics computed for the set of patches at the transverse location; and code for determining at least one thickness parameter for the specimen at the region of interest from at least a distribution of the determined depth values.

18. A computer readable storage medium having a program recorded thereon, the program being excutable by computerised apparatus to determine at least one thickness parameter of a specimen from a stack of images of the specimen captured at a series of depths, said program comprising: code for selecting, within a region of interest, a patch at a corresponding location in each image within the stack of images to form a set of patches; code for determining at least one initial thickness parameter of the specimen at the selected region of interest based on a set of contrast metrics computed for the set of patches; code for selecting sets of sub-patches at a plurality of corresponding transverse locations 8130727_1 P092066_speci lodge - 28 within each patch in the set of patches; code for determining a depth value at each transverse location from a set of contrast metrics computed for the sub-patches at the transverse location; and code for determining at least one revised thickness parameter for the specimen at the region of interest from a distribution of the determined depth values and the initial thickness parameter.

19. A microscope system comprising: a microscope having a contollable stage upon which a specimen is positionable; a camera for capturing images of the specimen via the microscope; and a processor system coupled to the camera for receiving and storing the images and for controlling movement of the stage during image capture, the processor system having a memory with a program recorded thereon, the program being executable by a processor to determine at least one thickness parameter of the specimen from a stack of the images of the specimen captured at a series of depths, said program comprising: code for selecting, within a region of interest, a patch at a corresponding location in each image within the stack of images to form a set of patches; code for determining at least one initial thickness parameter of the specimen at the selected region of interest based on a set of contrast metrics computed for the set of patches; code for selecting sets of sub-patches at a plurality of corresponding transverse locations within each patch in the set of patches; code for determining a depth value at each transverse location from a set of contrast metrics computed for the sub-patches at the transverse location; and code for determining at least one revised thickness parameter for the specimen at the region of interest from a distribution of the determined depth values and the initial thickness parameter.

20. A microscope system comprising: a microscope having a contollable stage upon which a specimen is positionable; a camera for capturing images of the specimen via the microscope; and a processor system (150,106,107) coupled to the camera for receiving and storing the images and for controlling movement of the stage during image capture, the processor system having a memory with a program recorded thereon, the program being executable by a processor to determine at least one thickness parameter of a specimen from a stack of images of the 8130727_1 P092066_speci lodge - 29 specimen captured at a series of depths, said program comprising: code for selecting, within a region of interest, a patch at a corresponding location in each image within the stack of images to form a set of patches; code for determining at least one initial thickness parameter of the specimen at the selected region of interest based on a set of contrast metrics computed for the set of patches; code for selecting sets of sub-patches at a plurality of corresponding transverse locations within each patch in the set of patches; code for determining a depth value at each transverse location from a set of contrast metrics computed for the sub-patches at the transverse location; and code for determining at least one revised thickness parameter for the specimen at the region of interest from a distribution of the determined depth values and the initial thickness parameter. CANON KABUSHIKI KAISHA Patent Attorneys for the Applicant Spruson&Ferguson 8130727_1 P092066_speci lodge