Classifications

• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B21/00—Microscopes
  • G02B21/24—Base structure
  • G02B21/241—Devices for focusing
  • G02B21/244—Devices for focusing using image analysis techniques
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/0002—Inspection of images, e.g. flaw detection
  • G06T7/0012—Biomedical image inspection
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B21/00—Microscopes
  • G02B21/0004—Microscopes specially adapted for specific applications
  • G02B21/002—Scanning microscopes
  • G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
  • G02B21/008—Details of detection or image processing, including general computer control
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B21/00—Microscopes
  • G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
  • G02B21/365—Control or image processing arrangements for digital or video microscopes
  • G02B21/367—Control or image processing arrangements for digital or video microscopes providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/0002—Inspection of images, e.g. flaw detection
  • G06T7/0012—Biomedical image inspection
  • G06T7/0014—Biomedical image inspection using an image reference approach
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B21/00—Microscopes
  • G02B21/0004—Microscopes specially adapted for specific applications
  • G02B21/002—Scanning microscopes
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10056—Microscopic image
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10141—Special mode during image acquisition
  • G06T2207/10148—Varying focus
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30004—Biomedical image processing
  • G06T2207/30024—Cell structures in vitro; Tissue sections in vitro
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30168—Image quality inspection

Definitions

• the present invention relates to a method for analysing the efficacy of an imaging scan acquired using a microscope scanner.
• a biopsy of tissue is mounted in a block of paraffin as support.
• a thin layer of approximately 4pm is taken from the block and placed onto a slide.
• the slide is stained to show the tissue of interest and the background tissue.
• the paraffin is removed, by evaporation, and a protective coverslip is glued over the tissue.
• the tissue is then scanned to a digital file for either the pathologist to examine or software algorithms to examine and indicate if disease is present.
• the pixel size for digitising an image is around 0.25pm and the natural aperture of the imaging limits the depth of focus to around 1 pm.
• the slide on which the tissue is mounted is not flat to within 1 pm.
• Standard equations of a perfectly manufactured slide indicate that the slide will bend by around 2pm under its own weight and distortions from the manufacture and mounting of the slide can give much larger errors.
• the tissue itself is typically not always a uniform thickness with some parts of the tissue being sliced thinner than other types, or the tissue is sliced in a wedge shape.
• a Whole Slide Imaging scanner typically has a focus tracking mechanism such as that described by US 9116035 to achieve the correct focus across all of the tissue. However, for a number of reasons the tracking of the focus of the tissue may not be perfect across the whole of the tissue.
• the invention is set in the context of finding a quicker and more effective method for analysing the efficacy of the scanned image so that a second scan may be performed, if needed, that corrects any defects found in the first scan.
• a first aspect of the invention provides a method for operating a microscope scanner comprising a detector array, the method comprising: performing a first imaging scan of one or more area(s) of interest, AOI, on a target, wherein the target comprises a sample, and wherein performing the first imaging scan comprises: moving the detector array relative to the target along an image scan path and acquiring an image of the target at each of a plurality of locations along the image scan path using the detector array; generating focus control data during the imaging scan, wherein generating focus control data comprises calculating a focus merit value at each said location along the image scan path; and adjusting the focal height of the detector array along the image scan path based on said focus merit values; analysing the efficacy of the first imaging scan using the focus control data and determining a change to one or more scanning parameters from the first imaging scan, for the performance of a second imaging scan, based on this analysis.
• An improved method is therefore provided which analyses the efficacy of an imaging scan using focus control data generated during the imaging scan.
• the focus control data is generated for the purposes of performing the imaging scan only, whereas it is the image that is produced from the imaging scan that would be looked at to assess if the scan had been successful.
• the above method uses the focus control data to analyse the efficacy of the imaging scan. This may be achieved through a number of different techniques (as will be discussed), and each of these facilitate a quick and effective result without placing a high demand on computer processing resources. This result may be the determination of a change to one or more scanning parameters from the first imaging scan so that corrective action can be taken in a second imaging scan. This ensures that the user may obtain a high quality scanned image of the target with minimal delay.
• the first imaging scan is performed across a first scanning region comprising the one or more AOI, and determining a change to one or more scanning parameters comprises selecting a second scanning region on the target that is different from the first scanning region.
• determining a change to one or more scanning parameters comprises selecting a second scanning region on the target that is different from the first scanning region.
• Analysing the efficacy of the first imaging scan therefore preferably comprises: identifying one or more boundary locations on the target where the focus control data indicates that the detector array was predominantly in focus at a boundary of the first scanning region; and selecting a second scanning region preferably comprises adjusting the first scanning region to incorporate an area of the target outside of the first scanning region surrounding the one or more boundary locations.
• the first scanning region may be increased by a pre-set amount surrounding the boundary location(s) in attempt to encompass the outline of the sample outside of the first scanning region.
• a mathematical prediction may be made of the shape of the sample on the target.
• analysing the efficacy of the first imaging scan may comprise identifying one or more sample focus regions on the target from the focus control data, wherein the focus control data indicates that the detector array was predominantly in focus throughout the one or more sample focus regions; and selecting a second scanning region may comprise interpolating a region of the target across which the sample extends outside of the one or more sample focus regions imaged during the first imaging scan.
• the shape of the interpolated region may be found by applying a shape fitting to an outline of a said sample focus region, such as fitting a circle or arc or spline curve to a tissue edge determined from the focus control data. This enables a more intelligent estimation to be made of where the sample extends outside of the first scanning region so that the scanning region is extended by the amount necessary to cover the sample, without also extending over regions of the target where the sample is absent.
• Evaluating the efficacy of the first imaging scan preferably further comprises identifying one or more poor focus regions on the target, wherein the focus control data indicates the that the detector array was predominantly out of focus throughout the one or more poor focus regions during the first imaging scan.
• the poor focus region(s) may be used in the process of determining the second scanning area.
• determining a change to one or more scanning parameters may comprise adjusting a focusing procedure so as to obtain images of the target at a different focal height within one or more regions of the target, such as the one or more poor focus regions, during the second imaging scan in comparison with the corresponding images from the first imaging scan.
• Such a scenario may occur where the analysis of the focus control data indicates that the detector array was not properly focused on the sample in one or more AOI during the first imaging scan.
• a change in a scanning parameter, in the form of a new focusing procedure, is then determined so that the detector array has a different focus within these regions during a second imaging scan.
• the method preferably further comprises: performing a pre-scan so as to generate a pre-scan image of the target, wherein the depth of focus for the pre-scan is in excess of 500 micrometres and wherein the depth of focus for each image acquired during the first imaging scan is below 3 micrometres; and performing an automated image analysis of the pre-scan image to produce a first probability map indicating where the sample is expected to be present on the target, wherein the one or more AOI are selected from the first probability map.
• This process of performing a pre-scan to identify one or more AOI has been found to be particularly fast and effective in enabling the parameters to be set from which the first imaging scan can be performed.
• the detector array preferably comprises an array of photodetectors, typically arranged as a narrow strip of CCDs. As such the detector array is typically a line detector but it could also be a 2D camera.
• the first imaging scan typically further comprises generating a first image of the target from the images acquired along the image scan path. Such an image may correspond to the combination of each of the individual images or “lines” acquired by the detector array along the image scan path. The first image may therefore have a relatively high resolution and so require significant computer processing resources to load. It is preferable therefore for a second image of the target to be derived from the first image, wherein the resolution of the second image is lower than the resolution of the first image.
• the second image may be viewed by a user to obtain a “snapshot” of the target at low magnification without the additional data present in the first image needing to be loaded by a computing system.
• this second image may be further used to assist with the identification of the one or more poor focus regions earlier described because it will have a much smaller depth of focus in comparison with the pre-scan image used to select the one or more AOI.
• Surface detail, such as dirt or scratches on the target, which may have been visible in the pre-scan image and incorrectly identified as corresponding to sample regions, will likely therefore not appear in the first or second image.
• Analysing the efficacy of the first imaging scan may therefore further comprise: performing an automated image analysis of the second image to produce a second probability map for regions of the target where the sample is expected to be present; and identifying surface detail in the pre-scan image corresponding to locations on the target where the first probability map indicates the sample is expected to be present and the second probability map indicates that the sample is not present; wherein said one or more poor focus regions exclude locations on the target having said surface detail.
• the resolution of the second image is preferably equal or approximately equal to the resolution of the pre-scan image. This enables the same or similar algorithms to be used to generate the second probability map as were used to generated the first probability map without incurring significant processing resources. This also enables effective comparison between the first and second probability maps for the identification of the surface detail.
• the focus control data may further comprise an estimated focal surface for the target (also referred to as the “target height”) and/or the actual focal height of the detector array. From this data it is hence be possible to identify the difference between where the detector array was and where it was desired to be for any image acquired during the first imaging scan. This difference is a known positional error. There may be a tolerance to the positional error but when this tolerance is exceeded the corresponding locations may be identified for the performance of a second imaging scan.
• the focus control data may comprise an estimated focal surface of the target at each location along the image scan path, and the actual focal height of the detector array may be monitored for each location along the image scan path where an image is acquired.
• Identifying one or more poor focus regions on the target may then comprise identifying one or more positional error locations at which the difference between the estimated focal surface and the actual focal height of the detector array exceeds an error threshold.
• the error threshold is typically approximately equal to or exceeds the depth of focus for the detector array.
• the first imaging scan further may comprise the generation of a first image of the target from the images acquired along the image scan path, as earlier discussed.
• Identifying one or more poor focus regions may then further comprise the performance of an automated image analysis of one or more sub-regions of the first image to identify if the focus of the first image is acceptable and designating each sub-region of the target for where the focus is unacceptable as a ‘poor focus region’, wherein the one or more sub-regions are selected based on the focus control data.
• the one or more sub-regions include the positional error locations.
• the one or more sub-regions may also be determined based on the area of the target occupied by the positional error locations.
• Determining a change to one or more scanning parameters may comprise adjusting a focusing procedure for the second imaging scan, as earlier discussed. This focusing procedure may rely on the measurement of an in-focus position for at least one seed location on the target.
• the method prior to the first imaging scan the method typically further comprises measuring the in-focus position for at least one seed location in the one or more AOI, and setting the focal height at one or more locations along the image scan path based on the measured in-focus position for the seed location(s).
• the one or more scanning parameters that are changed may therefore be selected from the group comprising: the area of the target to be imaged, the number of seed locations on the target, and the position of the seed location(s) on the target.
• analysing the efficacy of the first imaging scan may comprise identifying an error in the in-focus position measured for one or more said seed locations, which may be performed by comparing the focus control data from the first imaging scan with the measured in-focus position for the one or more seed locations prior to the first imaging scan.
• comparing the focus control data from the first imaging scan with the measured in-focus position for the one or more seed locations prior to the first imaging scan may comprise comparing the focus control data from the seed location during the first imaging scan with the measured in-focus position for the one or more seed locations prior to the first imaging scan.
• the focal height of the detector array is typically modulated along the image scan path so as to track an estimated focal surface of the target based on the focus control data acquired.
• the focal height of the detector array will therefore not necessarily arrive at the measured in-focus position for the seed location during the first imaging scan, however if the focal height is significantly different (for example the difference is above a threshold separation) then the infocus position may be identified as being in error.
• the first imaging scan may be performed across a plurality of AOI and the in-focus position measured by adjusting the focal height of the detector array through a range of positions normal to the surface of the target at each said seed location. If the average focus of each of these are not similar or within expected tolerances due to slide tilt, slide distortion, thickness variance of tissue then it can be assumed that the detector array will not have correctly focused on the sample in at least one of the AOIs during the imaging scan. Analysing the efficacy of the first imaging scan may therefore comprise calculating an average focal height of the sample in each said AOI from the focus control data.
• the determining of a change to one or more scanning parameters from the first imaging scan may comprise modifying the range of positions for which the focal height is adjusted when monitoring the in-focus position at the seed location in the first AOI to exclude the average focal height for the first AOI calculated from the first imaging scan, wherein the average focal height calculated for the first AOI exceeds the average focal height calculated for the second AOI.
• the lowest focus level of the AOIs will typically give a good indication of the correct tissue focus level.
• the average focal height calculated for the second AOI is preferably therefore the lowest focal height from the group comprising the average focal height calculated for each said AOI. This technique enables the parameters of a second imaging scan to be determined so that each AOI is correctly imaged.
• the second imaging scan may be performed.
• the method typically therefore further comprises performing the second imaging scan of the target using the detector array with the determined change in the one or more scanning parameters relative to the first imaging scan implemented.
• a second aspect of the invention provides a computer program product containing instructions which, when executed using a microscope scanner comprising a detector array and a target comprising a sample, causes the microscope scanner to perform the method of the first aspect.
• the second aspect shares similar features and advantages as discussed in connection with the first aspect.
• the microscope scanner is typically a bright-field microscope scanner, preferably still a virtual microscope, and the computer program product typically corresponds to a non-transient computer readable medium, such as a hard drive.
• the target may take a number of forms however preferably the target comprises a biological tissue sample.
• the target preferably shows no substantial variation in surface height and so is essentially flat.
• the target could be a stained tissue extract that has been sliced and retained between flat glass slides.
• Figure 1 is a schematic illustration of a microscope scanner to be operated in accordance with an embodiment of the invention
• Figure 2 is a flowchart indicating an embodiment of the invention
• Figure 3 is a flow chart indicating processes for analysing the efficacy of an imaging scan in accordance with an embodiment of the invention
• Figure 4A is an example of a pre-scan image acquired in accordance with an embodiment of the invention.
• Figure 4B is an example of a second image acquired in accordance with an embodiment of the invention.
• Figure 5A is a first example of a poor focus regions on the target identified in accordance with an embodiment of the invention.
• Figure 5B is a second example of a poor focus regions on the target identified in accordance with an embodiment of the invention.
• FIG. 1 An example of a bright-field microscope scanner 1 is shown in Figure 1.
• This comprises a scan head 2 comprising a line scanner 3 in the form of a linear photodetector array.
• the scan head 2 further comprises an adjustable focus system 4 and a pre-scan camera 12.
• a platen 5 is provided upon which is positioned a target 6 in the form of a pathology slide containing a sample.
• a drive mechanism 7 attaches the scan head 2 to a track 8 such that the scan head can be moved with respect to the target 6 as indicated by the arrows 9.
• the track 8 can also allow the scan head 2 to be displaced laterally (perpendicular to the optical axis 11 of the scan head 2 and the scanning direction 9) for imaging adjacent swathes.
• the microscope scanner 1 is controlled using an electronic controller in the form of a computer system 10. Coordinate axes are also provided in Figure 1 for reference to in the following example.
• the ordinate z-axis is aligned with the optical axis of the scan head 2 and the focal height of the target 6, whereas the abscissa x-axis represents the scan direction (parallel to the arrow 9 in Figure 1).
• the surface of the target 6 is aligned in the xy plane and a lens of the adjustable focus system 4 is moveable along the z-axis.
• the scan head 2 remains stationary and the target 6 is moved in the xy plane in order to achieve relative motion between the scan head 2 and the target 6 during the scanning processes.
• the pre-scan camera 12 may also form a separate unit from the scan head 2 and operable independently of the scan head 2.
• the operation of the microscope scanner 1 in performing a method according to an embodiment of the invention will now be described with reference to Figures 2 to 5.
• the method begins at step 100 ( Figure 2) where a pre-scan is performed to acquire a pre-scan image of the target 6.
• the scan head 2 is traversed along the track 8 with images of the target 6 being acquired using the pre-scan camera 12.
• the prescan camera 12 is configured to acquire images of the target at a depth of focus of 1 mm, which is significantly higher than the thickness of the sample (typically between 2 to 10 pm).
• the focal height of the pre-scan camera 12 is therefore not adjusted during the pre-scan and the pre-scan image is rapidly produced at a relatively low resolution, for example at 1200 dpi (47 lines per mm).
• the pre-scan image provides an overview of the outline of the sample on the target 6 from which one or more areas of interest, AOI, can be selected for imaging during a first imaging scan.
• the imaging scan progresses at a much slower speed in comparison with the pre-scan and so it is desirable to only image those areas on the target 6 where the sample is present during the imaging scan.
• the “blank” or “empty” regions of the target can be excluded.
• the pre-scan image is processed at step 101 using conventional software so that each region in which the sample is visible in the pre-scan image is identified as an AOI.
• an automated image analysis of the prescan image is used to produce a first probability map indicating where the sample is expected to be present on the target, with the one or more AOI being selected from the first probability map.
• the boundaries of the AOI may be selected so as to lie just outside the outer edges of the sample (as identified from the pre-scan image) so that the sample is fully contained within the AOI.
• the pre-scan image may then be displayed to a user using the computer system 10 with the one or more AOI demarcated.
• the user may then manually adjust the boundaries of the one or more AOI using the computer system if desired.
• step 102 at which point a seed location is selected in the xy plane within each AOI (typically at the centre of the AOI) and the focal height of the sample measured at that location using the line scanner 3.
• the focal height of the adjustable focus system 4 is varied through a predefined range of focal heights so as to obtain image information from the target 6 at a number of different focal positions along the z-axis with a depth of focus to around 1 pm.
• An in-focus position is then calculated from the image information at each focal position through use of a focus parameter in the form of a focus merit value.
• a focus merit value is a numerical measure of the degree of complexity within an image, wherein the greater the value the greater the detail in the image. A more in-focus image has a corresponding higher merit value.
• the focal height having the highest merit value is then stored as the measured in-focus position for the seed location and the process repeated for any remaining seed locations identified.
• An image scan path is calculated across which the scan head 2 will be moved so that the line scanner 3 images the entire surface of the one or more AOI.
• the line scanner 3 is moved to a starting location along the image scan path and focused at the in-focus position for the respective AOI.
• the first imaging scan then commences at step 103, at which point the line scan head 2 is moved across the image scan path and images obtained of a plurality of adjacent locations on the target 6.
• focus control data is recorded including: the focus merit value for the line scanner 3, an estimate of where the in-focus level is and the actual focal height of the line scanner 3.
• the estimated in-focus level may be calculated using techniques known in the art, such as described in US 9116035 B2.
• the focal height of the line scanner 3 is adjusted between each xy location along the image scan path in the direction of or to the estimated in-focus position.
• the target 6 is imaged in one or more swathes in this manner until each AOI has been imaged entirely. Then, at step 104, the individual images acquired by the line scanner 3 are combined together to form a first image of the target.
• the first image is desirable for the first image to include the entirety of the sample across the xy plane and to be in-focus throughout. However, a number of errors can occur in the preceding method steps that prevent this from occurring.
• the efficacy of the first imaging scan (which may be thought of as the extent to which the first imaging scan produced the intended result) is then analysed at step 105. This analysis may involve any number of individual sub-processes for checking respective criteria, examples of which will be discussed in further detail with reference to Figure 3. Typically, however, it involves an assessment regarding whether the area on the target imaged sufficiently covered the area of the target 6 occupied by the sample, and/or whether the sample is correctly focused throughout the image. Importantly, this process is performed with reference to the focus control data acquired during the first imaging scan.
• a change to a scanning parameter is determined at step 106 so that these may be corrected in a second imaging scan.
• the second imaging scan is then performed at step 107 with the relevant changes in the scanning parameter (such as the focusing procedure used or the area of the target 6 imaged) so as to improve the efficacy of the imaging scan.
• step 201 the first imaging scan is performed so that each AOI is imaged using the line scanner 3. This is equivalent to step 103 in Figure 2.
• step 202 the computer system 10 performs the first of a series of tests to analyse the efficacy of the first imaging scan. The question is asked whether more than one AOI were imaged during the first imaging scan, and if the answer is “yes” then the method proceeds to step 203, otherwise it proceeds to step 205.
• the average focal height of the AOI are not similar to each other or within expected tolerances due to slide tilt, slide distortion, thickness variance of tissue then it can be assumed that at least one of the AOIs will not have the correct focus.
• the line scanner 3 will have mistakenly focused on something on the top of the coverslip for the target 6 and if there is enough detail present on the top of the coverslip it will appear to have scanned. In this case there will be typically be the thickness of a coverslip difference between focus levels, around 170pm. Therefore, at step 203, an assessment is made whether the average focus of each AOI matches a tolerance. The AOI having the lowest average focal height is found and the tolerance is set to within a threshold difference from the lowest average focal height.
• the threshold may be set depending on the application but it typically exceeds the thickness of the sample and in this case is 20pm. More generally the threshold might typically be selected within the range of 10-30pm. If the average focus of each AOI is below this threshold then it can be assumed that the line scanner 3 has not focused on a region of the coverslip and the method proceeds to step 205. Otherwise, if the average focus one of the AOI exceeds the threshold then the method proceeds to step 204. At step 204 a change in a scanning parameter from the first imaging scan is determined for the performance of a second imaging scan. In particular, a change in the focusing procedure is determined. The lowest average focal height of the AOI will give a good indication of the correct tissue focus level.
• a focus merit tissue map is generated indicating the focus merit value calculated for each location along the image scan path during the preceding imaging scan. From the focus merit values it can be determined where the line scanner 3 was in focus. Where tissue is present, even on slightly out of focus areas, focus merit values will be large. This gives an alternative map of where tissue is present. If high focus merit values are recorded at the edge of a first scanning region imaged during the first imaging scan then it can be assumed that the AOI was set too small to capture all of the tissue. At step 206 any “boundary locations” where the focus merit values are above a threshold number at a boundary of the AOI are identified. If there are a series of adjacent boundary locations identified, indicating that the first scanning region was too small to capture the sample, the method proceeds to step 207.
• a change to a scanning parameter is determined by calculating a second scanning region covering a larger portion of the target than the first scanning region.
• the second scanning region may simply be a small increase in the boundary where the first scanning region was not large enough or a mathematical prediction of the sample shape, such as fitting a circle or arc or spline curve to a tissue edge determined from the focus merit tissue map. It is desirable not to increase the scanning region unnecessarily so as to cover regions of the target where the sample can be assumed to be absent as this will unnecessarily increase the scanning time.
• a second imaging scan is then performed across the new scanning region so as to image the sample in the locations not included to the first imaging scan. If the focus merit map does not indicate that the first scanning region was too small to sufficiently capture the sample then the method proceeds from step 206 to step 208. Any surface detail visible in the pre-scan image is identified at step 208. This is achieved through a comparison of the pre-scan image with the imaging data acquired from the first imaging scan. It will be recalled from step 104 ( Figure 2) that a first image of the target was produced corresponding to a combination of the individual images acquired during the imaging scan. This image will have a completely different numerical aperture and as such a completely different depth of focus to the pre-scan image.
• the pre-scan image will have a larger depth of focus which means defects such as dirt and scratches on the surface of the slide and coverslip are in focus whilst the small depth of focus of the imaging scan means the dirt and scratches on the surface of the slide and coverslip are not in focus.
• the resolution of the first image is scaled down to a resolution similar or identical to the pre-scan image.
• a typical Whole Slide Imaging image file contains multiple layers of images, each at a different resolution. This enables the image viewing application to zoom out of the image and move around the image without having to access large amounts of data. For example, if the image file has 0.25pm pixels and the viewing application shows 2000x3000 pixels to show this requires 6 Mpixels to be read which, uncompressed, is 18 Mbytes of data. With compression, this might only be 2Mbytes of data. If the viewer were asked to display a larger area at lower resolution “zoom out” at, for example 10x the area, then to read this from the 0.25pm pixel data would require 60Mpixels or 20Mbytes of compressed data. To avoid this condition, it is typical that the file contains an addition image that is near to the required resolution.
• the second image of the target and is analysed at step 208 to identify surface detail.
• the second image typically has a resolution that is similar to or identical to the pre-scan image. For example, the resolution of these images may be comparable to within a factor of two.
• the second image may then resolve similar sized features to the pre-scan image, and can be processed considerably faster than the HR “first image” (for example approximately 4000 times faster).
• Figure 4A show an example of a pre-scan image of the target with two scratches shown in the upper middle portion of the image.
• the corresponding second image, derived from the first image, of similar or equal resolution to the pre-scan image is shown in Figure 4B and does not show these scratches.
• a first probability map of the pre-scan image was generated in step 101 in order to identify the one or more AOI.
• a probability map (referred to herein as the “second probability map”) is generated in a similar manner but using the second image of the target to identify regions of the image apparently showing the sample. The second probability map is then subtracted from the first probability map to show the regions of the target where surface detail is visible in the first image.
• step 208 the focus control data from the first imaging scan is analysed to identify one or more “poor focus regions” where the line scanner 3 was not focused on the sample. This may be achieved by identifying locations on the target where the focus merit value was below a threshold number during the first imaging scan. It can be expected that the focus merit values recorded for the locations on the target where the surface detail was found may be below this threshold. Such locations may have only been imaged because they were incorrectly identified as having tissue in step 101 and so it is not necessary to repeat the imaging scan in these locations. However, if one or more poor focus regions remain (excluding the surface detail locations) then the method proceeds to step 210.
• the method only proceeds from step 209 to step 210 if the poor focus region(s) have a minimum size, for example a length of at least 0.5 mm in any direction, otherwise it proceeds to step 211.
• the minimum size may optionally be set by the user of the computer system 10.
• a new focusing procedure is determined for the performance of a second imaging scan. In this case it is determined that the first imaging scan should be repeated but with the in-focus position being found for additional seed locations at step 101 within the one or more poor focus regions. With the in-focus position being measured at these additional locations, the second imaging scan will proceed with the line scanner 3 imaging the target at a different focal height across the one or more poor focus regions.
• step 209 If the outcome of step 209 is that there are no remaining poor focus regions then the method instead proceeds to step 211 .
• the focus control data For each location along the image scan path the focus control data provides an estimate of the correct focal height for the sample (also referred to as the “focal surface”) and a record of the actual focal height for the line scanner 3 when the image was acquired. From this it can be determined where the scanner focus was outside one depth of focus of the tissue focus, thereby enabling areas of tissue where the focus was known to be in error to be identified. This might be more typical at the very edge of the tissue as the scanner has to predict where the focus of the tissue is as it approaches the tissue.
• the focus mechanism identifies the prediction error but it can take a little distance to correct the prediction error.
• Other conditions that may occur which give this focus tracking error may include the tissue changing focus rapidly such as when the tissue is folded over itself or buckled.
• a positional error map is created in which all areas of the tissue for which the actual focal height of the line scanner 3 deviated from the actual focal surface of the sample by at least 1 pm, or where the correct focal height was unknown, are identified.
• Figure 5A shows an image of the target in which the clear slide is shown as white and the regions of the sample that are in focus are shown as grey. The poor focus regions (where the positional error exceeds 1 pm or the focus is unknown) are shown as black. There will be a tolerance to the positional error but when the error is too large it may be deemed unacceptable.
• an automated analysis of the quality of the first image may be performed at step 213 that is limited to these regions, for example using an algorithm similar to that described in US 2019/0266726 (earlier discussed).
• Other automated analysis techniques may alternatively be used, including 2D standard deviations, gradient analysis and edge filtering (as known in the art).
• a result of the analysis will either be that the focus of the image is acceptable for the purposes of the user (typically for performing a pathology assessment), or it is unacceptable.
• the process of image analysis is typically computationally demanding, because only a small proportion of the first image is now required for analysis, it can be completed relatively quickly. In the example of Figure 5A less than 1% of the image is required to be examined. Alternatively, it is possible to only examine the HR “first image” for the area of tissue where the focus is unknown, meaning that an even smaller proportion of the image is analysed in this manner.
• Figure 5B provides a second example of an image of the target that is the same as Figure 5A except that the regions now identified as being of acceptable focus quality (following performance of the above image analysis technique) are grey also. If any poor focus areas remain in the first image (corresponding to those regions marked black), or if such areas exceed a threshold proportion of the first image, a change to a scanning parameter may be determined for the performance of a second imaging scan. For example, it may be determined that the first imaging scan should be repeated with in-focus measurements being taken for additional seed locations inside the remaining poor focus regions. Optionally the resulting map of errors may instead be displayed to a user as a guide to which areas of the image to examine, which will improve the speed and accuracy of the task.
• step 212 If at step 212 it is determined that the positional errors do not exceed the depth of focus, or that the poor/unknown focus areas regions being of unacceptable quality do not occupy a threshold proportion of the first image, the method instead proceeds to step 214.
• the in-focus position initially calculated for one or more seed locations prior to the first imaging scan is compared with the estimated focal surface for that location identified during the first imaging scan. If these are not similar, for example to within a threshold separation of each other) then an error can be assumed to have occurred and the method proceeds to step 215.
• step 215 a change in a focusing procedure is determined for the performance of a second imaging scan. In particular, the position of the seed location(s) is moved on the target. A second imaging scan may then proceed as described in Figure 2 but with the new seed locations.
• step 215 If the result of step 215 is that each seed focus matches the tracked focus ascertained from the focus control data then first imaging scan is determined to be acceptable. The scanning parameters will then not require changing for the performance of a second imaging scan and the first image of the target generated in step 104 may be displayed to the user for analysis.
• each of steps 203, 206, 209, 212 and 214 relate to examples of processes for analysing the efficacy of the first imaging scan using the focus control data and determining a change to one or more scanning parameters from the first imaging scan, for the performance of a second imaging scan, based on this analysis.
• the quality-control process discussed with reference to Figure 3 will generally be repeated until the imaging data produced by the preceding imaging scan is determined to be acceptable or a threshold number of iterations has been reached.
• any of these processes may be performed alone or in any combination before the second imaging scan is initiated. Multiple changes to scanning parameters may therefore be identified prior to commencing the second imaging scan in order to further reduce the overall time elapsed between starting the first imaging scan and outputting an image of the target that is suitable for analysis by the user.
• an improved method for operating a microscope scanner including a method for analysing the data generated by an imaging scan so as to ensure the target has been properly imaged.
• the analysis is advantageously performed using focus control data generated during the imaging scan. This enables a result to be obtained in a quick and efficient manner. If any errors are identified in the first imaging scan as a result of this analysis then a change to a scanning parameter is identified so that a second imaging scan may be performed that corrects this error.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Computer Vision & Pattern Recognition (AREA)
• General Physics & Mathematics (AREA)
• Chemical & Material Sciences (AREA)
• Analytical Chemistry (AREA)
• Optics & Photonics (AREA)
• Multimedia (AREA)
• Theoretical Computer Science (AREA)
• Radiology & Medical Imaging (AREA)
• Quality & Reliability (AREA)
• Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
• Medical Informatics (AREA)
• General Health & Medical Sciences (AREA)
• Health & Medical Sciences (AREA)
• General Engineering & Computer Science (AREA)
• Microscoopes, Condenser (AREA)

Priority Applications (4)

Applications Claiming Priority (2)

Publications (1)

Family

ID=71616083

Family Applications (1)

Country Status (6)

Families Citing this family (3)

Citations (6)

Family Cites Families (9)

• 2020
  • 2020-06-09 GB GB2008690.6A patent/GB2595873B/en active Active
• 2021
  • 2021-05-26 JP JP2022573304A patent/JP7655947B2/ja active Active
  • 2021-05-26 WO PCT/GB2021/051271 patent/WO2021250373A1/en not_active Ceased
  • 2021-05-26 CN CN202180041233.4A patent/CN115836241B/zh active Active
  • 2021-05-26 US US18/009,152 patent/US12422658B2/en active Active
  • 2021-05-26 EP EP21731566.2A patent/EP4107570B1/en active Active

Patent Citations (6)

Non-Patent Citations (7)

Also Published As

Similar Documents

Legal Events