EP3549057A1

EP3549057A1 - Device and method for identifying a region of interest (roi)

Info

Publication number: EP3549057A1
Application number: EP17808913.2A
Authority: EP
Inventors: Fei Zuo; Anke Pierik; Reinhold Wimberger-Friedl; Koen DE LAAT
Original assignee: Koninklijke Philips NV
Current assignee: Koninklijke Philips NV
Priority date: 2016-12-05
Filing date: 2017-12-05
Publication date: 2019-10-09
Also published as: WO2018104254A1

Abstract

A device for identifying a region of interest to be dissected from a biological tissue sample is presented. The device comprises an image analyzing unit which iteratively grows the region of interest from an initial size to a final size as long as a target parameter which is based on the two detected densities D₁ and D₂ and which describes the region of interest, is above a predefined target value. Preferably, the target parameter is the density ratio D₁/D₂ of the region of interest and the first type of detected cells are tumor cells whereas the second type of cells are non-tumor cells. Furthermore, optimization or cost functions can be applied which take into account the percentage of number of cells of the first cell type and the total number of cells of the first cell type in the respective region of interest.

Description

DEVICE AND METHOD FOR IDENTIFYING A REGION OF INTEREST (ROI)

FIELD OF THE INVENTION

The present invention relates to automatic image analysis. In particular, the present invention relates to a device for identifying a region of interest (ROI) to be dissected from a biological tissue sample, and relates to a digital pathology and molecular diagnostic system for oncology, a method for identifying a ROI to be dissected from a biological tissue sample, a program element and a computer-readable medium.

BACKGROUND OF THE INVENTION

Molecular tests are becoming increasingly important in cancer diagnostics. These tests can be based on detecting genetic mutations in the cancer cells and/or aberrant gene expression. For the sensitivity and specificity of those tests it is important to have enough tumor cells and a high enough fraction of tumor cells in the sample, respectively. Therefore, it is common practice to select parts of a tumor sample that fulfills the

requirements of the test in question. Currently, this is done manually in two steps: (i) the pathologist investigates the sample on a slide under a microscope and indicates with a pen on the slide which area needs to be selected for the test; (ii) a lab technician makes a new section of the same sample and dissects the indicated part by comparing the new section with the slide that has been annotated by the pathologist. The annotation of the region of interest (ROI) is typically done on a Hematoxylin and Eosin (H&E) -stained tissue sample. The new section for sample dissection can be paraffin embedded or deparaffinized and optionally

Hematoxylin stained, depending on the preference of the lab. The dissection is generally done by scraping the material from the glass slide and putting it into a vial. Also alternative techniques have been proposed. A very accurate dissection technology like Laser Capture Microdissection can be used to select very small groups of tumor cells, or removing small islands of non-tumor cells within larger areas of tumor cells. In this way, a relatively pure sample is obtained. Manually scraping the tumor cells from the microscope slide is relatively inaccurate, so it will generally lead to lower purity samples. However, due to the relatively simple and low-cost procedure, this method is used the most often. It has been shown that the quantitative estimation of tumor cell numbers and purity is difficult for pathologists. Therefore, the use of computer algorithms on digitized images (Digital Pathology) has been suggested. For example, WO 2014181123 or

WO 2014184522A3 propose to compute a two state map identifying tumor and non-tumor regions, to provide a spatial map of the image data which classifies regions in one of three categories (background, tumor tissue, and non-tumor tissue). Then the processor computes a percentage number of tumor cells based on tumor and the non-tumor areas. This percentage may be taken into account for defining an ROI for later extraction. After ROI annotation, the ROI is being removed via dissection and analyzed in the medical diagnostic (MDX) test. Different dissection processes exist, each with its accuracy. In these two patent applications, a smoothing algorithm has been mentioned to smoothen the boundary of selected ROI for easy cutting along the boundary. The smoothing algorithm is based on frequency domain filtering and/or a smoothing operation on the sequence of transition indicators along the ROI boundary.

WO 2015150392 also describes a method for selecting a ROI for later extraction. In a first ROI (computed or defined by the pathologist and considered as reference ROI) the processor detects at least two types of cells in a slide image, one cell type for positive selection (i.e. tumor cells) and one cell type for negative selection (e.g. lymphocytes) by means of two different cell nucleus detectors. Characteristics of the objects in the ROI are extracted, for example density in the ROI classified as tumor cells. A purity score, e.g. the percentage of tumor cells or lymphocytes is computed. Alternative possible ROIs are determined by analyzing in the image alternative regions with a similar characteristic.

SUMMARY OF THE INVENTION

There is thus the need for an improved ROI identification in digital pathology.

The object of the present invention is solved by the subject-matter of the independent claims. Further embodiments and advantages of the invention are incorporated in the dependent claims.

The described embodiments similarly pertain to the device for identifying a ROI to be dissected from a biological tissue sample, to the digital pathology and molecular diagnostic system for oncology, to the method for identifying a ROI to be dissected from a biological tissue sample, to the program element and to the computer-readable medium. Synergetic effects may arise from different combinations of the embodiments although they might not be described in detail.

Technical terms are used by their common sense. If a specific meaning is conveyed to certain terms, definitions of terms will be given in the following in the context of which the terms are used.

According to a first aspect of the present invention, a device for identifying a region of interest to be dissected in/from a biological tissue sample is presented. The device comprises an image analyzing unit. The image analyzing unit is configured for receiving a slide image of the biological tissue sample and is configured for analyzing the slide image with at least two different cell nucleus detectors to detect at least two types of cells in the slide image. Furthermore, the image analyzing unit is configured for creating a first density map of a density Di of the first type of detected cells. And the image analyzing unit is configured for creating a second density map of a density D₂ of the second type of detected cells. The image analyzing unit is further configured for defining a region of interest (ROI) with an initial size. The ROI with the initial size will also be referred to hereinafter as the initial ROI. The image analyzing unit is further configured for iteratively growing the ROI from the initial size to a final size as long as a target parameter, which is based on the two detected densities Di and D₂ within the ROI and which target parameter describes the region of interest, is above a predefined target value.

In other words, an adaptive sample selection strategy for molecular diagnostics based on automatic image analysis of key areas for inclusion and exclusion can be presented. This will be explained in more details hereinafter.

In general, the device presented herein is able to provide for an optimal determination of the ROI which has a sample purity that meets the required level for the later processing.

With such a device, an automatic image analysis is facilitated which derives key areas from whole slide histopathology images which are used to guide the following dissection procedure for molecular diagnostics. In particular, a region growing algorithm can be used in the device suggested here to guide the dissection region creation, favoring high density areas of a specific cell type, for example tumor cells, and staying away from high density interfering area such as, for example lymphocytes. Candidate regions can be obtained during the growing iterations and the best candidate can be selected based on a predefined criterion by the device. Adaptive strategies can be further applied to customize the shape of the region of interest to a specific device, as will be explained hereinafter in more detail for specific embodiments. The results of the identification carried out by the device of the present invention can also be applied for optimally choosing the dissection device. Also this will be explained in more detail in the following.

In other words, the device is configured for automatic ROI identification since the device automatically identifies the ROI to be dissected. The corresponding result of the automatic ROI identification can then be communicated, shown and/or displayed to, for example, the pathologist and he may check the result and transfer and/or communicate it to a lab for starting the dissection.

In general, the image analyzing unit of the present invention may be part of a digital pathology and molecular diagnostic system for oncology. The image analyzing unit may be embodied as, for example, a processor or a calculation unit of a computer or a server on which the ROI growing process described herein is carried out.

A particular application for the device presented herein is the identification of tumor cells from the biological tissue sample, where the tumor cells shall be dissected from the pathological sample. Hence, the identification of tumor cells in the slide image is a preferred embodiment of the present invention as will be explained in the following.

Consequently, the first type of cells detected by the first cell nuclei detector of the image analyzing unit may be tumor cells whereas the second type of cells detected by the second cell nuclei detector of the image analyzing unit may be non-tumor cells, like for example tumor-associated immune cells, lymphocytes, stroma cells, fibroblasts, etc. Specific embodiments thereof will be addressed hereinafter.

Furthermore, in general, the device and the method presented herein may be based on image analysis algorithms that generally operate on stained images, particularly H&E images.

In general, the skilled person knows how to realize or implement a cell detector or cell nucleus detector to derive a density map from a real cell count as used in the present invention. In particular, a cell detector or cell nucleus detector as used in the context of the present invention is a pattern recognition device, program element or algorithm and can be based on machine learning, where a defined set of annotated, ground-truth cell images are used for training. The training images can be fed into a machine-learning based classifier such as Adaboost. The resulting classifier can be used as a detector to scan the whole slide image to detect the relevant cells or cell nuclei. In terms of a tumor cell detector, typical image features like tumor nuclear shape is recognized, using morphological filters, radial symmetry filters, etc. For the design of a lymphocyte detector one can use features that capture round, dark objects such as blob features, circular Hough transforms, and also the morphological features. One can also use deep neural networks as detectors which are capable of identifying discriminative features automatically based on a set of ground truth image data.

In other word, this type of cell nucleus detectors allows creating two density maps of two different cell types which can be used together to determine the best dissection ROI. Hence, the device according to the present invention simultaneously promotes one type of tissue region while it suppresses another type of tissue region during the growing of ROI.

The density map used by the present invention is a map in which a cell density distribution in the mapped area of the sample is provided. It may have a graphical format but may also be only present as a calculative result in form of data. In particular, the density may be a two-dimensional density, because it is calculated in most cases from a two-dimensional image. However, such two-dimensional density correlates with a real three-dimensional density which again correlates to the amount of cells contained in a particular area of the sample.

In other words, the density maps of the present invention are based on the real cell counting, thus, they are different from the probability maps as applied in the state of the art.

For the skilled person a density map implies a certain resolution. Also one can apply smoothing algorithms to the density distributions for an optimum performance. As an example, resolutions could be of the order of 10, 100, or 1000 microns.

Based on the determined densities of the first and second types of cells, the image analyzing unit defines a ROI in the slide, the so-called initial ROI. This can be understood as an automatic initialization based on different possible criteria. Such different criteria will be explained in the following hereinafter for specific embodiments. However, the definition of the initial ROI may also be done by means of a user input, which is then processed by the image analyzing unit and the growing process is then carried out by the image analyzing unit.

The initial ROI is then used as a starting point for the iterative growing process. In other words, the initial ROI is grown in a first step to a first grown ROI, in a second step to a second grown ROI, in a third step to a third grown ROI and so on. This can be done until a final size of the ROI is reached. For each intermediate ROI, like in this example, the first grown, the second grown and third grown ROIs, the image analyzing unit may calculate the respective value of the target parameter for each of said ROIs. It may then compare it with the predefined target value to automatically decide - based on the result of said comparison - whether the iterative growing process shall proceed. The predefined target value may thus be understood as a predefined threshold of the target parameter.

The iterative growing process carried out by the device, in particular by the image analyzing unit, can be done such that for each incremental growing step, the ROI can be calculated and compared with the predefined target value. The finally grown ROI with the final size is then the identified ROI which shall then be used for the dissection. As will be explained in the context of different embodiments, the grown ROI with the final size can be further adapted or evaluated.

The region growing may incorporate any additional shape constraints, for example to facilitate a preferred dissection method, e.g. to always grow a rectangular region for easy manual scrapping. Such shape constraints may be provided by a user input or may be defined automatically by the device itself. This will be explained in more detail hereinafter.

Different target parameters, which are based on the two detected densities Di and D₂ within the ROI and which describes the ROI can be used. For example, the ratio Di/D₂ can be used. The ratio of the densities Di/D₂ is only checking for purity but not for numbers, so other target parameters in which also the area of the ROI plays a role can be used as an alternative or in combination with the ratio of the densities Di/D₂.

In other words, the two detected densities Di and D₂ are numerical value, and the target parameter has a quantitative nature being applied in the iterative growing algorithm of the ROI.

For example, an optimization function can be used, which uses e.g. the percentage of the number of cells of the first cell type and the total number of cells of the first type of the respective region of interest as input variables of the function and thus as target parameters. In this case the image analyzing unit is configured for calculating the percentage of the first cells and the total number of first cells. An explicit embodiment thereof will be explained in detail hereinafter. However, also other variations of the target parameters are possible.

In other words, the growing of the ROI is carried out in iterative steps as long as the latest ROI of the iterative ROI growing process still fulfills the requirement which was previously set for the target parameter. Thus, the growing process is carried out as long as the predefined target parameter does not fall under a specific predefined value. For example, this can be continuously checked by the image analyzing unit by calculating the respective value of the target parameter for at least some or even for each incrementally grown ROIs. In particular, the iteratively growing as defined herein may grow the initial ROI in all directions as long as the predefined criterion described herein is met.

In other words, the image analyzing unit ensures an iterative growing of the ROI until a predefined condition associated to the detected densities Di and D₂ is satisfied.

It should be noted that in an embodiment also a plurality of initial ROIs may be defined which are grown separately and in parallel by the image analyzing unit. Thus, at least one initial ROI is used, but also more than one initial ROI can be initialized. In this it is clear for the skilled person that two or more ROIs are grown simultaneously while all fulfilling the set requirement, e.g. the requirement for D1/D2.

According to another exemplary embodiment of the present invention, the target parameter is the density ratio Di/D₂ of the region of interest.

Therefore, the density ratio Di/D₂ of some or all incremental ROIs is calculated during the growing process by the image analyzing unit.

In another, further specified embodiment, the first type of detected cells are tumor cells and the second type of cells are non-tumor cells, preferably lymphocytes.

When combining the before mentioned two embodiments, a novel way of defining an optimal ROI to be distracted for being processed in a molecular analysis is presented. The device which identifies the final ROI favors higher density tumor content while ensuring a sufficiently large tumor area to secure enough DNA content. At the same time, the device can generate a ROI size and/or shape that excludes as much as possible the high density non-tumor regions, to minimize the risk of polluting the tumor content.

The presented device allows to exclude a certain area from the sample that will interfere mostly with the sample purity. Lymphocytes are one type of cells that are easier to detect via the means of image analysis. They are easily identified from H&E images (or H- only) images. Typically, they are densely clustered, resulting in a high DNA concentration and the cell count within a very small area can be very high. The DNA density per cell is not higher, only their total size is smaller so the effective concentration can become higher. This is particularly true for some tissue types such as colon samples, where the immune cells, lymphocytes, are present in large quantities.

According to another exemplary embodiment of the present invention, the image analyzing unit is configured for detecting an area in the slide image which has the highest density of the first cell type. The image analyzing unit is further configured for defining the area with the highest density of the first cell type as the ROI with the initial size.

The iterative growing may then start from the area with the highest density of the first cell type. In an embodiment, the device starts with determining the highest density tumor area and uses this area as the initial ROI for starting the growing process.

According to another exemplary embodiment of the present invention, the image analyzing unit is configured for detecting an area in the slide image which has the maximum density of the second cell type. Furthermore, the image analyzing unit is configured for iteratively growing the ROI as long as a distance of the ROI to the area with the maximum density of the second cell type does not fall below a predefined distance.

In other words, the image analyzing unit may calculate in parallel to the growing procedure how large the distance between the current ROI to the region with the highest density of the second cell type is. In an example, the region boundary of the ROI should have at least a certain distance from the nearest high density lymphocyte area. This distance can depend on the preferred dissection method.

In general, the ROI may have a boundary or is defined by a boundary. The image analyzing unit may compute the distance to the maximum density of the second cell type, wherein the distance is defined as the length of the shortest possible path from any part of the boundary of the ROI to the maximum density area of the second cells.

According to another exemplary embodiment of the present invention, the image analyzing unit is configured for ensuring that the ROI with the initial size complies with a predefined minimum size condition for the ROI.

Preferably, the condition that the ROI with the initial size must contain at least the predefined number of cells is used. In other words, a minimum size condition is used, for example that the initial ROI must contain at least 10, 20, 100 or 300 cells.

According to another exemplary embodiment of the present invention, the image analyzing unit is configured for receiving information which defines a preferred dissection method with which the biological tissue sample shall be dissected. The image analyzing unit is further configured for adapting a shape of the ROI based on the received information which defines the preferred dissection method.

For example, the initial ROI can be initialized like this and the shape is also kept during the iteratively growing step. In another further specified embodiment thereof, a post-processing step is applied by adapting the shape of the ROI after the ROI has been determined based on the iterative growing process.

According to another exemplary embodiment of the present invention, the image analyzing unit is configured for receiving information regarding a predefined minimum cell number which is required by a specific molecular diagnostic test. The image analyzing unit is configured for determining whether the total number of cells of the first cell type in the ROI with the final size is below the predefined minimum cell number or area size. The imaging analyzing unit is configured for defining a second ROI with an initial size and is configured for iteratively growing the second ROI from the initial size to a final size as long as the target parameter, which is based on the two detected densities Di and D₂ within the second ROI and which describes the ROI, is above the predefined target value, if the image analyzing unit has determined before that the total number of cells of the first cell type is below the predefined minimum cell number for the first ROI.In other words, the ROI growing procedure is applied to a first ROI. And if necessary, as defined by the criterion regarding the total number of cells of the first cell type in the ROI with the final size, a second initial ROI is defined and the ROI growing procedure is repeated with this different, second initial ROI. If the grown ROI does not fulfill the requirement of the molecular diagnostic test, a second region can be initiated by applying the same region growing strategy, starting from the second highest density center. The target parameter used for the determination of the final size of the second ROI is same as the target parameter used for the determination of the final size of the first ROI.

In other words, the image analyzing unit may be configured for detecting an area in the slide image which has a second highest density of the first cell type. The image analyzing unit is then further configured for defining the area with the second highest density of the first cell type as the second ROI with the initial size.

In an embodiment of the present invention, the device is configured for setting the region of interest with the final size.

However, it also be explained hereinbefore, that a plurality of ROIs can be identified. Thus, in an embodiment, if there are two or more comparable ROI's on the same slide, these two or more ROIs are identified and can be selected for dissection.

Thus, at least one initial ROI is used, but also more than one initial ROI can be initialized. In this it is clear for the skilled person that two or more ROIs are grown simultaneously while all fulfilling the set requirement, e.g. the requirement for D1/D2. According to another exemplary embodiment of the present invention, the procedure of iteratively growing is carried out by the image analyzing unit using the first and second density maps such that one type of tissue region which predominantly contains cells of the first cell type is promoted during the growing while suppressing another type of tissue region which predominantly contains cells of the second cell type.

This embodiment can be explained very well for e.g. the constellation that the first cell type are tumor cells and the second type cells are lymphocytes. In this way, a device is presented which in an optimal way selects the best dissection area by avoiding high density lymphocyte areas via region growing. The two density maps are used together to determine the best dissection ROI. Hence, a simultaneous promotion of one type of tissue region while suppressing another type of tissue region is facilitated. Clearly, the presented device chooses automatically the best ROI.

In another exemplary embodiment, the image analyzing unit can identify the best region for dissection based on selecting the ROI which gives the highest value for an optimization function favoring both high density and large number of cells of the first type. Specific examples of such possible optimization functions will be explained in detail hereinafter. For this purpose, in a specific embodiment, the image analyzing unit is configured for storing each ROI of each iteration of the iteratively growing step. Furthermore, the image analyzing unit is configured for determining for each stored ROI of each iteration of the iteratively growing a percentage of the number of cells of the first type in said ROI and the total number of cells of the first type in said ROI, wherein the image analyzing unit is configured for calculating for each stored ROI a value of an optimization function using the percentage of the number of cells of the first cell type and the total number of cells of the first type of the respective ROI as input variables for the optimization function. The image analyzing unit is further configured for identifying the ROI which has a highest value of the optimization function as the ROI to be dissected.

According to another exemplary embodiment, the second type of cells are tumor-associated immune cells and the image analyzing unit is configured for detecting an area in the slide image which has a highest density of tumor-associated immune cells. The image analyzing unit is configured for automatically defining the area with the highest density of tumor-associated immune cells as the ROI with the initial size.

This exemplary embodiment may be used in combination with a printing device. In the case of using a printing device for the creation of a barrier layer for extraction of the ROI, the exclusion area needs to be printed on. Therefore, an alternative grow algorithm can be utilized starting from the high density tumor-associated immune cells. In this case, the device selects the region from the highest density tumor-associated immune cells area and grows this region. Depending on the device properties, the minimum size of the region can be determined based on the resolution of the printing device. Different masks can be generated which can be suited for a more accurate printing device, a medium accurate printing device and a less accurate. It should be noted in general that the term "tumor- associated immune cells" is understood by the skilled person in the art as a type of cells that comprises tumor-infiltrating immune cells, and as an example for such immune cells lymphocytes may be mentioned.

In general, when selecting an ROI one can also include areas outside the tumor which contain non-tumor cells and also immune cells.

According to another exemplary aspect of the present invention, a digital pathology and molecular diagnostics system for oncology is presented. The system comprises a device for defining the ROI to be dissected from a biological tissue sample as described herein. Furthermore, the system comprises a dissection unit, which is configured for receiving the biological tissue sample. The device is further configured for transferring information regarding the defined region of interest to be dissected to the dissection unit which then dissects the defined region of interest from the received biological tissue sample based on the received information.

This system for oncology may comprise several different components like the device presented herein and the dissection unit. In addition, it may also comprise in another embodiment the molecular diagnostic test unit which carries out the respective diagnostic test with the dissected sample part. The component of the oncology system may be connected wire-bound or wireless and may be distributed over several locations in e.g. a lab or in a hospital.

According to another aspect of the present invention, a method for identifying a ROI to be dissected from a biological tissue sample is presented. The method comprises the steps of receiving a slide image of the biological tissue sample, analyzing the slide image with at least two different cell nucleus detectors and detecting at least two types of cells in the slide image. In another step, a first density map of a density Di of the first type of detected cells is created and a second density map of a density D₂ of the second type of detected cells is created. Moreover, an ROI with an initial size is defined in the slide image and the ROI is iteratively grown from the initial size to a final size as long as a target parameter, which is based on the two detected densities Di and D₂ within the ROl and which describes the ROl, is above a predefined target value.

It must be noted, that the herein presented method and also the program element may be carried out by a computer, the device comprising the image analyzing unit described herein, but may also be carried out by using a cloud solution in which parts of the method are processed or carried out on a device at first location and parts or other steps of the method are carried out on a remote server or another computer at a second location.

A corresponding program element for identifying a ROl is presented. The computer program element may be part of a computer program, but it can also be an entire program by itself. For example, the computer program element may be used to update an already existing computer program to get to the present invention.

Furthermore, a computer-readable medium, in which a computer program for identifying a ROl to be dissected from a biological tissue sample is stored, is presented. The computer readable medium may be seen as a storage medium, such as for example, a USB stick, a CD, a DVD, a data storage device, a hard disk, or any other medium on which a program element as described above can be stored.

The method for identifying a ROl to be dissected from a biological tissue sample is explained in the following in the context of an exemplary but general embodiment.

a) Using at least two different cell nucleus detectors to detect at least two types of cells in a slide image, one cell type for positive selection (i.e. tumor cells) and one cell type for negative selection (e.g. lymphocytes).

b) Creating density maps (heat maps) for each type of cell nuclei. For example, D_t for tumor density and Di for lymphocyte density based on the results of the previous step.

c) A region-growing algorithm is applied to select an optimal area for dissection. It can start with determining the highest density tumor area At. A minimum size condition is obeyed (for instance, the area must contain at least 100 cells). Multiple areas are allowed.

d) The identified area(s) is then grown iteratively in all directions as long as the following criterion is met: The region will grow as long as the density ratio Dt / Di is higher than a given target value and the target value can be chosen depending on the test requirements, (e.g. D_t / Di >1)).

To further improve the identification, more test requirements may be considered. The region boundary should have at least a certain distance from the nearest high- density lymphocyte area. This distance can depend on the preferred dissection method. The region growing can incorporate any shape constraints, for example to facilitate a preferred dissection method, e.g., to always grow a rectangular region for easy manual scraping. Each new shape is then analyzed in terms of the total number of cells to compare with the requirement. The growth process can be stopped once Nt > NT, where Nt is an optimization function. Alternatively, the growth process can be continued following the criteria described before under item d) to generate a large number of areas and then select the area with the highest Nt or an alternative function that is maximized. If the grown region does not fulfill the requirement for NT of the MDx tests, a second region can be initialized by applying the same region-growing strategy, starting from the second highest density tumor center.

Additional segmentation strategies can be applied to the region growing algorithm to adapt to the device used for removing the ROI from the whole slide. For example, one can apply boundary smoothing and size filtering based the specification of the device and desired accuracy. Design of the dissection strategy (selection of dissection method) can be also adapted according to the above analysis. Depending on the sample characteristics, low accuracy dissection methods might not be suitable. A preferred dissection method can be suggested. Furthermore, specific strategies can be applied to improve the sample selection process.

It may be seen as a gist of the invention to provide for a new device and method for defining an optimal ROI to be extracted for being processed in a molecular analysis. The method favours higher density tumor content while being able to ensure a sufficient large tumor area to secure enough DNA content. At the same time, the device and method can generate a ROI shape that excludes as much as possible the high density non- tumor regions, to minimize the risk of polluting the tumor content. The device and method can be based on image analysis algorithms that generally operate on H&E-stained images.

These and other features of the invention will become apparent from and elucidated with reference to the embodiments described hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

Fig. 1 schematically shows a device for identifying a ROI to be dissected from a biological tissue sample according to an exemplary embodiment of the present invention.

Fig. 2 shows a flow diagram of a method for identifying a ROI to be dissected from a biological tissue sample according to an exemplary embodiment of the present invention.

Fig. 3 shows histopathological images in which lymphocytes, tumor cells and epithelial cells forming a gland are shown.

Fig. 4 shows histopathological images and density maps of two different cell nucleus detectors used in an embodiment according to the present invention.

Fig. 5 shows an example of an optimal dissection area selection for molecular diagnostics according to an exemplary embodiment of the present invention.

Fig. 6a to Fig. 6c schematically show boundary adaption of the identified ROI for different dissection methods according to another exemplary embodiment of the present invention.

DETAILED DESCRIPTION OF EMBODIMENTS

In Fig. 1 , a digital pathology and molecular diagnostic system for oncology 100 is presented. The system 100 comprises a device 101 for defining the ROI to be dissected from a biological tissue sample 105. The device 101 comprises an image analyzing unit 102, which can be embodied for example as a processor or calculation unit. The image analyzing unit 102 is configured for analyzing the slide image 103 with at least two different cell nucleus detectors to detect at least two types of cells in the slide image. The image analyzing unit is further configured for creating a first and second density map for the two types of cells. The image analyzing unit is further configured for defining an initial ROI with an initial size. The image analyzing unit is further configured for iteratively growing the initial ROI from the initial size to a final size as long as a target parameter which is based on the two detected densities Di and D₂ within the ROI and which describes the ROI is above a predefined target value. Alternatively, the device 101 may be configured for setting the final size of ROI.

The final ROI with the final size can then be communicated to the dissection unit 106 which then dissects the defined ROI from the received biological tissue sample 105. Furthermore, the device 101 can communicate the ROI which has been identified by using the procedure described herein to a user or computer 104. The dissected part may be processed further into a vessel 107 in which a molecular diagnostic test by means of apparatus 108 is carried out. Generally, a clean-up process is done to purify the RNA or DNA before amplification and detection, depending on the MDX test. The result thereof can also be transferred to the user or computer 104.

As has been explained herein before, the device and method of the present invention can be used for various different types of cells. However, for the ease of application, in the following, different embodiments will be explained with respect to tumor cells being the first type of cell and lymphocytes being the second type of cells. However, also other tumor-associated immune cells can be used instead of lymphocytes.

According to another exemplary embodiment of the present invention, a method for identifying a ROI to be dissected from a biological tissue sample is presented which comprises steps S I to S6. In step S I , the slide image of the biological tissue sample is received and the slide image is analyzed with at least two different cell nucleus detectors in step S2. Furthermore, creating a first density map of a density Di of the first type of detected cells is done in step S3 followed by or in parallel to creating a second density map of a density D₂ of the second type of detected cells S4. Moreover, a ROI with an initial size in the slide image is defined in step S5. Furthermore, the ROI is iteratively grown from the initial size to a final size as long as a target parameter, which is based on the two detected densities Di and D₂ within the ROI and which describes the ROI, is above a predefined target value. This iterative growing process is shown in Fig. 2 with reference numeral S6. In particular, in the case of a biological tissue sample containing tumor cells, the presented method provides four particular advantages. A new method for defining an optimal ROI is disclosed which shall be extracted for being processed in a molecular analysis. The method favors higher density tumor content while ensuring a sufficient large tumor area to secure enough DNA content. At the same time, the method will generate a ROI shape that excludes as much as possible the high density non-tumor region, to minimize the risk of polluting the tumor content. The method can be based on image analysis algorithms that generally operate on stained images, particularly H & E stained images. Further combinations of specific embodiments will be explained in the context of and elucidated with the following embodiment.

Fig. 3 shows two histopathological colon images 300 are shown. The left image shows an adenoma, with tumor-infiltrating lymphocytes. The right image shows normal tissue, with lymphocyte clusters and between the glands. In the left image, lymphocytes 301 can clearly be differentiated from tumor cells 302. In the right image, lymphocytes 303 can clearly be distinguished from epithelial cells forming a gland.

Fig. 4a schematically shows a histopathological image 400 and a corresponding density map 401. In image 400, the result of tumor cell detection from the first nucleus or cell detector is shown. The region 402 is the region where a large density of tumor cells was detected. The corresponding density map of tumor cells 401 shows an area 403 of a high tumor density and in contrast thereto another area 404 in which no or nearly no tumor cells were detected by the cell nuclei detector. Furthermore, in Fig. 4b, lymphocyte detection with the second nucleus or cell detector was carried out in image 405 and the region 407 shows a high density of detected lymphocytes. The corresponding density map of

lymphocytes 406 shows an area 408 with a high density of lymphocytes and a lower density area 409.

Fig. 5 schematically shows an example of optimal dissection area selection for molecular diagnostic tests. In image 500, the optimal dissection area 501 is the end result of the initial ROI 502.

Figs. 6a to 6c illustratively show that various post-processing strategies can be applied to adapt the results to the capabilities of the dissection device. For example, one can adapt the curvature, smoothness, of the mask boundary to the device properties that shall be used for dissection. The boundary smoothness can be also used as a parameter to control the dissection performance. For example, the image analyzing unit of the device of the present invention may be configured for receiving information which defines the preferred dissection method with which the biological tissue sample shall be dissected. The image analyzing unit may thus be configured for adapting the shape of the ROI based on the received information which defines the preferred dissection method. In Fig. 6a for example, image 601 shows a ROI defined in the case that a high resolution device can be used for a dissection. In contrast thereto, Fig. 6b shows an image 602 in which a smoother dissection boundary is used in case a dissection device is applied which favors such smoother boundaries. Moreover, Fig. 6c shows an image 603 in which a manual selection shall be applied which favors rectangular region shapes.

The device and method presented herein provides a way to optimally select the best dissection area by avoiding high-density of the second cell type, e.g. lymphocytes, via region-growing. Further, the two density maps are used together to determine the best dissection ROI. Thus one key element here is to simultaneously promote one type of tissue region while suppressing another type of tissue region. The method and device of the present invention provide a calculation method which can be computed implemented such that the optimal ROI which is used for dissection subsequently automatically determined. Further on, it shall be noted that all embodiments of the present invention concerning a method, might be carried out with the order of the steps as described, nevertheless this has not to be the only and essential order of the steps of the method. The herein presented methods can be carried out with another order of the disclosed steps without departing from the respective method embodiment, unless explicitly mentioned to the contrary hereinafter.

Where an indefinite or definite article is used when referring to a singular noun, e.g. "a", "an" or "the", this includes a plurality of that noun unless something else is specifically stated. The terms "about" or "approximately" in the context of the present invention denote an interval of accuracy that the person skilled in the art will understand to still ensure the technical effect of the feature in question. The term typically indicates deviation from the indicated numerical value of ±20 %, preferably ±15 %, more preferably ±10 %, and even more preferably ±5 %.

Furthermore, the terms "first", "second", "third" or "(a)", "(b)", "(c)", "(d)" or "(i)", "(ii)", "(iii)", "(iv)" etc. and the like in the description and in the claims, are used for distinguishing between similar elements and not necessarily for describing a sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances and that the embodiments of the invention described herein are capable of operation in other sequences than described or illustrated herein.

Claims

CLAIMS:

1. Device (101) for identifying a region of interest to be dissected from a biological tissue sample (105), the device comprising

an image analyzing unit (102),

wherein the image analyzing unit is configured for receiving a slide image (103) of the biological tissue sample,

wherein the image analyzing unit is configured for analyzing the slide image with at least two different cell nucleus detectors to detect at least two types of cells in the slide image,

wherein the image analyzing unit is configured for creating a first density map of a density Di of the first type of detected cells, and is configured for creating a second density map of a density D₂ of the second type of detected cells,

wherein the image analyzing unit is configured for defining a region of interest (ROI) with an initial size, and

wherein the image analyzing unit is configured for iteratively growing the region of interest from the initial size to a final size as long as a target parameter, which is based on the two detected densities Di and D₂ within the region of interest and which describes the region of interest, is above a predefined target value.

2. Device according to claim 1,

wherein the target parameter is the density ratio Di/D₂ of the region of interest.

3. Device according to claim 1 or 2,

wherein the first type of detected cells are tumor cells, and

wherein the second type of cells are non-tumor cells, preferably lymphocytes.

4. Device according to any of the preceding claims,

wherein the image analyzing unit is configured for detecting an area in the slide image which has a highest density of the first cell type, and wherein the image analyzing unit is configured for defining the area with the highest density of the first cell type as the region of interest with the initial size.

5. Device according to any of the preceding claims,

wherein the image analyzing unit is configured for detecting an area in the slide image which has a maximum density of the second cell type, and

wherein the image analyzing unit is configured for iteratively growing the region of interest as long as a distance of the region of interest to the area with the maximum density of the second cell type does not fall below a predefined distance.

6. Device according to any of the preceding claims,

wherein the image analyzing unit is configured for ensuring that the region of interest with the initial size complies with a predefined minimum size condition for the region of interest, preferably the condition that the region of interest with the initial size must contain at least the predefined number of cells.

7. Device according to any of the preceding claims,

wherein the image analyzing unit is configured for receiving information which defines a preferred dissection method with which the biological tissue sample shall be dissected, and

wherein the image analyzing unit is configured for adapting a shape of the region of interest based on the received information which defines the preferred dissection method.

8. Device according to any of the preceding claims,

wherein the image analyzing unit is configured for receiving information regarding a predefined minimum cell number which is required by a specific molecular diagnostic test,

wherein the image analyzing unit is configured for determining whether the total number of cells of the first cell type in the region of interest with the final size is below the predefined minimum cell number, and

wherein the imaging analyzing unit is configured for defining a second region of interest with an initial size, and is configured for iteratively growing the second region of interest from the initial size to a final size as long as the target parameter, which is based on the two detected densities Di and D₂ within the second region of interest and which describes the second region of interest, is above the predefined target value, if the image analyzing unit has determined that the total number of cells of the first cell type is below the predefined minimum cell number.

9. Device according to claim 8,

wherein the image analyzing unit is configured for detecting an area in the slide image which has a second highest density of the first cell type, and

wherein the image analyzing unit is configured for defining the area with the second highest density of the first cell type as the second region of interest with the initial size.

10. Device according to any of the preceding claims,

wherein the iteratively growing is carried out by the image analyzing unit using the first and second density maps such that one type of tissue region which

predominantly contains cells of the first cell type is promoted during the growing while suppressing another type of tissue region which predominantly contains cells of the second cell type.

1 1. Device according to any of the preceding claims,

wherein the image analyzing unit is configured for storing each region of interest of each iteration of the iteratively growing step,

wherein the image analyzing unit is configured for determining for each stored region of interest a percentage of the number of cells of the first type in said region of interest and the total number of cells of the first type in said region of interest,

wherein the image analyzing unit is configured for calculating for each stored region of interest a value of an optimization function using the percentage of the number of cells of the first cell type and the total number of cells of the first type of the respective region of interest as input variables for the optimization function, and

wherein the image analyzing unit is configured for identifying the region of interest which has a highest value of the optimization function as the region of interest to be dissected.

12. Device according to claim 1 or 2,

wherein the second type of cells are tumor-associated immune cells, wherein the image analyzing unit is configured for detecting an area in the slide image which has a highest density of tumor-associated immune cells, and

wherein the image analyzing unit is configured for defining the area with the highest density of tumor-associated immune cells as the region of interest with the initial size.

13. Digital pathology and molecular diagnostics system for oncology ( 100), the system comprising

a device (101) for defining the region of interest to be dissected from a biological tissue sample (105) according to any of the preceding claims,

a dissection unit (106),

wherein the dissection unit is configured for receiving the biological tissue sample,

wherein the device for defining the region of interest to be dissected from a biological tissue sample is configured for transferring information regarding the defined region of interest to be dissected to the dissection unit, and

wherein the dissecting unit is configured for dissecting the defined region of interest from the received biological tissue sample.

14. Method for identifying a region of interest to be dissected from a biological tissue sample, the method comprising

receiving a slide image of the biological tissue sample (SI),

analyzing the slide image with at least two different cell nucleus detectors and detecting at least two types of cells in the slide image (S2),

creating a first density map of a density Di of the first type of detected cells

(S3),

creating a second density map of a density D₂ of the second type of detected cells (S4),

defining a region of interest (ROI) with an initial size in the slide image (S5), and

iteratively growing the region of interest from the initial size to a final size as long as a target parameter, which is based on the two detected densities Di and D₂ within the region of interest and which describes the region of interest, is above a predefined target value (S6).

15. A program element for identifying a region of interest to be dissected from a biological tissue sample, which program element, when being executed by a processor, is adapted to carry out:

receiving a slide image of the biological tissue sample,

analyzing the slide image with at least two different cell nucleus detectors and detecting at least two types of cells in the slide image,

creating a first density map of a density Di of the first type of detected cells, creating a second density map of a density D₂ of the second type of detected cells,

defining a region of interest (ROI) with an initial size in the slide image, and iteratively growing the region of interest from the initial size to a final size as long as a target parameter, which is based on the two detected densities Di and D₂ within the region of interest and which describes the region of interest, is above a predefined target value.

16. A computer-readable medium, in which a computer program for identifying a region of interest to be dissected from a biological tissue sample is stored, which, when being executed by a processor, is adapted to carry out:

receiving a slide image of the biological tissue sample,