EP1908009A2

EP1908009A2 - Method and device for the segmentation of regions

Info

Publication number: EP1908009A2
Application number: EP06760799A
Authority: EP
Inventors: Rupert Ecker; Georg E. Steiner; Vincent Laurain
Original assignee: TISSUEGNOSTICS GmbH
Current assignee: TISSUEGNOSTICS GmbH
Priority date: 2005-07-26
Filing date: 2006-07-24
Publication date: 2008-04-09
Also published as: WO2007012098A3; US8023737B2; AU2006274475A1; ZA200800131B; WO2007012098A2; AU2006274475B2; CA2616011A1; AT503459A4; US20080193014A1; AT503459B1

Abstract

The invention relates to a method for segmentation of locally-defined regions. According to the invention, the input data obtained for the image region under investigation is separated into a foreground mask and a background mask by application of a threshold value, two different filter operations are applied to the input data to determine segmentation starting points, the resulting data sets are subtracted and the image regions viewed as segmentation starting points, a filter operation is applied to generate a less restrictive mask for the input data, the resulting data subjected to a threshold value reduction, the less restrictive mask subjected to a labelling with regard to determination of the local values for generation of a restrictive mask, the mean values determined, segmentation starting points not obtained in the less restrictive mask are removed, a growth into all regions of the restrictive mask is carried out and the regions obtained in the restrictive mask are viewed as the segmented brightness regions.

Description

Method and device for segmentation of areas

The invention relates to a method according to the preamble of claim 1. Furthermore, the invention relates to a device for carrying out this method according to the preamble of claim 8 and a computer program product.

The evaluation of images of human, plant or animal cells or grains of metals or metal alloys or two-dimensional probability distributions or the like. takes place by taking appropriate images and evaluating these, in particular enlarged or imaged by a microscope, images.

For the evaluation, it is necessary to display as precisely as possible the distributions, in particular brightness distributions, or their limits on the recorded images. The aim of the invention is to segment the locally limited areas appearing in such images, in particular brightness ranges, in order to have the recorded images or images as well as possible or true to nature or to make them available for further evaluation.

According to the invention, these objects are achieved in a method of the type mentioned above with the features contained in the characterizing part of claim 1. A device for carrying out the method according to the invention is characterized by the features of claim 8.

With the procedure or device according to the invention, a largely natural or dimensionally true segmentation or imaging of locally limited regions of color images or gray value images is achieved. The obtained data or images can be further evaluated. The erfiήdungsgemäße approach requires the greatest accuracy a not too large computational effort and therefore provides the desired results quickly.

In order to improve the representation it can be provided that a single-stage dilation is applied to the determined segmentation starting points or that discrimination of the gray values or the gray value mask during labeling results in discrimination with a predetermined gray value. Advantageously, in order to improve the image quality obtained, a data distribution diagram or a histogram, in particular a brightness histogram, of the input data is created for separating the input data into bright foreground data and dark background data. Furthermore, the difference quotient of the smoothed diagram or histogram and the absolute values of the difference quotients are calculated and if necessary a constant value is added. The data obtained are with the Input data correlated, in particular multiplied, and possibly rescaled. The threshold value chosen is the value at which the function obtained adopts a specific, predetermined limit value, in particular the value zero, or has approached this value. The method according to the invention is used particularly advantageously when images or images of cell nuclei or cell membrane agglomerates dyed dark with respect to other cell areas or against the cytoplasm or colored stained nuclei or cell nuclei agglomerates contained in tissue sections are to be segmented. In the following the invention will be explained in more detail with reference to the drawing.

Fig. 1 shows an input data set or the recorded areas.

FIG. 2 shows a foreground mask or the foreground data.

Fig. 3 shows the mask of the segmentation start points.

Fig. 4 shows an input data set, Laplace and Gaussian filtered input data and a less restrictive mask.

Fig. 5 shows Laplacian and Gaussian filtered input data, a less restrictive mask and a restrictive mask.

Figures 6a, 6b, 6c and 6d show a less restrictive mask, a labeled, less restrictive mask, laplaced filtered image data and a restrictive mask. Fig. 7 shows the mask of the input data and the segmentation mask.

Fig. 8 shows the procedure for the dilation of the segmentation start points.

Fig. 9 shows the flow chart concerning the determination of the threshold value for separating the foreground data from the background data.

10 shows a device for carrying out the method according to the invention.

In advance, a device according to the invention will be explained with reference to FIG. 10 in principle.

From a tissue preparation or tissue section 1, an image is recorded by means of an image acquisition unit and a corresponding set of digital input data made available.

In a discriminator 3, the input data are separated by specifying a brightness threshold value into a foreground mask containing correspondingly high brightness values and a background mask containing lower values. The dark background data is no longer used. The input data are fed to a filter unit 4, in which a correlation, in particular multiplication, of the input data with the foreground mask takes place. As a correlation could also be a weighted addition of the data or values or another Type of linkage. The output data of the filter unit 4 are fed to two different filter units, preferably on the one hand to a Gaussian filter 5 and on the other hand to a local maximum filter 6. Instead of a Gaussian filter, especially low-pass filters or filters that reduce sharp contours come into consideration. The output data of the two filters, in the present case the Gaussian filter 5 and the local maximum filter 6, are supplied to a subtraction unit 7, in which the two data sets resulting from the respective filterings are subtracted and which have a threshold value, in particular the value zero approximate data or filter areas at the output of the subtraction unit are provided as segmentation start points. These segmentation starting points are also stored in the subtraction unit to be available for further calculations.

Furthermore, an edge-detecting filter, preferably a high-pass filter, in particular a Laplace filter 9, is provided for the input data, to which at most a median filter and / or Gaussian filter 8 is connected upstream. The edge-detecting filter, in the present case a Laplace filter 9, is followed by a threshold value generator 10, in which the data obtained by the filter 9 are subjected to a threshold value setting with a specific threshold value, in particular zero. The zero crossing of the values is an indication of the gradient maximum, so that the best possible differences to neighboring regions can be obtained. In a correlator downstream of the threshold value generator 10, in particular multiplier 11, the data set obtained with the threshold value value is correlated, in particular multiplied, with the foreground mask, whereby a less restrictive mask is obtained.

To create a restrictive mask, a labeling unit 12 is connected to the output of the correlator or multiplier 11, in which the outputs of the multiplier 11 corresponding to the less restrictive mask are made with respect to the determination of the local gray values of each of the locally limited areas or clusters and the average gray values of the respective areas are determined. As a result, the obtained gray values in each cluster undergo discrimination with a predetermined threshold. This gives a restrictive mask.

The data set corresponding to the less restrictive mask is fed to a comparator 13, which is connected downstream of the subtraction unit 7. In the comparator 13, the segmentation starting points not included in the less restrictive mask are eliminated.

In a growing unit 14, growth takes place starting from the individual segmentation starting points in the region of the restrictive mask and at most one following growth of these areas in the less restrictive mask and optionally subsequently in the foreground mask, wherein the data sets concerning the restrictive mask are supplied by the labeling unit 12 and the data sets concerning the less restrictive mask are supplied by the multiplier 11 and the discriminator 3, respectively, to the growing unit 14.

To the Growingeinheit 14, a memory unit and / or evaluation and / or display unit 15 is connected.

Between the subtraction unit 7 and the comparator 13, a dilation unit 16 can be interposed for the calculated segmentation points. A device according to the invention results in a relatively low computational effort to improve the results obtained when the features of claim 10 are provided.

In the following, the procedure according to the invention is explained in greater detail, for example, based on the segmentation of cells. To this end, cell preparations, in particular cell sections, are stained in order to identify the cell nuclei differently from the remaining cell components. Thus, the cell nuclei or the nuclear agglomerates have a different coloration to the remaining cell constituents or the image background or other constituents contained in the tissue section, and can be localized in the recorded image. For the evaluation of the image, there is the uncertainty that in cell nuclear agglomerates the cell nuclei are covered or not reproduced in their entirety, or the brightness ranges do not have sharp boundaries or brightness values that are not easily distinguishable, ie. Color values or gray values are present. This should be counteracted according to the invention. From the image area to be examined of a tissue preparation or

Tissue sections 1 are placed over the inserted receiving unit 2, e.g. a video camera or a microscope, received digital input data. By specifying a selected or based on a predetermined or determined for this input brightness threshold value, the input data of the image area in a light foreground data containing foreground mask and a dark background data containing background mask in the discriminator 3 are separated. Fig. 1 shows an example of an input data set. FIG. 2 shows a mask with which the background of the input data can be separated from the foreground or shows the input data record after discrimination with the predefined or computationally determined brightness threshold value.

In order to determine segmentation start points, the foreground data or the foreground mask can be correlated in advance with the input data, in particular multiplied. In the filter unit 4, two different filters, eg both a Gaussian filter and a local maximum filter, are applied next to one another in the filter unit 4. The filter size of the local maximum filter is advantageously half the size of the Gaussian filter previously applied to the data. Thereupon, the data sets resulting from these two filters are subtracted in the subtraction unit 7 and the image areas fulfilling a threshold criterion, in particular those having the value zero, are regarded or selected as segmentation starting points. The mask or the data record of the segmentation starting points is shown for example in FIG. 3. The segmentation starting points lie within the bright areas of the foreground mask, and the segmentation starting points correspond to the bright areas of the background ground mask.

^/ It is possible to apply a single-stage dilation to these segmentation starting points and to regard the resulting point areas as the segmentation starting point. This avoids that subsequently two closely juxtaposed segmentation start points lead to segmentation errors.

Subsequently, two differently restrictive segmentation masks are determined, namely a less restrictive mask and a restrictive mask.

For this purpose, initially an edge-detecting filter, in particular the Laplace filter 9, is applied to the input data corresponding to FIG. 1, if appropriate after median filtering and / or Gaussian filtering. In order to create the less restrictive mask, the data obtained, in particular lap data, is subjected to a threshold value setting, in particular with a threshold value of zero, and the data set thus obtained is correlated, in particular multiplied, with the data of the foreground mask. In Fig. 4, this procedure is shown. On the left in Fig. 4, the input data shown in FIG. 1 are shown. In Fig. 4 center, the laplace and Gaussian filtered input data are shown. In Fig. 4 right, the resulting restrictive mask is shown. For this less restrictive mask, a function could be used to close holes in the mask. To create the restrictive mask in the labeling unit 12, the less restrictive mask is subjected to labeling with regard to the determination of the local gray values of each of the locally limited regions or clusters, and the average gray values of the respective regions are determined, whereupon the gray values obtained in each cluster undergo discrimination with a given threshold. This threshold is calculated from the respective data average of the laplaced filtered data within each individual cluster of the less restrictive mask. The cluster becomes a continuous white or bright area in the less restrictive mask. FIG. 5 shows on the left the lap-laced and Gauss-filtered input data corresponding to the middle illustration of FIG. 4. The less restrictive mask is shown in the middle in FIG. The combination of the Laplacian and Gaussian filtered input data and the less restrictive mask results in the restrictive mask which is shown on the right in FIG.

For labeling, the pixels of each individual contiguous region in the less restrictive mask of FIG. 6 are assigned an ascending number. This process enables a subsequent, separate processing or analysis of each individual region. In Fig. 6b, the label image is shown. The individual regions are assigned ascending numbers, which in FIG. 6b depict gray values corresponding to the colors from dark blue (SP ^{: '} Niern) to dark red (middle region); /

Each eh .. * ie region of Fig. 6b is placed successively on the spatially corresponding, laplace filtered image data of Fig. 6c. For each individual region the mean value is calculated from the laplace filtered image data. Each of these values then serves as a threshold for the respective region. By applying the individual threshold values to the respective corresponding regions in the Laplace image, the restrictive mask according to FIG. 6d is derived from the less restrictive mask

Subsequently, the segmentation starting points not contained in the less restrictive mask are eliminated. This is done by a corresponding comparison of the less restrictive mask with the mask with the already determined segmentation starting points in the comparator 13.

Subsequently, in the growing unit 14, within the regions of the restrictive mask, growth is made starting from the segmentation starting points present in the individual regions, followed by growth of these obtained regions in the less restrictive mask. This already achieves a good segmentation of the brightness distributions present in the initially obtained image data.

Subsequently, the growth of the areas left in the less restrictive mask could be continued in the foreground mask. It was found that the areas grown in the foreground mask are close to the original distribution. In Fig. 7, the input mask on the left, that is, the mask shown in FIG. 1. Right in Fig. 7, the segmented input data is shown after waxing in the foreground mask has been made. If individual segmentation start points are not in the restrictive segmentation mask, then the restrictive segmentation mask can be modified before the start of the diiation process.

In Fig. 8, the processing procedure for dilating segmentation start regions in a given segmentation mask, i. the restrictive mask, shown. The result of this processing is dimensionally accurate segmented input data (Figure 8).

The segmented regions shown on the right in Fig. 7 may subsequently be used, e.g. also be filtered interactively with a defined surface area, gray value or probability threshold, in order to improve their further processing or further evaluation.

From these segmented images or data, various parameters of the imaged nuclei, brightness or probability distributions, grain sizes, etc. can be calculated.

The segmented input data is visually, e.g. printed on a screen or printed or possibly stored with the calculated parameters.

To calculate a background threshold value for separating the input data into a background and foreground area in the discriminator 3, the procedure is as shown in FIG. 9. The background is assumed to be the dark image areas and the foreground the bright image areas. In the right image of Fig. 1 is the Intensitätsbzw. Brightness histogram of the input data is shown in which the optimum range for the background threshold is marked in advance accordingly. This threshold value is determined by means of a procedure explained in more detail with reference to FIG. 7 by analytical processing and processing of the brightness data distribution diagram, in particular histogram, of the input data.

For this purpose, it is advantageously provided that to separate the input data into bright foreground data and dark background data, a histogram, in particular a brightness histogram, of the input data is calculated, the difference quotient of the smoothed histogram is calculated, the absolute values of the difference quotients are calculated and, if necessary, a constant value is added Data are multiplied by the input data and, if necessary, rescaled, and the threshold value chosen is that value at which the function obtained is zero or has approached this value.

By applying the calculated optimal threshold to the input data, the background becomes as well as possible from the areas relevant to the segmentation, i. foreground or foreground data, separated. In Fig.

2 is the mask with which the background of the input data from the foreground can be separated, shown. For the further processing steps or the further segmentation then only the foreground area of the input data is relevant.

The inventive device can be realized to achieve high computing speeds with hardware components. The filters used can be of various types. Advantageously, conventional low-pass filters or high-pass filters or Gaussian filters and Laplace filters come into question. The use of filters acting in the same way is readily possible.

The threshold values are selected according to the desired accuracy or the desired data separation and are to be specified. The provided correlation methods may e.g. in a weighted addition of the data or in a multiplication, possibly also in a division or other combination of the data, exist, as long as a corresponding increased distinctness of the data with the correlation is achieved.

The invention has been explained with reference to the evaluation of brightness values or determination of gray levels. When evaluating data fields, the values contained in the data field and derived from other variables replace these values

Values or determined values. In a similar way, the evaluation could also

Color values are used.

Claims

claims:

1. A method for segmentation of locally limited, two-dimensional regions, preferably brightness distributions or regions or data fields, in particular for images of nucleus agglomerates or aggregates, particle size distributions or the like, characterized

that the input data obtained from the image area to be examined are separated by specifying a threshold value, in particular brightness threshold value, in a larger or higher values, in particular brightness values, foreground mask containing corresponding foreground data and a background mask containing low values, in particular brightness values, corresponding background data; for determining segmentation starting points on the input data preferably correlated with the foreground mask, in particular multiplied, two different filter operations, namely a sharp contours or contrasts reducing filter, preferably a low pass filter, in particular a Gaussian filter, and a filter for determining local maxima in a region or a local maximum filter is applied,

- That then subtracts the resulting in the two different filter operations records and a specific limit or threshold range, preferably the value zero, possessing image areas are regarded as Segmentierungsstartpunkte - that to create a less restrictive mask on the input data, optionally after median filtering and / or a Gaussian filtering, a high-frequency-emphasizing filter operation, preferably a high-pass filter, in particular a Laplace filter, and subjecting the resulting data to a threshold with a specific threshold, preferably zero, and the resultant Data set is correlated with the data of the foreground mask, in particular multiplied.

in order to create a restrictive mask, the less restrictive mask is subjected to a labeling with regard to the determination of the local values, in particular gray values, of each of the locally limited areas or clusters and the average values, in particular gray values, of the respective areas are determined and the obtained values, in particular gray values, are subjected in each cluster to a discrimination with a predetermined threshold value,

that segmentation starting points not contained in the less restrictive mask are eliminated, that growth takes place starting from the individual segmentation starting points in the respective areas of the restrictive mask,

- That a growth of these areas in the less restrictive mask and optionally then in the foreground mask is continued and

- That the areas obtained in the restrictive mask or in the foreground mask are regarded as the segmented brightness areas.

2. The method according to claim 1, characterized in that on the determined Segmentierungsstartpunkte a single or multi-stage Dilation is applied.

3. The method of claim 1 or 2, characterized in that when discrimination of the gray values or the gray value mask during labeling a discrimination with a predetermined gray value takes place.

4. The method according to any one of claims 1 to 3, characterized in that the filter size of the local maximum filter is correlated or adjusted with the size of the Gaussian filter.

5. The method according to any one of claims 1 to 4, characterized

in that, to separate the input data, in particular into bright foreground data and dark background data, a diagram representing the data distribution, in particular a brightness histogram, of the input data is produced,

that the difference quotient of the smoothed diagram representing the data distribution is calculated,

that the absolute values of the difference quotients are calculated and, if appropriate, a constant value is added,

- that the obtained data correlates with the input data, in particular multiplied, and possibly rescaled, and

that the value selected is the threshold value at which the function obtained corresponds to a specific limit value, preferably the value zero, or has approached this limit value.

6. The method according to any one of claims 1 to 5, characterized in that the images or images are imaged and segmented with respect to other cell areas or against the cytoplasm light colored cell nuclei or nuclear agglomerates.

7. The method according to any one of claims 1 to 6, characterized in that in tissue sections contained colored cell nuclei or nuclear agglomerates are segmented.

8. Device for segmenting locally delimited, two-dimensional regions, preferably brightness distributions or regions, in particular for images of cell nucleus agglomerates or assemblies, particle size distributions, data fields or the like, in particular for carrying out the method according to one of claims 1 to 7, characterized in that - a discriminator (3) is provided, the input data from the image to be examined input data by specifying a threshold, in particular brightness threshold, in a larger values, in particular bright values, corresponding foreground data containing foreground mask and a smaller values, in particular dark Separates values containing background data containing background data,

in that a filter unit (4) is provided for determining segmentation start points, which independently, two filters, namely a sharp contour or contrast reducing filter, preferably a low-pass filter, in particular a Gaussian filter (preferably correlated with the foreground mask, especially multiplied input data). 5), and a local maximum filter (6) applies,

that a subtraction unit (7) is provided which subtracts the data sets resulting from these two filters and determines and stores the image areas possessing a specific limit value or limit value range, in particular the value zero, and makes them available for further calculations,

in that an edge-detecting filter or a filter emphasizing the high-frequency components, preferably a high-pass filter, in particular a Laplace filter (9), for the input data, optionally with upstream median filter and / or Gaussian filter (8), is provided for producing a less restrictive mask, is followed by a correlator, preferably a multiplier (11), with which the data of a threshold value setting with a specific threshold, in particular the threshold zero, subjected and the resulting data set with the data of the foreground mask correlated, in particular multiplied.

a masking unit (12) is provided for creating a restrictive mask, which labels the less restrictive mask with regard to the determination of the local values, in particular gray values, of each of the locally limited areas or clusters, and the average Values, in particular gray values, of the respective regions are determined, the obtained values, in particular gray values, being subjected to discrimination with a predetermined threshold value in each cluster, - a comparator (13) is provided which eliminates segmentation starting points not contained in the less restrictive mask .

- That a Growing unit (14) is provided, which makes a waxing from the individual Segmentierungsstartpunkten in the respective areas of the restrictive mask, and continues the growth of these areas in the less restrictive mask and optionally then in the foreground mask and

- That the areas obtained in the restrictive mask or in the foreground mask stored as the desired segmented brightness ranges, displayed and / or further processed or a storage unit and / or a display unit and / or a further evaluation unit (15) are supplied.

9. Device according to claim 8, characterized in that a single-stage dilation unit (16) is provided for the determined segmentation starting points.

10. Device according to one of claims 8 or 9, characterized in that - the discriminator (3) for separating the input data, in particular in bright foreground data and dark background data, a data distribution diagram representing diagram builder, in particular Histogrammbildner, for the input data υmfasst,

that a difference quotient generator is provided for the smoothed diagram, in particular histogram, which calculates the absolute values of the difference quotients and optionally adds a constant value,

- That a corrector, in particular multiplier, is provided for linking the data obtained from the difference quotient generator with the input data, and

- That a threshold value generator is provided, which selects that value at which the received function has a specific threshold, in particular zero, or has approached this value.

11. The device according to claim 10, characterized in that the device comprises a computer and the hardware and software required to perform the functions.

12. Use of a device according to any one of claims 8 to 10 for the segmentation of compared to other cell areas or against the cytoplasm dark colored cell nuclei or nuclear agglomerates and / or contained in tissue sections stained nuclei or nuclear agglomerates.

A computer program product having program code means stored on a computer readable medium for carrying out the method of any one of claims 1 to 7 when the program product is executed on a computer.