WO2011154543A1

WO2011154543A1 - Texture characterisation

Info

Publication number: WO2011154543A1
Application number: PCT/EP2011/059745
Authority: WO
Inventors: Reyer Zwiggelaar
Original assignee: Oncomorph Analysis Ltd
Priority date: 2010-06-11
Filing date: 2011-06-10
Publication date: 2011-12-15
Also published as: GB201009842D0

Abstract

A method for creating an electronic library for characterising the texture type of a portion of a digital image having a particular texture type, wherein a digital image comprises a plurality of pixels, each pixel having a texture value which is representative of texture at a respective position, the method comprising: determining a numerical value for pixels within the portion, wherein a numerical value for a pixel is the sum of the texture values of the pixels of a local window of pixels around the pixel and is representative of the particular grey level configuration within the local window; using the numerical values and their occurrences to generate a histogram for the texture type; and storing in the electronic library the histogram and an associated indication of the particular texture type.

Description

TEXTURE CHARACTERISATION

Field

The present invention relates to texture characterisation. In particular, the present invention relates to characterising texture types of digital images, for example digital images comprising medical imaging data (e.g. magnetic resonance imaging (MRI) data). Texture characterisation is important in a range of applications including the characterisation of tissue texture to assist in identifying abnormal growth such as cancerous growth, for example in the prostate gland.

Background

One known technique to characterise texture type is for a specialist such as a radiologist to analyse an image such as an MRI image.

The present invention seeks to provide a more automated technique.

Summary

One aspect of the present invention provides a method for creating an electronic library for characterising the texture type of a portion of a digital image having a particular texture type, wherein a digital image comprises a plurality of pixels, each pixel having a texture value which is representative of texture at a respective position, the method comprising: determining a numerical value for pixels within the portion, wherein a numerical value for a pixel is determined based on the texture values of the pixels of a local window of pixels around the pixel; using the numerical values and their occurrences to generate a histogram for the texture type; and storing in the electronic library the histogram and an associated indication of the particular texture type.

Another aspect of the present invention provides a method of identifying the particular texture type of a region of a digital image, the digital image comprising a plurality of pixels, each pixel having a texture value which is representative of texture at a respective position, the method using an electronic library which stores for a plurality of particular texture types a histogram for the particular texture type, the method comprising processing the digital image to allocate a texture type to the region of the digital image by: determining a numerical value for each pixel in the region, based on the texture values of the pixels of a local window of pixels around the pixel; using the numerical values and their occurrences to generate a generated histogram for the region; comparing the generated histogram with one or more histograms from the electronic library to identify a best fitting histogram; and allocating the texture type associated with the best histogram to the region.

Another aspect of the present invention provides a method of identifying the particular texture type for pixels within a region of a digital image, the digital image comprising a plurality of pixels, each pixel having a texture value which is representative of texture at a respective position, the method using an electronic library which stores for a plurality of particular texture types a histogram for the particular texture type, the method comprising processing the digital image to allocate a texture type to pixels with the region of the digital image by, for each pixel: defining a local region of pixels around the pixel; extracting local windows of pixels from the local region; determining a numerical value for each local window based on the texture values of the pixels within the local window; using the numerical values and their occurrences to generate a generated histogram for the pixel; comparing the generated histogram with one or more histograms from the electronic library to identify a best fitting histogram; and allocating the texture type associated with the best histogram to the pixel.

In embodiments of the present invention digital images may comprise medical data/medical imaging data, and optionally medical data of soft tissue such as magnetic resonance imaging data.

Another aspect of the present invention provides a system comprising a memory and a processor, wherein the processor is arranged to perform one or more of the above methods.

Another aspect of the present invention provides a computer-readable medium having computer-executable instructions adapted to cause a computer system to perform one or more of the above methods.

Accordingly, a more automated technique is provided.

Other aspects and features of the present invention will be appreciated from the following description and the accompanying claims.

Brief description of the drawings

Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings in which like reference numerals are used to depict like parts. In the drawings:

Figure 1 is a flow diagram illustrating a method in accordance with an embodiment of the invention;

Figures 2-4 illustrate digital images; Figure 5 illustrates a technique to compare generated histograms with those from an electronic library;

Figure 6 illustrates a digital image which has had its pixels classified by texture type; and

Figure 7 illustrates another technique to compare generated histograms with those from an electronic library;

Detailed description of embodiment(s)

Configuring the electronic library

A method is provided for creating an electronic library for characterising the texture types of portions of digital images, for example MRI images of prostates. Such a method will be described with reference to Figure 1 , but to assist with understanding the steps of the method of Figure 1, a description of Figures 2 to 4 will first be given.

Figure 2 illustrates a digital image 20. The digital image comprises a plurality of pixels, and each pixel has a texture value which is representative of the texture of the article at a respective position. For simplicity, not all pixels of the digital image 20 are illustrated. Some of the pixels of the digital image are illustrated as a 20x15 grid, and within this grid three example pixels 22, 24 and 26 are shown. Each pixel is represented by a location in the digital image and by a single or a plurality of values

(representing, for example, grey-level, colour, etc), which will be referred to as the texture value. A texture value for a pixel is a numerical representation of the texture of the article at the position of the location represented by the pixel, for example any numerical value between 0 and 255. The digital image may comprise medical imaging data, for example soft tissue discrimination imaging data such as magnetic resonance imaging (MRI) data. MRI data is generated by an MRI scanner and the data typically forms a set of digital images which together form 3D/volumetric data.

Figure 3 also illustrates the digital image 20 and example pixels 22, 24, 26. Around each pixel a respective window 32, 34, 36 of the image is illustrated. Typically, a window is a subset of the digital image. Window 32 is a 3x3 array of pixels around and including pixel 22. Window 36 is a 5x5 array of pixels around and including pixel 26. Window 34 is an irregular collection of 12 connected pixels around and including pixel 34. In different embodiments, different shaped and sized windows may be used, although windows in the form of 3x3 arrays such as window 22 are used in a particular implementation which will be described and referred to throughout this description.

A set of predetermined particular ranges of texture values can be defined. Figure 4 shows a digital image 20 in which the texture value of each pixel falls into one of four distinct ranges. For example, the pixels marked with vertical shading such as pixel 45 each have a value within a first range; the pixels marked with diagonal top left to bottom right shading such as pixel 46 each have a value within a second range; the pixels marked with horizontal shading such as pixel 47 each have a value within a third range; and the pixels marked with diagonal bottom left to top right shading such as pixel 48 each have a value in a fourth range. For example, if the texture values can have any possible value between 0 and 255, the first range could be between 0 and 63, the second range between 64 and 127, the third range between 128 and 191 and the fourth range between 192 and 255. In this example, texture values of say 7, 9 27, or 36 would all fall in the first range and so on. Embodiments may use any number of ranges. The particular implementation uses a set of eight ranges.

A digital image may comprise data representing one or more particular texture types. In Figure 4 two texture types 42 and 44 are illustrated. For the method of creating the electronic library the particular texture types would be determined and classified by a texture classification expert such as a radiologist where the image is a medical image. The radiologist would classify the texture types as "nodular tissue" or "radiolucent tissue" for example.

Referring now back to Figure 1, a flow diagram of a method for creating an electronic library for characterising the texture types of at least a portion (or optionally the whole) of a digital image is shown. Each pixel in the portion has a texture value which is representative of texture at a respective position of the digital image. Typically, the steps S2 to S6 are performed for some or all of the pixels within a portion of a digital image which have been ascribed a particular texture type.

Referring to step S2, a numerical value for pixels within the portion are determined, wherein the numerical value for a pixel is determined based on the texture values of the pixels of a local window of pixels around the respective pixel.

A defined local window shape and size is used at this stage, for example the defined window shape and size can be the 3x3 local window 32 of Figure 3.

In one embodiment the resolution of the digital image is reduced before determining the numerical values. In the particular implementation, the resolution is reduced by defining a set of eight ranges and allocating a respective value (e.g. 0, 1, 2, 3, 4, 5, 6, 7) to each pixel based on whether the texture value of that pixel falls within a particular range. For example, if texture values can take values of between 0 and 255, pixels with texture values between 0 and 31 would be allocated the value 0, those with texture values between 32 and 63 would be allocated the value 1, those with texture values between 64 and 95 would be allocated the value 2 and so on (with those with texture values between 224 and 255 would be allocated the value 7). This can be considered as setting the grey level resolution.

An array is defined with elements corresponding to the pixels of the portion of the digital image being processed.

A numerical value for each pixel is then determined using the following equation:

where counter (i, j) starts at a value of zero and increments at each location in the local window, and I(i, j) is the (reduced resolution) grey level texture value at position (i, j) in the local window. Each numerical value can be considered as representing the particular grey level configuration or appearance within the respective local window.

In the equation above, i and j indicate the position within a local window (of arbitrary shape and size), #bins is the number of grey level bins used in the grey level range (for an image with 256 possible grey levels the maximum number of bins is 256, but other values are 256/n, where n is an integer number), counterQ is a function that starts at zero for the first position in the local window and increments by a value equal to one for each subsequent position in the local window (so, for example, for a local window that contains 9 positions the range of counter() ranges from 0 to 8). I(i,j) represents the grey level value at position (i j) within the local window.

The calculated numerical values are stored in the array for the portion.

The process of step S2 is typically performed for all pixels for which the respective local window around the pixel falls within a particular portion or area of the digital image. Another way of considering this is that the step is typically performed for all windows within the portion of the image to determine a numerical value for the pixel around which the window is formed. For example with reference to Figure 4 these steps can be performed for all pixels which have their respective local windows falling within area 42 (in which all pixels have a particular texture type). That is, for example, for a 3x3 local window and a rectangular portion of the digital image, typically a numerical value would not be calculated for pixels along the edge of the region. In some embodiments numerical values may be calculated for pixels within the portion of the digital image for which the respective local window around the pixel falls partially outside the particular portion, by for example using texture values from pixels within a local window which are outside the portion. In other embodiments numerical values may be calculated for such pixels by weighting appropriately the texture values of the pixels in the local window which are within the region. For example, if two thirds of the pixels within a local window are within the region and the a third of the pixels within a local window are outside the region, then a numerical value for the pixel can be generated by summing the texture values of the pixels within the region and multiplying them by 1.5 to account for the "missing" one third of the texture values. Referring to step S4, the numerical values and their occurrences within the portion (using the array) are used to generate a histogram for the texture type. The stored numerical values can be transformed into a histogram - which can be considered as a Local Grey level Appearance (LGA) histogram - that contains the combination of unique numbers and their occurrences. At this stage it is possible to remove from the histogram as "noise" numerical values which have low occurrences. The histograms can be normalised, for example the histograms can be normalised so that the total area covered by the histogram is 1. This can be achieved with a LI metric, for example.

This completes the formation of the histogram. It is possible to generate a single histogram from a single image, or a number of histograms representing various textures types, or a single histogram

from a series of images (all representing similar texture type). LI normalisation would typically take place after all relevant histograms have been combined.

Referring to step S6, the histogram and an associated indication of the particular texture type is stored in the electronic library.

The process described with reference to Figure 1 can be performed for a plurality of test articles, i.e. articles with particular, known texture types, such as MRI test images of prostates, each having an ascribed texture type. Accordingly, an electronic library can be generated which stores for a plurality of particular texture types a histogram and an associated indication of the particular texture type. Using the electronic library to identify texture type

Two methods are provided of identifying texture types of pixels of a digital image, for example an MRI image of a prostate of a patient. Accordingly, an automated identification tool for identifying particular texture type within an image, for example cancerous cells, can be provided.

The methods of identifying the texture type of pixels of a digital image has similar features to the method of configuring the electronic library, with the notable exception that whereas to configure the electronic library the texture type of the relevant part of the digital image was known, when identifying the texture type of the pixels of a digital image this characteristic is initially unknown and the electronic library is used to identify it. For simplicity, features of the earlier figures will be referred to when describing using the electronic library to identify the texture type, although it should be noted that in this section the texture type(s) of the region (in the first method) or the pixels within the region (in the second method) are initially unknown and then determined using the data in the electronic/texture library.

The digital image which is to have the texture type of its region or pixels within a region classified comprises a plurality of pixels, each pixel having a texture value which is representative of the texture of the article at a respective position (see for example the digital image 20 of Figure 2).

Typically, the methods of identifying the texture type of pixels generate histograms using a technique which is similar to the technique used to generate histograms when the electronic library was configured. The first method of identifying texture types

In overview, in the first method to identify the texture type of a region of a digital image, a histogram is generated for the region. Typically, the region would have a uniform looking texture type; that is, the region would be one that is a region of interest and for which a classification of the particular texture type is desired. For example the region can be a 96x96 pixel region. The generated histogram is compared with one or more, or typically all, histograms from the electronic library to identify a best fitting histogram from the library. The texture type associated with the best fitting histogram is allocated to the texture type for the region.

Referring to Figure 5, a method of identifying the particular texture type for a region (such as a 96x96 pixel region) of a digital image is illustrated. The digital image comprises a plurality of pixels, each pixel having a texture value which is representative of texture at a respective position. The method uses an electronic library which stores for a plurality of particular texture types a histogram for the particular texture type. The method comprises processing the digital image to allocate a texture type to the region of the digital image by performing the steps S10 to S16 of Figure 5.

With reference to the Figure, at step S10 numerical values for pixels within the region are determined, wherein the numerical value for a pixel is based on the texture values of the pixels of a local window of pixels around the pixel, for example a 3x3 local window such as 3x3 local window 32 of Figure 3. Note that the region is typically larger than the local window.

In one embodiment the resolution of the digital image is reduced before determining the numerical values. In the particular implementation, the resolution is reduced by defining a set of eight ranges and allocating a respective value (e.g. 0, 1, 2, 3, 4, 5, 6, 7) to each pixel based on whether the texture value of that pixel falls within a particular range, as has been described earlier when describing configuring the electronic library.

An array is defined with elements corresponding to the pixels of the region of the digital image being processed.

A numerical value for each pixel is then determined using the following equation (which is the same equation discussed earlier when describing configuring the electronic library).

The numerical values are stored in the array.

The process of step S10 is typically performed for all pixels for which the respective local window around the pixel falls within the region of the digital image. The discussion above in relation to step S2 applies equally here, but for the sake of brevity will not be repeated.

At step S 12 the numerical values and their occurrences are used to generate a generated histogram for the region. The stored numerical values can be transformed into a histogram - which can be considered as a Local Grey level Appearance (LGA) histogram - that contains the combination of unique numbers and their occurrence. At this stage it is possible to remove from the histogram as "noise" numerical values which have low

occurrences. The histograms can be normalised, for example the histograms can be normalised so that the total area covered by the histogram is 1. This can be achieved with a LI metric, for example.

At step SI 4 the generated histogram is compared with one or more histograms from the electronic library to identify a best fitting histogram. The histograms can be compared using an appropriate distance metric (e.g. Euclidean, transportation or hybrid transportation). The texture type identified for the region may have a respective probability value associated with it, based on the amount of fit, for example using the distance metric, there is between the generated histogram and the best fitting histogram. In addition, probability values may also be given for different texture types based on how well the generated histogram fits with the respective histogram for that texture type.

At step S16 the texture type associated with the best histogram is allocated to the region. A representation of the digital image which has had texture types for two regions classified using the described method is shown in Figure 6. The pixels in a first region 262 have each been allocated one texture type and the pixels in another region 264 have been allocated another texture type.

The second method of identifying texture types

The second method to identify the texture type is similar to the first method, except histograms and texture types or probabilities of texture types are determined for pixels within the region of a digital image. This is achieved by defining a local region (which is typically larger than the size of the local windows) around each pixel and determining a histogram for the pixel by generating histograms using local windows within the local region. The histogram for the local region is compared with one or more, or typically all, histograms from the electronic library to identify a best fitting histogram from the library. The texture type associated with the best fitting histogram is allocated as the texture type for the pixel.

Referring to Figure 7, a method of identifying the particular texture type for pixels of a region of a digital image is illustrated. The digital image comprises a plurality of pixels, each pixel having a texture value which is representative of texture at a respective position. The method uses an electronic library which stores for a plurality of particular texture types a histogram for the particular texture type. The method comprises processing the digital image to allocate a texture type to a pixel of the region of the digital image by performing the steps S20 to S30 of Figure 5 for each pixel.

With reference to the figure, at step S20, a local region of pixels around the pixel is defined. In one particular implementation the local region is a 24x24 pixel region.

At step S22, the local windows from the local region are extracted.

Typically, all local windows from the local region are extracted. In the particular implementation the local windows are 3x3 pixel local windows. Note, typically the local window is smaller in size than the local region

At step S24, a numerical value is determined for each local window based on the texture values of the pixels within the local window.

The numerical values are stored in the array.

At step S26, the numerical values and their occurrences are used to generate a generated histogram for the pixel. The stored numerical values can be transformed into a histogram - which can be considered as a Local Grey level Appearance (LGA) histogram - that contains the combination of unique numbers and their occurrence. At this stage it is possible to remove from the histogram as "noise" numerical values which have low

At step S28, the generated histogram is compared with one or more histograms from the electronic library to identify a best fitting histogram. The histograms can be compared using an appropriate distance metric (e.g. Euclidean, transportation or hybrid transportation). The texture type identified for the region may have a respective probability value associated with it, based on the amount of fit, for example using the distance metric, there is between the generated histogram and the best fitting histogram. In addition, probability values may also be given for different texture types based on how well the generated histogram fits with the respective histogram for that texture type.

At step S30, the texture type associated with the best histogram is allocated to the pixel. The digital image can then be displayed with the texture type of each pixel indicated, for example using colour coding.

The process of steps S20 to S30 is typically performed for all pixels for which the respective local region around the pixel falls within the region of the digital image. In some embodiments the process may be performed for pixels within the region of the digital image for which the respective local region around the pixel falls partially outside the region, by for example using texture values from pixels within a local region which are outside the portion. In other embodiments numerical values may be calculated for such pixels by weighting appropriately the texture values of the pixels in the local region which are within the region, for example in a similar way to that described earlier in relation to step S2.

In one or more implementations the electronic library is a database having records, each record comprising: a histogram; and an indicator of texture type.

A system or computer such as a general-purpose computer can be configured or adapted to perform the described methods. In one

embodiment the system comprises a processor, memory, and a display. Typically, these are connected to a central bus structure, the display being connected via a display adapter. The system can also comprise one or more input devices (such as a mouse and/or keyboard) and/or a communications adapter for connecting the computer to other computers or networks. These are also typically connected to the central bus structure, the input device being connected via an input device adapter.

In operation the processor can execute computer-executable instructions held in the memory and the results of the processing are displayed to a user on the display. User inputs for controlling the operation of the computer may be received via input device(s).

A computer readable medium (e.g. a carrier disk or carrier signal) having computer-executable instructions adapted to cause a computer to perform the described methods may be provided.

Embodiments of the invention have been described by way of example only. It will be appreciated that variations of the described embodiments may be made which are still within the scope of the invention.

For example, although not a limited list, the techniques can be used to analyse texture in images in the fields of medical imagery, materials science, food science, and satellite, airborne and other remote sensing imagery. In medical imagery, CT (computed tomography), MRI,

Ultrasound, PET (positron emission tomography), microscopy images of human or animal tissues, cells, bones or nerves can be processed. In materials science, images of materials such as manufactured fabrics or metals, or raw materials can be processed for quality assurance purposes. Similarly, in food science, images of fimshed products or raw materials can be processed for quality assurance purposes. In satellite, airborne and other remote sensing imagery, images can be processed for automated and knowledge based mapping of weather patterns, geographical features (e.g. wet lands/drought), topological and geological structures (e.g.

dunes/coastline/riverbeds/mountains), plants (e.g. crop types/woodland density and edges), animals (e.g. insect swarms/herds/flocks/classification), and man-made features (e.g. traffic congestion/road boundaries). It will be appreciated that in each case a different article or articles are scanned. Typically, the digital image is therefore a scanned digital image or, more particularly, a scanned digital image of a real world article or articles (or entity/entities or object/objects).

Claims

1. A method for creating an electronic library for characterising the texture type of a portion of a digital image having a particular texture type, wherein a digital image comprises a plurality of pixels, each pixel having a texture value which is representative of texture at a respective position, the method comprising:

determining a numerical value for pixels within the portion, wherein a numerical value for a pixel is the sum of the texture values of the pixels of a local window of pixels around the pixel and is representative of the particular grey level configuration within the local window;

using the numerical values and their occurrences to generate a histogram for the texture type; and

storing in the electronic library the histogram and an associated indication of the particular texture type.

2. A method according to claim 1, comprising:

reducing the resolution of the digital image and determining the numerical value for each pixel in the portion based on texture values at the reduced resolution.

3. A method according to claim 1 or 2, wherein the numerical value is determined usin the equation:

where I(i, j) is the texture value at position (i, j) in the local window, counter (i, j) starts at a value of zero and increments at each location in the local window and #bins is the number of bins used.

4. A method according to any preceding claim, wherein the histogram is normalised.

5. A method according to any preceding claim, wherein the local window is a 3x3 window around the respective pixel.

6. A method according to any preceding claim, comprising:

generating a single histogram from a single digital image.

7. A method according to any of claims 1 to 5, comprising:

generating a plurality of histograms from a plurality of portions of a single digital image, each portion having a different texture type.

8. A method according to any of claims 1 to 5, comprising:

generating a single histogram from a plurality of digital images, each digital image having a portion for the same particular texture type.

9. A method of identifying the particular texture type of a region of a digital image, the digital image comprising a plurality of pixels, each pixel having a texture value which is representative of texture at a respective position, the method using an electronic library which stores for a plurality of particular texture types a histogram for the particular texture type, the method comprising processing the digital image to allocate a texture type to the region of the digital image by:

determining a numerical value for pixels within the region, wherein determining the numerical value for a pixel is the sum of the texture values of the pixels of a local window of pixels around the pixel and is

representative of the particular grey level configuration within the local window; using the numerical values and their occurrences to generate a generated histogram for the region;

comparing the generated histogram with one or more histograms from the electronic library to identify a best fitting histogram; and

allocating the texture type associated with the best histogram to the region.

10. A method of identifying the particular texture type for pixels within a region of a digital image, the digital image comprising a plurality of pixels, each pixel having a texture value which is representative of texture at a respective position, the method using an electronic library which stores for a plurality of particular texture types a histogram for the particular texture type, the method comprising processing the digital image to allocate a texture type to pixels with the region of the digital image by, for each pixel: defining a local region of pixels around the pixel;

extracting local windows of pixels from the local region;

determining a numerical value for each local window, wherein a numerical value for a pixel is the sum of the texture values of the pixels within the local window and is representative of the particular grey level configuration within the local window;

using the numerical values and their occurrences to generate a generated histogram for the pixel;

allocating the texture type associated with the best histogram to the pixel.

11. A method according to claim 9 or 10, wherein comparing the generated histogram with one or more histograms from the electronic library to identify a best fitting histogram comprises determining the distance between the histograms using a distance metric.

12. A method according to any preceding claim, wherein the digital image comprises medical data.

13. A method according to claim 12, wherein the digital image comprises medical data of soft tissue.

14. A system comprising a memory and a processor, wherein the processor is arranged to perform the method of any preceding claim.

15. A computer-readable medium having computer-executable instructions adapted to cause a computer system to perform the method of any of claims 1 to 13.