WO2004047609A2

WO2004047609A2 - A method and a system for establishing a quantity measure for joint destruction

Info

Publication number: WO2004047609A2
Application number: PCT/DK2003/000813
Authority: WO
Inventors: Michael Grunkin; Johan Doré HANSEN; Karl Rudolphi; Manfred Keil; Manfred Schudok
Original assignee: Visiopharm Aps; Aventis Pharma Deutschland Gmbh
Priority date: 2002-11-27
Filing date: 2003-11-26
Publication date: 2004-06-10
Also published as: WO2004047609A3; AU2003283206A8; AU2003283206A1

Abstract

The present invention relates to a method and a system for establishing a measure of joint destruction on a bone surface in an individual animal, in particular an animal suitable for pre-clinical studies of drug development, such as in animal models for degenerative joint diseases. The method includes a quantitative image based method capable of reflecting the morphological changes related to joint destruction, wherein a multispectral image of said bone surface is provided and a property of at least one spectral component or a combination of spectral components is used to establish the measure.

Description

A method and a system for establishing a quantity measure for joint destruction

Field of invention

The present invention relates to a method and a system for establishing a measure of joint destruction on a bone surface in an individual animal, in particular an animal suitable for pre-clinical studies of drug development, such as in animal models for degenerative joint diseases.

Background of invention

During development of medicaments the compounds of interest pass through a series of tests in order to assess the effect and security of the final medicament. The series of tests are divided into pre-clinical phases and clinical phases, the latter being tests in humans. The tests in the pre-clinical phases are for example provided to test the assumed effect of the compound in question in mammals. For a variety of diseases animal models have been developed; for example an animal model for osteoarthritis has been developed, said model including mice known to acquire os- teoarthritis with aging. Osteoarthritis leads to destruction of cartilage, sometimes even to such an extent that the underlying bone becomes exposed. Such an animal model may be used for testing medicaments expected to have a therapeutic effect on osteoarthritis. The effect of medicaments and control compounds on the joints and cartilage in the animal may be examined after sacrificing the mouse, normally by staining the cartilage and examining histological sections of the cartilage and thereby grading the effect of the medicament on the cartilage.

Currently, the examination is carried out principally by an expert in histology (grader) examining each bone "manually". This is a very time-consuming process, since even an experienced grader can take several minutes to assess a single histological section. Both inter- and intra grader variations are known to have significant influence on the results. The grader has to evaluate the degree of joint destruction and cartilage destruction in each joint examined, i.e. to provide a qualitative characterisation of the joint damage. Joint destruction may be observed as follows: Mild lesions characterized by focal surface disruption may be observed by scanning electron microscopy (SEM). Light microscopic (LM) alterations in joints may consist of varying degrees of focal chon- drocyte death, decreased matrix staining, and surface fibrillation. Small chondrocytic clones may be observed by LM in animals with moderate focal degeneration. Osteo- phyte formation, mild synovial hyperplasia, cartilage degeneration, may be seen by LM. More severe lesions may include subchondral sclerosis of the bone.

It is desirable to provide ways of assessing joint degradation quantitatively based on computerized image analysis.

Attempts to conduct a quantitative characterisation of osteoarthritis by imaging are discussed in Richardson et al. "Quantitative characterisation of osteoarthritis in the guinea pig", poster, 47^th Annual Meeting, Orthopedic Research Society, February 25-28, 2001 , San Francisco, California wherein India ink staining and digital imaging methods could be used to identify age- and site-dependent degeneration of cartilage in the knee joints of guinea pigs. 3 groups of guinea pigs, at 7, 11 and 15 months of age, respectively were examined. Digital averages of the registered images from each experimental group were computed, and difference images were computed to highlight and localize differences in ink-staining patterns with aging. Differences between the 3 groups are shown, but the method is not capable of revealing any information relating to the individual guinea pig examined.

In animal studies in the pre-clinical phases of drug development groups of animals are typically subjected to the medicament or a control (not active) compound. The results of the studies are often presented as the average effect in the treatment group as opposed to the average effect in the control group. However, it is also of importance to classify the individual animals in each group with respect to joint destruction and cartilage destruction.

Summary of invention

The present invention relates to a quantitative image based method capable of reflecting the morphological changes related to joint destruction. This is achieved by a method for establishing a quantity measure for joint destruction on a bone surface in an individual animal, wherein said bone surface is a bone surface originally being covered by cartilage, said method comprising

> - providing a multispectral image of said bone surface,

- determining at least one region of interest in the bone surface,

- establishing at least one property of

1 ) at least one spectral component or - 2) a combination of spectral components,

- from the image of the region or regions of interest, and

- using at least one property for establishing the quantity measure for joint destruction.

The method allows a sensitive and reproducible detection of joint destruction. Fur- thermore, the method is preferably capable of being used in animal models by establishing an image feature reflecting changes of relevance to disease progression and treatment effect in individual animals. Accordingly, the spectral component(s) established should be selected to comprise information that may be correlated to the joint destruction so that the quantity measure allows classification of individual ani- mals with respect to joint destruction.

In another aspect the method according to the invention allows quantification of the joint destruction, and accordingly the invention further relates to a method for quantifying joint destruction on a bone surface in an individual animal, wherein said bone surface is a bone surface originally being covered by cartilage, said method comprising

- providing a multispectral image of said bone surface,

- determining at least one region of interest in the bone surface, - establishing at least one property of

- from the image of the region(s) of interest, and

using said at least one property for establishing a quantity measure for joint destruction,

- comparing said quantity measure to at least one standard for said quantity measure, thereby quantifying the joint destruction.

By establishing a quantity measure for an individual animal it is possible to determine the joint destruction on individual level instead of only being able to determine joint destruction on group level, wherein the group comprises several animals.

Standards for the quantity measure may be standards for healthy animals as well as for animals having various degrees of disease. By using the standards for comparison the joint destruction may be quantified.

Furthermore, the invention relates to a system for carrying out the methods herein. Accordingly, a further aspect of the invention relates to a system comprising

- a sample area,

- means for acquiring a multispectral image

- means comprising algorithms for establishing at least one property of a spectral component from the image and means for establishing a quantity measure from said at least one property, and optionally means for comparing said quantity measure with one or more standards. Description of Drawings

Fig. 1 shows an example of determination of the Region-of-interest

Fig. 2 shows the tibial bone from two different animals 6 weeks and 21 weeks, respectively, indicating the joint destruction due to aging.

Fig. 3 shows, in the top row, the R, G and B components of the RT02 image. In the lower row, the Y, I and Q components are shown for the same image. Note that the Q component is a contrast which appears very bright at areas of strong (blue) coloration, whereas it is darker at areas of white coloration.

Fig. 4 The H, S, and I colour components are shown for the RT02 image. Again, it is seen that areas of strong blue coloration are enhances in the saturation component (middle).

Fig. 5 shows a graphical representation of the Q-component in relation to age of right tibia.

Fig. 6 shows a graphical representation of the Q-component in relation to age of average of right and left tibia.

Fig. 7 shows a schematic drawing of experimental setup.

Fig. 8 shows an image of right tibia with ROI mask defining the medial side. The closed contour (black and white dashed line) defines the Region Of Interest. Only the pixels (picture elements) inside the closed contour are used in the future analysis.

Fig. 9 shows air bubble and other artifacts excluded by "holes" in the mask.

Fig. 10 Schematic illustration of focal plane and Region Of Interest. Left: side view of tibia. Right: top view. Fig. 11 shows a diagram of the path of the light ray from the light source to the image.

Fig. 12 shows the electromagnetic spectrum.

Fig. 13 shows green chromaticity for young and old animals.

Fig. 14 shows a pixel intensity histogram.

Fig. 15 shows a mode for two different histograms.

Fig. 16 shows visualization of Mode Density Level.

Fig. 17 shows examples of high and low entropy.

Fig. 18 shows average trichromatic histograms from young/old animals, R,G,B.

Definitions

Chromaticity components: The term means each of the primary colour components normalized with the intensity.

CMY: The term means Cyan, Magenta and Yellow being the secondary colours of light or, alternatively, the primary colours of pigment.

Fluorescence component: The term means a component arising from analysis of excitation wavelength and emission wavelength.

HSI: The term means colour attributes Hue, saturation and intensity, wherein Hue is a colour attribute that describes a pure colour, whereas saturation gives a measure of the degree to which a pure colour is diluted by white light.

Image: The term image is used to describe a representation of the region to be ex- amined, i.e. the term image includes 1 -dimensional representations, 2-dimensional representations, 3-dimensional representations as well as n-dimensional representations. Thus, the term image includes a volume of the region, a matrix of the region as well as an array of information of the region.

Joint destruction: By the term joint destruction is meant destruction of one or more of the components of a joint, such as the cartilage, the synovialis membrane, and the bone. In a preferred embodiment the term relates to destruction of the cartilage on a bone surface or of the underlying bone surface as well.

Multispectral image: The term refers to an image where reflectance and/or transmit- tance have been registered for multiple wavelengths.

OR: Acquiring optical images in more than one spectral band of the same physical area and in the same scale.

NIR components: The term means near-infrared components.

IR components: The term means infrared components.

Quantity measure: The term means a measure for the quantity of joint destruction.

RGB: see Red-green-blue image.

Red-green-blue image: The term relates to the image having the red channel, the green channel and the blue channel, also called the RGB image.

Region-of-interest (ROI): The term is used in its normal meaning, i.e. the region in the image that is relevant to include in image analysis.

YIQ: The term refers to the luminance (Y) and the colour information (I and Q).

Detailed description of the invention

It is an object of the present invention to provide in an automated manner a measure reflecting the gross-morphological changes seen in destructed joints, such as changes as a consequence of cartilage destruction. Accordingly, it is an objective to obtain a quantitative measure of quantity of joint destruction. The quantity measure thus includes one or more of the following types of information:

• Cartilage volume

• Cartilage surface lesions, i.e. presence or absence of surface lesions

• Area of cartilage lesions

• Area of bone without cartilage cover

• Area of bone destruction • Subchondral sclerosis

In a preferred embodiment the term quantity measure in particular includes one or more of the following:

• Cartilage volume

• Cartilage surface lesions

• Area of cartilage lesions

In a more preferred embodiment the term quantity measure in particular includes at least:

• Cartilage surface lesions

In Fig. 2 the cartilage changes during aging are shown.

The present invention offers the possibility of establishing the quantity measure for an individual animal, i.e. for an individual animal it is possible to define whether said animal has any joint destruction or not evaluated in relation to the quantity measure. This is in contrast to prior art methods wherein it has been possible only to describe a group of animals, and not the individual animal.

The joint and thereby the bone surface may be any joint related bone surface in a mammal, including humans, such as bone surfaces selected from tibial bone surface, femoral bone surface, ulnar bone surface, humeral bone surface, mandibular bone surface, metacapal bone surface or metatarsal bone surface. In the present context a preferred bone surface is the tibial bone surface and/or the femoral bone surface, since the knee joint often exhibits the destruction if the animal is suffering from joint destruction.

In the knee the bone surface may be selected from one or more of the medial femoral condyle, the lateral femoral condyle, the medial tibial plateau, and the lateral tibial plateau.

The individual animal may be any animal in which it may be of interest to evaluate joint destruction. In particular the animal may be selected from mouse, rat, guinea pig, rabbit, goat, sheep, dog, horse, cow, cat, and human being, such as selected from mouse, rat, and guinea pig.

In particular the method is useful for monitoring the effect of treatment during studies in pre-clinical phases of drug testing and therefore the method is especially suitable for carrying out on small animals, such as mouse, rat, and guinea pig. Such animals are normally used in pre-clinical studies for evaluating the effect of a drug candidate on a mammal. For pre-clinical studies the present invention offers the possibility of providing quantity measures useful for the data handling and correlation in the studies.

The method may be carried out on bone surface in vivo or ex vivo. In particular for in vivo studies or diagnosis the method may be applied to humans. In vivo imaging may be conducted through an arthroscope for example.

Also the image may be acquired by imaging ex vivo on the exposed joint surface. In particular when using animals in (pre)clinical phases it is preferred to acquire the image ex vivo of the exposed joint surface.

Region of interest

During preparation of the bone surface before providing the image the bone surface may be damaged due to the handling as such. Furthermore, the acquisition of image may introduce artefacts to the image, such as poorly illuminated areas, and areas out of focus. Since the captured images of the joints may contain artefacts such as air-bubbles, dirt, torn off cartilage, etc, it is advantageous to define a so-called Region Of Interest (ROI) in the images. Fig. 1 shows an example of determination of a region of inter- est. The ROI is a mask that may be drawn by a human operator in order to specify which parts of the image should be included or excluded in the degradation analysis. In another embodiment an automatic tool, such as a software tool may conduct the masking.

The general idea is to include only areas with relevant information and to exclude areas with artefacts. If, e.g., the cartilage has been torn off during the manual handling of the bone, the bone exposure in the area where the cartilage has been removed is not due to the disease process and should therefore have no influence on the cartilage degradation measurement.

Fig. 8 shows how the mask is used to define the ROI so that only the interesting part of the image (the medial side of the right tibia) is included in the analysis. The masks can also be used to exclude artefacts in the image, e.g., air-bubbles (see Fig. 9). In these cases, the "holes" inside the outer mask line are excluded from the analysis.

It is furthermore an advantage to include only areas that are well illuminated, well focused and representative of the entire bone surface.

Multispectral image

The multispectral image of the region of interest may be composed of any type of images of the same physical region, such as photograph, digital images in terms of an image presented on a CCD array or the like.

In particular the multispectral image is a colour image, a grey-tone image, a fluorescence image, an infrared image, or a near-infrared image or a combination thereof.

It is preferred that the image is a digital image in order to facilitate the subsequent image processing when establishing the quantity measure. Spectral component

The purpose of a colour model is to facilitate the specification of colours in some standard and generally accepted way. In essence, a colour model is a specification of a 3-D coordinate system and a subsystem within that system where each point is represented by a single point. Joint destruction changes will manifest themselves in the individual colour components. In the following, the colour components of a number of different colour models are described.

Owing to the structure of the human eye, all colours are seen as variable combinations of the so-called primary colours red (R), green (G), and blue (B).

A colour image is a collection of intensity values in the red, green and blue band, defined on a rectangular grid of pixel locations, (x, y) e [0, ... , M - 1; 0,... , N - i], where M and N are the numbers of rows and columns in the image respectively. Thus, R(x, y) denotes the magnitude of the red intensity value observed at location

(x, y) in the image.

In the images acquired using a digital camera, intensity values will typically range from 0 through 255 in the individual colour bands. In the following, however, it is assumed that the intensity values ranges from 0 through 1 , which is easily accomplished by e.g. dividing each intensity value in the individual colour bands, with the maximum intensity value observed across all colour bands.

The spectral component may be any spectral component of the multispectral image having information related to the joint destruction. In particular the spectral component may be at least one spectral component selected from primary colour components, chromaticity colour components, YIQ components, HSI components or from combinations of two or more of these.

In another embodiment at least one spectral component is selected from IR components), NIR component(s) or fluorescence component(s) or a combination of two or more of these. Furthermore, a combination of two or more spectral components from at least two of the following components may be preferred: from primary colour components, chromaticity colour components, YIQ components, HSI components, IR components), NIR component(s) or fluorescence component(s).

Sometimes, the normalized RGB colour model is considered:

R(x, V) r(^χ,y) =

R(x, y) + G(x, y) + B(x, y)

G(x, Y) g(^χ,y) = R(x, y) + G(x, y) + B(x, y) (x, y) b(^χ, y) = R(x, y) + G(x, y) + B(x, y)

Cyan, Magenta and Yellow are the secondary colours of light or, alternatively, the primary colours of pigment. For example, when a surface coated with cyan pigment is illuminated with white light, no red light is reflected from the surface. The conversion is very simple:

C(x,y) = l - R(x, y) M(x, y) = l - G(x, y)

From these equations, it is clear that there is no independent information in the CMY colour model that could not be obtained in the RGB model.

The YIQ model is designed to take advantage of the human visual system's greater sensitivity to changes in luminance than to changes in hue or saturation. The principal advantage of this model is that the luminance (Y) is decoupled from the colour information (I and Q). Thus the colour content can be characterized independently from changes in the luminance components that are attributable differences in lighting conditions.

The RGB to YIQ conversion is defined as: ^rY ^χ, _yy fθ.299 0.587 (κ^(χ, yT i(^χ, y) 0.596 -0.275 G(x, y) Q(^χ, y) 0.212 -0.523

B(^χ, y)_y

In Figure 3 the R, G, B and Y, I, Q colour components are shown for image RT02. Both the I and the Q components are contrasts between colours.

Hue is a colour attribute that describes a pure colour, whereas saturation gives a measure of the degree to which a pure colour is diluted by white light. The HSI colour model owes its usefulness to two principal facts. First, the intensity component, I, is decoupled from the colour information in the image. Second, the hue and saturation components are intimately related to the way in which human beings perceive colour and are called chromaticity. The I component of the HSI model is different from that of the YIQ model.

The RGB to HSI conversion is defined as:

^f

An example of the colour components is shown in Figure 4.

By IR components is meant infrared components, defined as shown in Figure 12.

By NIR components is meant near-infrared components, which refers to that part of the infrared spectrum that is closest to visible light (see Figure 12).

In a fluorescence model the fluorescence components are most frequently the intensity as a function of excitation wavelength and/or the emission wavelength. How- ever, any other information contained in photoluminescence may be obtained from the sample such as fluorescence lifetime, phosphorescence intensity, phosphorescence lifetime, polarization, polarization lifetime, anisotropy, anisotropy lifetime, phase-resolved emission, circularly polarized fluorescence, fluorescence-detected circular dichroism, and any time dependence of the two last mentioned parameters.

These components may be used individually or a combination of two or more of these components may be used when establishing the property.

In one embodiment the at least one spectral component is selected from primary colour components, such as wherein the at least one spectral component includes the green colour component, or wherein the at least one spectral component includes the blue colour component, or wherein the at least one spectral component includes the red colour component, or a combination of two of these.

Property of spectral component

The property of the spectral component(s) may be a function of spectral bands from any of the spectral properties as defined above.

Accordingly, in one embodiment the at least one property is selected from first order statistics or second order statistics within a given Region of Interest for said spectral component. The first order statistics may be any useful statistics. In particular the first order statistics as measured on a histogram are selected from one or more of the features: the mean, median, mode, intensity, mode density level, inverse mode density level, standard deviation, coefficient of variation (CV), variance, entropy, skewness, kurtosis, or from a function of one or more of said features. More particularly the first order statistics as measured on a histogram are selected from one or more of the features: the mean, the mode, the mode density level and the entropy, or from a function of one or more of said features.

An image histogram provides a plot of the distribution of pixel values in the image. The histogram may be created by splitting the dynamic range of the pixel intensities into bins of equal width and count the number of pixels with intensities inside the individual bins. This can be written as: p H(i) = ∑B_t(I_p) , i= ..Number of bins, /_p=pixel value of pixel p,

P where:

For a simple pixel intensity histogram as the one shown in Fig. 14, several features may be of interest to describe the colour band.

Examples of preferred features are the mean value, the standard deviation, the CV, the skewness, the inverse mode density level, and the entropy. All these features can be calculated based on any single band in one of the colour models. For some features, the Image pixel values are used directly, for some, the histogram density values are used instead. The basic parameters which are used are described in the following.

Basic parameters for calculation

Pixel value = l(x,y)

Mask value = M(x,y) (0 for outside mask, 1 for inside mask) Number of image rows = r

Number of image columns = c

Number of pixels inside mask = N

Value of histogram column = H(i)

Number of pixels in histogram column = Ni Number of histogram columns = M

These parameters are used to calculate the colour features as the following formula shows:

Mean

Standard Deviation

Skewness γ = ■∑∑M(x,y)(I(x,y) -μγ

Nσ- y=\ x=l

Inverse Mode Density Level (IMDL)

In the following, it is assumed that the calculations are based on normalized images. Thus, the histogram x-axis will describe pixel values between 0 and 1.

The position along the x-axis of the maximum of the histogram defines the mode value, i.e. the most frequent value in the image. For a completely symmetrical distribution, the mode is equal to the mean and the median. But as seen in Figure 15, distributions can be skewed. The height of the mode value is closely (inversely) related to the standard deviation, but independent of the location (level) of the values.

The IMDL is determined from the observed histogram where each pixel is placed in the bin corresponding to its intensity. From a practical point of view, this is quite easy as each value lies in the interval [0,1] . If the histogram has M bins, the i'th bin is obtained as:

/ - 1

H(ϊ) =

N N

For each pixel the appropriate bin is incremented, the number of pixels in each bin is referred to asN, , and the corresponding density is obtained as

MDL = M ^N' 1=1

The calculation is visualized in Figure 16. To obtain the IMDL, the MDL is simply inverted using IMDL = 1/MDL. Entropy

The entropy is calculated from the normalized histogram which has the area 1. The histogram is normalized by:

^Hn ^l) - Xj

∑H(i)

(=1

The entropy is then calculated as

M E = -∑H„ ( 1n(H„ ( )

(=1 where 0 < E < In .

The entropy measures the uniformity of the histogram. The lowest entropy values will be obtained when the pixel intensity is concentrated in few values, and the highest values will be for a uniform histogram where each value is equally represented (see Figure 17).

In one embodiment, the entropy of a histogram may be used as a measure of joint destruction:

where d_t is the histogram density of the i'th bin of a histogram with N bins.

The entropy reaches its maximum level (log(Λ/) ) for a completely uniform distribution, and its minimum (zero) for a histogram where all observations are located in one bin - i.e. for a completely homogenous surface. This property is completely independent of the location of the distribution, which is indeed a desirable property for a feature that should be independent upon phenomena as bleaching and absolute staining intensity, and only reflect 'spread' of the intensities.

Also, the second order statistics may be any useful statistics. In particular it relates to second order statistics selected from one or more of the features: Auto- covariance, autocorrelation or statistics made on parameters in the frequency domain, or from a function of one or more of said features. Furthermore, the property of at least one spectral component may comprise at least one texture feature, such as a texture feature(s) selected from Gray-Level Cooccurrence Matricer (GLCM), Gray-Level Run-Length (GLRM), and segmenting of the image in at least two classes.

Preparation of joint

The spectral component(s) may be obtained from the image of an unprepared joint or from a joint having been subjected to one or more preparation steps.

The unprepared joint may be illuminated with polarised light or with excitation light, IR or NIR light, in order to establish the quantity measure.

When preparation steps are conducted, these may typically include at least one type of staining or labelling. Accordingly, the invention relates to a method, wherein the bone surface is stained before the image is provided. Colour changes reflecting joint degradation are more pronounced in some spectral bands than in others.

Prior to acquisition of the image, the bone surface may thus be stained by chemical or biological dyes or suspensions of particles. It is preferred that the staining has specificity for or binding capability to cartilage on the bone surface; the invention thus includes a staining being cartilage staining. The staining may be for cells or matrix, such as collagen and proteoglycan.

However, as an alternative or in addition thereto, the staining may be staining of bone or staining of synovium.

Also combinations thereof may be useful, such as using staining cartilage and/or underlying bone.

In order to ensure a homogenous staining, in particular in small joints, such as a joint having its largest dimension less than 3 mm, for example mouse joints (< 3 mm), the particle size of the staining is important. The particle size of the staining particles is preferably in the range of from 10 nm to 1000 nm, especially 50 nm to 600 nm, such as 80 nm to 200 nm. The staining is preferably a histological staining, such as Alcian Blue, Alizarine Blue, Hematoxiline, Toluidine Blue, and/or Blue Indian Ink, Black Indian Ink, Graphite powder, more preferably Toluidine Blue with a particle size as defined above.

In another embodiment the staining is selected from fluorescent molecules coupled to antibodies, said antibodies being directed to the cartilage or bone.

Depending on the staining selected the staining may be conducted as a multi-step staining or a one-step staining.

In order to obtain more detailed information from the joint being imaged, the staining may be a staining with two of more different stainings, for example selected from the staining mentioned above.

After staining the bone surface may be subjected to various treatments for fixing the staining before imaging. In one embodiment this includes that the stained bone surface is incubated before providing the image.

The staining preferably aids in providing the quantity measure. Disease leading to cartilage degradation may be observed with "eroded" cartilage. For staining with Toluidine Blue, this process may for example be manifested in a blue stained bone surface as 'whitish' patches on the surface of the stained joints and in some cases, when the underlying bone surface is almost completely exposed, as a white back- ground with a blue texture superimposed. This is in contrast to the homogenous blue surface observed for very young animals where the cartilage is intact.

All the primary colours (RGB) have information about joint degradation; however, using Toluidine blue, it is preferred to use the green chromaticity effectively, captur- ing relevant changes in coloration. The green chromaticity is defined as a measure of the relative amount of green intensity in any given pixel:

g = R + G + B In the less degradated joints, the green chromaticity values are typically quite homogenous and therefore focused around the mean (or mode) of the distribution. As the degradation progresses, the chromaticity values are spread across a larger range of values.

Establishment of quantity measure

For young animals, the colour is fairly homogenous across the mask area, for example visualised through staining. As the disease progresses, the cartilage wears thinner. From visual inspection of the specimens, it would even appear that the underlying bone is even exposed in some cases. The result of this process is that the coloration of the specimens becomes less homogenous and the histogram density is spread over a wider range of chromaticity values. This phenomenon is expressed in the property of the spectral components established herein.

Accordingly, the quantity measure for joint destruction is established using the established property, for example using the mode density level and/or the entropy. For example, the quantity measure may be equal to the mode density level or the entropy.

Quantity measure standards

The quantity measure obtained according to the invention may be used directly as the information of the joint destruction. In one aspect of the invention the quantity measure obtained is compared with one or more standards for quantity measures for the animal tested. Accordingly, in one aspect the invention relates to a method for quantifying joint destruction on a bone surface in an individual animal, wherein said bone surface is a bone surface originally being covered by cartilage, said method comprising

- providing a multispectral image of said bone surface,

- determining at least one region of interest in the bone surface,

- establishing at least one property of - 1 ) at least one spectral component or

- 2) a combination of spectral components,

- from the image of the region(s) of interest, and

The standards may be obtained by providing a quantity measure for a plurality of joints and correlating the quantity measure with "manually" obtained morphological information from the same joint.

In a preferred embodiment the at least one standard for said property(ies) is selected from a standard for a healthy control bone surface, a standard for age- related cartilage degradation, and a standard for disease-related degradation.

In yet another aspect the invention also relates to a plurality of standards, each standard comprising a quantity measure for the correlated state of disease or aging.

Applications

The method and the system according to the invention may be used for evaluating joint destruction in relation to any disease or disorder as well as in relation to any treatment regime tested. In particular the invention is useful to evaluate cartilage degradation due to osteoarthritis.

However, the method and system may also be useful for evaluating cartilage degradation due to rheumatoid arthritis, degenerative joint disease, inflammatory joint disease or due to traumatic injury. System

The establishment of the quantity measure may be obtained using any suitable equipment. A preferred system is described in the following. Accordingly, in another aspect the invention relates to a system for establishing a quantity measure as discussed above.

In one embodiment the image acquisition part of the system is depicted in Fig. 7, showing a sample area, wherein the bone may be positioned in clay, the camera connected to a microscope and a light source. Furthermore, the system comprises a system for processing the data, such as a computer having a memory storing algorithms for carrying out the method according to the invention.

In a preferred embodiment a light source is arranged, so that the bone surface is exposed to a light source during acquisition of the image.

Image acquisition

Teksten kan ikke spasnde over mere end en Iinjellmage acquisition refers to the process of capturing digital images of the specimens in a study. This must follow a well- defined procedure in order to enable reliable subsequent analysis of the images.

Therefore, a calibration during image acquisition may be desirable. The steps of image acquisition may be divided into:

• Specimen handling

• Adjustment of camera, microscope and microscope

• Image capture procedure

In one embodiment of the invention the image is a calibrated image, such as a calibrated digital image.

The calibration may be conducted either during the acquisition of the image or subsequently before image processing. It is preferred that the calibration is conducted during the acquisition of the image. The bones are preferably held in a position allowing the joint bone surface to be held in a position approximately parallel to the image acquisition. Furthermore, the bone surface is preferably submerged in water to avoid reflections.

Adjustment of camera, microscope and illumination settings should preferably be conducted each time one of the parts has been reset or switched off.

When measuring features based on the colour content in images, the illumination conditions at the time of capture have an influence on the features. To enable comparison of images captured at different times, it is necessary to compensate for such different illumination conditions, if any. Usually, the illumination conditions are kept as constant as possible but the illumination may drift due to reasons like wear of equipment or non-constant characteristics of the light source.

The basic idea behind colour calibration is to use a series of so-called calibration targets. Before the actual capture session of the images of interest (specimen images), a series of images are captured of the calibration targets. These calibration images are used to "remember" the illumination conditions at the capture session in progress.

One calibration target may be a piece of paper with low reflectance and constant, known colour. This means that the true RGB values of the piece of paper are known and that this colour is uniform all over the sheet.

Another preferred calibration target is a metal alloy plate with layers of porcelain paint, the alloy for example being Orion WX white gold reduced (dental ceramic alloy: Au=52.0, Pd:38.0; ln=8.2, Ga=1.6, rest Ag, Re). For example 5 alloy plates may be used having the following colours: Dark grey, dark brown, black, light grey and light brown.

By capturing calibration images of a number of differently coloured targets it is possible to measure the RGB values as they appear under the current conditions. When compared to the true RGB values it is possible to estimate a set of parameters de- scribing the current illumination conditions. These parameters determine how much the current conditions vary from some well-defined standard conditions. It is then possible to transform the colours of the newly captured images according to these parameters so that the images are standardized and the differences in illumination conditions at the current capture session are compensated for. This process is re- ferred to as colour calibration.

Colour calibration allows for comparison of images captured at different illumination conditions. This is especially useful in the comparison across studies captured at different points in time, where the illumination conditions may have changed.

It is preferred that a calibration image of each of 3 (three) calibrations must be captured when the setup has been changed or switched off. The calibration images should be captured immediately before the capture session of the specimen images.

The system model

In one embodiment of the system, the theoretical path of a ray of light from the light source to the image was examined.

The model used for the colour calibration is the following:

0 = α + βW , (3.1 )

where O is the observed pixel value in the image, L is the intensity of the light from the light source, α is an offset parameter, β is a gain parameter and γ is a parameter describing the non-linearity in the system. These three parameters need to be estimated from corresponding measured and known intensity values.

The following describes the complete system model in more detail.

The light is emitted from the light source and is reflected by the specimen surface into the microscope optics. Behind the optics, a camera captures the light using for example a CCD-chip. The CCD-chip collects the light at each cell in a regular lattice and converts the amount of collected light to a digital number (pixel value) for each cell. The CCD-chip has a non-linear characteristic, described by a so-called gamma value. The output of the camera can be adjusted (offset, gain and gamma).

The diagram in Fig. 11 shows the path of a light ray from the light source to the im- age and the transformations it undergoes on the way. The total transformation can be described in a model as

0 = a + βL^r , (3.2)

A general model for the camera setup describing the transformation of the incoming light to pixel values on the CCD chip is described. This model is extended to model variations between capture sessions.

The general idea is that the "true" intensity, T is transformed by the camera according to the model:

O = f(T) = a + βT^γι (3.3)

where a is the brightness (offset), β is the contrast (gain) and γ_ is the CCD-chip gamma parameter. O is the observed pixel value corresponding to the true value T .

By first tuning the illumination conditions so that the images captured (images of specimens) appear in a satisfactory way (utilization of the entire dynamic camera range without saturation) and then capturing a number of different calibration sheets (at least 3) it is possible to estimate the parameters a, β and γ_ . Their values de- scribe an optimal configuration of the illumination conditions.

The rvalue of the camera is constant (same as in (3.3)), but the illumination (brightness and contrast) might change slightly between capture sessions. In addition, it may be desirable to have a different gamma for the display than for the camera. A model for the transformation of observed image pixels under possibly slightly differing illumination conditions is then

o = g(t) = a + btⁿ , (3.4) where only a and b need to be estimated.

In order to transform the observed values, from slightly different conditions to the common setting o , where the visual appearance is known to be good the inverse of g , g^~l is used to transform the observed values to the T space (true NCS values).

From here, the O space can be reached by transforming the values according to f .

The inverse of g(t) is

and the total transformation from o to O is described by

For γ_ = γ_ (3.6) reduces to a simple linear model: O = c + do (3.7) with c = α - ^- and d = ^- .

Accordingly, first, the parameters in f (-, , β,χ) , (see Eqn. (3.3)) are determined using 4 to 6 calibration targets. This is done once and for all and describes the relation between the true NCS values and an optimal illumination configuration providing pleasant visual appearance of the images.

At the beginning of each capture session, images are preferably captured of 3 calibration targets and these are used to determine the parameters in g (-, a,b) (see

Eqn. (3.4)). This transformation serves to compensate for small uncontrolled changes in the illumination conditions.

The specimen images are then transformed according to Eqn. (3.6) and this is referred to as the colour calibration. Examples

Example 1

A quantity measure based on standard deviation

The data analysis provided herein is based on microscope colour images from a total of 17 specimens, obtained from both left and right tibia. The joints were stained with toluidine blue.

The data were obtained from two strains of mice:

The ages of NMRI mice were all provided as intervals (6-8 weeks), and were in- eluded in the analysis at 7 weeks of age. Each specimen was given a unique study identification number from 01-17.

Images of the stained joints were acquired using a Leica M 651 microscope system with a digital Leica DC 200 camera-back connected to Matrox-Meteor frame- grabber. The Leica KL 1500 LCD was used as a light source. The images were stored as 1030x1300 TIFF RGB files, with 24 bits per pixel (8 bits per colour band).

Table 1 below shows the used settings of the adjustable parameters.

Fig. 7 shows a schematic drawing of the experimental setup for microscopy image capture of a mouse tibia.

Tibial Measurements

The RGB Colour Models

Regression analysis: For both the left and the right tibia, the strongest association with age of the animals was found for the standard deviation of the normalized green band, σ_g , when compared to other texture statistics computed in the RGB and normalized RGB models. For this texture statistic, the correlation with age was found to be 0.74 and 0.77 for the left and the right tibia, respectively.

T-test: The hypotheses of equal means for young and old animals were rejected for both left and right tibia, at a high significance level.

Left & right average: A correlation with age of 0.88 was observed. The hypothesis of equal means in young and old animals, using a Students T-test, is rejected at a significance level of 0.0001. Although it may not be feasible to stain both joints in practical experiments, it seems clear that an average over right and left tibia significantly improves the results, as one would expect.

YIQ and HSI Colour Models

Regression analysis: For both the right and the left tibia, the strongest association with age of the animals was found for the standard deviation of the Q component of the YIQ model, σ_Q , when compared to other texture statistics computed in the YIQ and HSI spaces. For this texture statistic, the correlation with age was found to be 0.84 and 0.805 for the left and the right tibia, respectively. These correlations are higher that those found for the standard deviation of the normalized green colour band.

T-test: The hypothesis of equal means for young and old animals is rejected for left and right tibia, at a high significance level. As mentioned above, it is an important observation that the same texture statistic is found optimal in both right and left tibia.

Left & right average: The correlation to age was estimated for an average over left and right tibial measurements, and was found to be 0.95. The hypothesis of equal means in young and old animals, using a Students T-test, is rejected at a significance level of 0.000012.

For the tibia, several features are found to have a strong association with age and at the same time capable of discriminating well between young and old animals based on the observed texture in the images. In particular σ_Q was identified as promising texture statistic.

In conclusion, the results presented herein indicate that it is possible to construct generally useful quantity measure capable of characterizing the progression of os- teoarthritis quantitatively, based on colour images of stained mouse tibia. Example 2

A quantity measure based on mode density level and entropy

Assessment of feature performance

A number of properties must be tested for a feature candidate to be accepted as a measure of osteoarthritis (OA). The tests concern both correlation to age (tested using regression analysis in the linear model), the ability to distinguish between treated and untreated animals (tested using analysis of variance), and good repro- ducibility (tested using the coefficient of variation - CV). The tests have all been performed in S-Plus. All the joints have been stained with Toluidine blue.

A good feature should display the following characteristics.

• Treatment effect 04: Since other tests have shown that the drug tested in this study does display a treatment effect, the feature must be able to distinguish between the treatment groups of STR1 N-04. The test results are found under: 04Tr Z and 04Tr p. • Treatment effect 09: Since the drug used in this study does not display any treatment effect, the feature should not separate the treatment groups of STR1 N-09 significantly. The test results are found under: 09Tr Z and 09Tr p.

• Treatment effect 12: Since the drug used in this study does not display any treatment effect, the feature should not separate the treatment groups of STR1 N-12 significantly. The test results are found under: 12Tr Z and 12Tr p.

• Age correlation (ADEP): The feature must display good correlation to age in the small age dependency study. The test results are found under: ADEP R and ADEP p.

• Age correlation (AIIAge): The feature must display good correlation to age in a study comprised of the STR1 N-13 study and the control groups from STR1 N-04

STR1 N-09 and STR1 N-12. The test results are found under: AIIAge Z and All- Age p.

• Age Trend: The studies from age correlation 1 and 2 must display the same correlation trend in a plot. The assessment of this condition is done by a visual inspection of the plot. • Biomarker correlation: The feature should display some relationship to the cartilage turnover as measured by the biomarker HPcreatmme (HPC). However, since the biomarker is a measure of degradation speed, and the staining method is used to measure the cumulated degradation, the correlation is not necessarily very high. It is expected that any correlation will be more significant on studies with treatment effect (STR1 N-04), since such studies will have a wider range of joint degradation levels. Also, the STR1 N-09 study exhibits no treatment effect and is not blinded, which will increase noise and reduce correlation. The test results are found under: 04Tr.Bio R, 04Tr.Bio p, 09Tr.Bio R, and 09Tr.Bio p. • Histology correlation. The most promising features are tested with respect to the correlation to histology. However, no large correlation can be expected to this measure. The histology measure displays some correlation to the biomarker values and a weak discrimination between the treatment groups. Also, the histology is a very local measure, and has been sampled from the opposite leg of the animal. The test results are found under: 04Tr.Hstl R, 04Tr.HstI p,

• Reproducibility: The feature must display low precision error when tested on the reproducibility study. This study enables calculation of inter-operator variance and variation between image takes. The test results are found under: ROI A - CV and ROI B - CV (effect between takes) and Takel - CV and Take 2 - CV (effect between operators).

Two features are analysed in this example: mode density level and entropy.

Mode Density Level

In the development of a better feature, systematic changes in the histogram for the tri-chromatic colour coefficients are considered. The animals in the STR1N-13 are 6 weeks old, whereas the animals in the STR1 N-09 study are 12 weeks old. From the STR1N-09 study, only the control group is included. Figure 18 shows a comparison of the average histograms of the young (6 weeks) and the old (12 weeks) animals for the three chromaticity values (red, green and blue).

It is seen that the average histogram for the green chromaticity exhibits a significant difference between young and old animals. The explanation for this systematic dif- ference is as follows: For young animals, the staining and thereby the colour is fairly homogenous across the mask area. As the disease progresses, the cartilage wears thinner. From visual inspection of the specimens, it would even appear that the underlying bone is exposed in some cases. The result of this process is that the coloration of the specimens becomes less homogenous and the histogram density is spread over a wider range of chromaticity values. This corresponds well with the reasonable results obtained with the standard deviation of the trichromatic green channel.

The position of the highest peak in the pixel intensity histogram is also named the mode. The mode can be used to form a measure of the pixel spread by dividing the area of the column at the mode position with the area of the entire histogram. This value is inverted to give a positive correlation to the spread.

Mode Results

The MDL feature yields quite consistent results for the data material, as is seen in Tables 2 and 3. The test has been performed for a number of different combinations of bin number and mode width, all of which produce good results. As for the standard deviation of the trichromatic green channel, the correlation is to the biomarker of STR1N-04 is good, the correlation to the biomarker of STR1N-09 is poor, and so is the correlation to the histology measure of STR1 N-04.

Table 3: Mode Density Level. Biomarker and histology correlation. Entropy

Another preferred method of estimating joint degradation is to measure the spread of the pixels in the intensity histogram of the trichromatic green channel. The stan- dard deviation provides usable results but more consistent results through all the different studies could be hoped for. The IMDL feature performs better, but since the mode feature only uses the height of the mode and the histogram area, it discards all information about the size of the rest of the histogram bins. It is desirable to use as much of the relevant information as possible.

The entropy provides a method of measuring the spread of the pixels in the intensity histogram, which is completely independent of the mean intensity of the image and which includes the information about the size of the individual histogram bins. The entropy is calculated as the sum of calculated values for the individual histogram bins as:

Λ.

E = -£H )ln(H( ) ι=l

where Η(i) is the number of pixels in the i'th bin of the intensity histogram, and M is the total number of bins in the histogram.

The entropy will be influenced by the number of bins. A high number of bins will result in higher entropy than a low number of bins. Thus, different bin numbers must be tested to find the optimal setting for the feature calculation. A number of different settings from 64 bins (B64) to 65536 bins (B65536) have been tested.

Entropy Results

The results of the testing of the entropy feature can be seen in the following tables. The results are good. The feature correlates very significantly to age, and distinguishes significantly between the treatment groups in STR1 N-04. Also, it does not discriminate significantly between the treatment groups of STR1 N-09 and STR1 N- 12 where no treatment effect is present (Table 4). The correlation to the biomarker of STR1 N-04 is very good, whereas the correlation to the biomarker of STR1N-09 is poor (Table 5), but as mentioned in the section on feature usability, this is to be expected. The correlation to the histology measure of STR1 N-04 is also poor, as for the standard deviation of the green chromaticity. The reproducibility of the entropy calculation is one of the best which has been experienced, with CVs below 1% for the ADEP study (Table 6).

The results indicate that the best result is obtained for 65536 bins, that is, when the number of bins resembles the number of different pixel values which exist in the images (16 bits).

Table 4: Entropy. Treatment and age significance.

Table 5: Entropy. Biomarker and histology correlation.

Table 6: Entropy. Precision error for ADEP. Conclusion

The testing of a number of features in different colour representations shows that features describing the spread of the histogram for the trichromatic green channel provide the best description of cartilage degradation.

Among the three features standard deviation, inverse mode density level and entropy, the entropy is found to be the most suitable feature. By using the entropy as a measure of joint degradation, it is possible to detect the expected treatment effect in the three treatment studies. Furthermore, the feature is able to describe the expected effect of aging in the subjects, and a direct correlation to the biomarkers can also be detected.

Claims

1. A method for establishing a quantity measure for joint destruction on a bone surface in an individual animal, wherein said bone surface is a bone surface originally being covered by cartilage, said method comprising

- providing a multispectral image of said bone surface,

- determining at least one region of interest in the bone surface,

establishing at least one property of

- 1 ) at least one spectral component or

- 2) a combination of spectral components,

- from the image of the region or regions of interest, and

using said at least one property for establishing the quantity measure for joint destruction.

2. The method according to claim 1 , wherein the bone surface is selected from tibial bone surface, femoral bone surface, ulnar bone surface, humeral bone surface, mandibular bone surface, metacapal bone surface or metatarsal bone surface.

3. The method according to claim 1 or 2, wherein the bone surface is selected from medial femoral condyle, lateral femoral condyle, medial tibial plateau, and lateral tibial plateau.

4. The method according to any of the preceding claims, wherein the animal is selected from mouse, rat, guinea pig, rabbit, goat, sheep, dog, horse, cow, cat, and human being.

5. The method according to any of the preceding claims, wherein the animal is selected from mouse, rat, and guinea pig.

6. The method according to any of the preceding claims, wherein the animal is selected from human being.

7. The method according to any of the preceding claims, wherein the multispectral image is a colour image, a grey-tone image, a fluorescence image, an infrared image, or a near-infrared image or a combination thereof.

8. The method according to any of the preceding claims, wherein the image is a digital image.

9. The method according to any of the preceding claims, wherein the image is a calibrated digital image.

10. The method according to claim 9, wherein the calibration is conducted during image acquisition.

11. The method according to any of the preceding claims, wherein the bone surface is immersed into a liquid during image acquisition.

12. The method according to any of the preceding claims, wherein the at least one spectral component is selected from primary colour components, chromaticity colour components, YIQ components, HSI components or from combinations of two or more of these.

13. The method according to any of the preceding claims, wherein at least one spectral component is selected from IR component(s), NIR component(s) or fluorescence component(s) or a combination of two or more of these.

14. The method according to claim 12 or 13, wherein the at least one spectral component is selected from primary colour components.

15. The method according to any of claims 12-14, wherein the at least one spectral component includes the green colour component.

16. The method according to any of claims 12-15, wherein the at least one spectral component includes the blue colour component.

17. The method according to any of claims 12-16, wherein the at least one spectral > component includes the red colour component.

18. The method according to any of the preceding claims, wherein the property is a function of spectral bands from any of the spectral properties as defined in claims 12-17.

19. The method according to any of the preceding claims, wherein the at least one property is selected from first order statistics or second order statistics within a given Region of Interest for said spectral component.

20. The method according to claim 19, wherein the first order statistics as measured on a histogram is selected from one or more of the features: the mean, median, mode, intensity, mode density level, standard deviation, variance, entropy, skewness, kurtosis, or from a function of one or more of said features.

21. The method according to claim 19, wherein the first order statistics as measured on a histogram is selected from one or more of the features: the mean, the mode, the mode density level and the entropy , or from a function of one or more of said features.

22. The method according to claim 19, wherein the second order statistics is selected from one or more of the feature: Autocovariance, autocorrelation or statistics made on parameters in the frequency domain, or from a function of one or more of said features.

23. The method according to any of the preceding claims, wherein the property of at least one spectral component comprises at least one texture feature.

24. The method according to claim 23, wherein the texture feature(s) is selected from Gray-Level Cooccurrence Matricer (GLCM), Gray-Level Run-Length (GLRM) , segmenting of the image in at least two classes.

25. The method according to any of the preceding claims, wherein the bone surface is exposed to a light source during acquisition of the image.

26. The method according to any of the preceding claims, wherein the light source is polarised light.

27. The method according to any of the preceding claims, wherein the region of interest is determined by visual inspection before providing the image.

28. The method according to any of the preceding claims, wherein the region of interest is determined by visual inspection of the image.

29. The method according to any of the preceding claims, wherein the bone surface is stained before the image is provided.

30. The method according to any of the preceding claims, wherein the bone surface before providing the image is stained by chemical or biological dyes or suspensions of particlesτ

31. The method according to any of the preceding claims, wherein the staining is staining of cartilage.

32. The method according to any of the preceding claims, wherein the staining is staining of bone.

33. The method according to any of the preceding claims, wherein the staining is staining of synovium.

34. The method according to any of the preceding claims, wherein the staining is staining of cartilage or underlying bone, e.g. cells or matrix such as collagen and proteoglycan.

35. The method according to any of the preceding claims, wherein the particle size of the staining particles is in the range of from 10 nm to 1000 nm, especially 50 nm to 600 nm, additionally 80 nm to 200 nm.

36. The method according to any of the preceding claims, wherein the staining is selected from histological stainings.

37. The method according to any of the preceding claims, wherein the staining is selected from Alcian Blue, Alizarine Blue, Hematoxiline, Toluidine Blue.

38. The method according to any of the preceding claims 1-36, wherein the staining is selected from Blue Indian Ink, Black Indian Ink, Graphite powder.

39. The method according to any of the preceding claims, wherein the staining is selected from fluorescent molecules coupled to antibodies, said antibodies being directed to the cartilage or bone.

40. The method according to any of the preceding claims, wherein the staining is a multi-step staining.

41. The method according to any of the preceding claims, wherein the staining is a staining with two of more different stainings.

42. The method according to any of the preceding claims, wherein the staining is fixed before providing the image.

43. The method according to any of the preceding claims, wherein the stained bone surface is incubated before providing the image.

44. The method according to any of the preceding claims, wherein the cartilage degradation is due to osteoarthritis.

45. The method according to any of the preceding claims, wherein the cartilage degradation is due to rheumatoid arthritis degenerative joint disease, inflamma- tory joint disease or due to traumatic injury.

46. The method according to any of the preceding claims, wherein the image is acquired through an arthroscope.

47. The method according to any of the preceding claims, said method being conducted ex vivo.

48. A method for quantifying joint destruction on a bone surface in an individual animal, wherein said bone surface is a bone surface originally being covered by cartilage, said method comprising

- providing a multispectral image of said bone surface,

determining at least one region of interest in the bone surface,

- establishing at least one property of

- 1 ) at least one spectral component or

- 2) a combination of spectral components,

- from the image of the region(s) of interest, and

49. The method according to any of the preceding claims, wherein the at least one standard for said property(ies) is selected from a standard for a healthy control bone surface, a standard for age-related cartilage degradation, and a standard for disease-related degradation.

50. The method according to claim 49, further comprising one or more of the features as defined in any of claims 1-47.

51. A system for carrying out the method according to any of the preceding claims, said system comprising

- a sample area, >

- means for acquiring a multispectral image,

- means comprising algorithms for establishing at least one property of a spectral component from the image,

- and means for establishing a quantity measure from said at least one property,

- and optionally means for comparing said quantity measure with one or more standards.

52. The system according to claim 51 , further comprising at least one calibration target.

53. The system according to claim 51 or 52, wherein the image is a digital image.

54. The system according to claim 53, wherein the image is a calibrated digital image.

55. The system according to any of claims 51-54, wherein the bone surface is immersed into a liquid during image acquisition.

56. The system according to any of claims 51-55, wherein the bone surface is exposed to a light source during acquisition of the image.

57. The system according to claim 56, wherein the light source is polarized light.