GB2393062A - Image transmission system and method for determining regions of interest in an image transmission - Google Patents

Abstract

A method (400) for identifying a feature in an image includes the steps of defining (410) a plurality of saliency values of an image or set of images over multiple parameters and calculating (414) an entropy value for each of the saliency values over substantially each of the multiple parameters. One or more peak entropy values are determined (430, 432) for the image or set of images based on the calculated entropy values. A feature of the image or set of images is identified (454), based on the weighted peak entropy value. An image transmission device, image transmission unit and image transmission system are also described. This provides a method and apparatus for extending scale definition to multiple parameters, thereby generalising the structures that can be identified, for example, to identify anisotropic regions.

Description

GB 2393062 A continuation (74) Agent and/or Address for Service: Colin

Treleven European Intellectual Property Department, Midpoint, Alencon Link, BASINGSTOKE, Hants, RG21 7PL, United Kingdom

Image Transmission System And Method For Determining Regions Of Interest In An Image Transmission Field of the Invention

This invention relates to image modelling in an image 5 transmission system. The invention is applicable to, but not limited to, image modelling using multiple parameters to identify anisotropic features.

Background of the Invention

10 Future generation mobile communication systems are expected to provide the capability for video and image transmission as well as the more conventional voice and À. data services. As such, video and image services will become more prevalent and improvements in video/image À 15 compression technology will likely be needed in order to.

match the consumer demand within available bandwidth..

À À À.. Current transmission technologies, which are particularly À suited to video applications, focus on interpreting image À À.

20 data at the transmission source. Subsequently, the interpretation data, rather than the image itself, is transmitted and used at the destination communication unit. The interpretation data may or may not be transmitted in compressed form.

Two alternative approaches to image interpretation are known - the 'imagedriven', or bottom-up, approach, and the 'model-driven', or top-down, approach. Model-based techniques tend to produce good results where the range 30 of objects (more generally content) of interest is limited, but where the environment is relatively unconstrained. Image-driven approaches perform better in

a relatively constrained environment. In practice, most systems tend to combine elements of the two approaches.

The image-driven approach relies on features in the 5 image, such as edges or corners, to propagate "naturally" and form meaningful descriptions or models of image

content. A typical example is 'figure-ground' image segmentation, where the task is to separate the object of interest in the foreground from the background.

In the model-driven approach, information regarding content expectation is used to extract meaning from À -.. images. A typical example is object recognition where an outline Computer-Aided Design (CAD) model is compared to À 15 edges found in the image - an approach commonly used in..

manufacturing line inspection applications...

The key difference between the image-driven and model- À driven approaches is in the feature grouping stage. In À À.

20 the image-driven approach, the cues for feature grouping emanate from the image, whereas in the model-driven approach the cues come from the comparison models.

In one variation of an image-driven approach, a number of 25 small salient patches or 'icons' are identified within an image. These icons represent descriptors of areas of interest. In this approach, saliency is defined in terms of local signal complexity or unpredictability or, more specifically, the entropy of local attributes. Icons 30 with a high signal complexity have a flatter intensity distribution and, hence, a higher entropy. Examples of intensity distributions are shown in figure 1, which is discussed later. In more general terms, it is the high

complexity of any suitable descriptor that may be used as a measure of local saliency.

Known salient icon selection techniques measure the 5 saliency of icons at the same scale across the entire image. The particular scale selected for use across the whole image may be chosen in several ways. Typically, the smallest scale, at which a maximum (peak) occurs in the average global entropy, is selected. However, the 10 size of image features varies. Therefore, a scale of analysis, that is optimal for a given feature of a given size, might not be optimal for the same feature of a different size.

À::. 15 It is known that scale information is important in the,.

characterization, analysis, and description of image.

content. For example, prior to filtering an image, it is À.e- necessary to specify the kernel size, or in other words À.

Àe the scale, of a filter to use as well as the frequency À.

20 response. It is also known that filters are commonly used in image processing for tasks such as edge-detection and anti-aliasing.

Alternatively, scale can be regarded as a measurement to 25 be taken from an image (region), and hence can be used as a descriptor. The description extracted from the image

may be used for subsequent matching or classification of that image (region). One example may be in segmenting parts of an aerial image into different regions. In 30 order to extract an appropriate scale description, the

method must capture the scale behaviour of the most dominant', or in other words the most salient, scales in an image.

Recent work in the area of matching and registration of aerial images has shown some promising results using a purely image-based approach. This is described in the 5 paper: "Robust Description and Matching of Images. PhD

thesis, University of Oxford, 1998", by Sebastien Gilles.

Gilles argues that it is possible to sufficiently describe an image by using a number of small (local) salient image patches or icons. The measure of saliency 10 is local entropy; this measures local signal complexity or unpredictability, which in the context of Shannon information theory relates to information content..

Salient icons are selected by an algorithm in each of the À images to be registered and, by matching these; the À 15 approximate global transform between the images may be,..

determined. i. Gilles' experiments with a global image matcher, based on À the maximization of mutual information, show that such. À.

20 (global) methods are prone to failure due to local minima in the optimization function. In order to overcome this problem, Gilles' approach is to use the local salient icon matcher to find the approximate transform, and then to use the global technique to fine-tune this.

The approach is entirely bottom-up and hence no model of expected content is imposed on the image. This generality makes this approach very attractive. A focus of the present invention is to define how those areas of 30 an image (or image sequence) are selected to be representative of the image content (or salient). Hence, the selected areas can be used as the source for efficient description or content interpretation.

Gilles defines saliency in terms of local signal complexity or unpredictability; more specifically he suggests the use of the entropy of local attributes.

Referring now to FIG. 1, local histograms of intensity from various image segments are illustrated. Areas corresponding to high signal complexity are shown to have a flatter distribution, and hence a higher entropy. In 10 general terms, it is the high complexity of any suitable descriptor that may be used as a measure of local saliency. Given a point X=(x,y) in the image and a local neighbourhood Rx, and a descriptor D that takes on values:.

{dl...dr}, local entropy is defined as: À À: |7X = P{., 1082 X

15 did [1].:.

Where Pd. Rx is the probability of descriptor 'd' taking on....

value 'all' in the local region 'Rx'.

The inventors of the present invention have recognized a 20 problem with the Gilles technique, in that it is necessary to select a scale or size of the local neighbourhood region for the entropy calculation and subsequent description. Gilles' original algorithm

measures the saliency of image patches at the same scale 25 across the entire image. Since the size of image features vary, any feature identification technique needs to account for this by selecting an optimum scale (or set of scales) of analysis for a given feature.

30 Furthermore, the Gilles method considers only the feature-space saliency of an image, whereas the analysis

should include the scale dimension as well. That is, the measure of saliency should measure saliency over scale as well as feature-space.

5 The applicant of the present invention described a mechanism for generating these salient points in co-

pending UK patent application GB-A-02367966. In this regard, a Scale Saliency algorithm was introduced as an improvement of the original Gilles algorithm in order to 10 address the aforementioned problem. In order to analyse the scale space behaviour of signals and select appropriate sizes of local scale (the size of the region of interest window used to calculate the entropy), the À À. method searched for peaks in entropy for increasing À 15 scales (at each pixel position). The method then...

weighted the entropy value with a scale-normalised..

À.. measure of the statistical self-dissimilarity at that À - - peak value, as shown in FIG. 2...

À À À. 20 A summary of the known Scale Saliency method is described

below, where notably the behaviour of an entropy scale measure is considered by varying a single scale parameter. 25 It is commonly assumed that a feature that is present across a large number of scales is particularly salient.

However, saliency is based on complexity, defined in terms of unpredictability. In real-life images, this exists at a small number of scales and spatial locations, 30 and hence is considered relatively rare. If an image was complex and unpredictable at all spatial locations and scales, then it would either be a random image or fractal-like. UK patent application GB-A-02367966

indicated that the width of the entropy plot, taken about different points on an image, could serve as a useful indicator, as a saliency estimate based on peak height alone does not enable a distinction to be made between 5 multiple peaks.

FIG. 2, is a flowchart illustrating the known method for determining icon salience, as described in UK patent application GB-A-02367966.

A region defined by a scale s, is typically initialised to a minimum scale value sl in step 200. The region is defined around a pixel at coordinates (x,y) in an image, À.

as shown in step 210. The probability density function....

15 (PDF) of the values of the pixels within the region is À then estimated, in step 220. The values may be grey-

level intensity values, colour levels, or any other À..

characteristic used to define the type of interest..

À. desired for image features...

A Parzen window PDF estimator, for example using a Gaussian kernel method or a basic histogram binning method may be used as the PDF estimation algorithm. Once the PDF is calculated, the entropy HD of region (x,y,s) is 25 then calculated in step 230. Scale s is then increased by the next scale increment in step 240, and the previous steps are repeated for all scales of s between s1 and a maximum scale value s2 in step 250.

30 Once the entropy has been calculated for all scales between s1 and s2 for each pixel at co-ordinates (x,y), those regions having a peak entropy relative to the entropy of the immediately preceding and succeeding

regions are determined in step 260. The entropy HD of each peak region is then weighted in accordance with a value WD(X,Y,S), which is proportional to its peak width estimate, in step 270, in order to provide a measure of 5 saliency S. Steps 200 to 270 are then preferably repeated for all pixels in the image, as shown in step 280. Once the entropy values of regions of peak entropy have been 10 weighted, they are preferably ranked by weighted peak entropy value, thus resulting in a ranking of the pixels by peak saliency, in step 290. In this manner, peak width is used as a weighting factor in the calculation of..

saliency. The idea is that since features are considered salient if.

they are complex or unpredictable in the feature-space, À - --

then in the scale dimension self-similarity corresponds a À to predictability in that dimension. Therefore,: À.

20 unpredictable behaviour over scale should be preferred; that is narrowpeaks in entropy for increasing scales.

The measure for self-similarity that is used is the sum of absolute difference in the histogram of the local 25 descriptor, although there are many alternative ways in which two PDFs may be compared (for example Kullback-

Leibler). The calculation in the continuous case is as follows: (Sp,X) (8p'X) (8p'X) [2]

Where (x)=(x,y) is the spatial position of the pixel in the image. Where entropy HD is defined in the continuous case by: RD ('S7 X) --|P(' S. X) kegs PI 'S. X) 64 BED [3. 1]

5 and where p(d,s,x) is the probability density as a function of scale s, position (x), and descriptor value (d), which takes on values in D, the set of all descriptor values.

10 For a practical implementation, the discrete case of.

equation [3. l] is required, as shown in equation [3. 2]...

(,8, X)- Pd,.,x kit P43.5,x. À...DTD: [3. 2] À

The saliency over scale measure, WD, is defined in the:. ', À. continuous case by: WD (, X) lS I Pack 'S7 X) Id dEl;1 15 [4.1]

For a practical implementation, the discrete case of equation [4.1] is required, as shown in equation [4.2]: 2.5 - 1 1 Pat.S 1X PA s-1 1 [4.2] The absolute partial derivative term is approximated by 20 the sum of absolute difference in the discrete case. The

vector of scales at which the entropy peaks, sp, is defined by: J. 02fS'X) All l 0.2 I [5.1] For a practical implementation, the discrete case of 5 equation [5.1] is required, as shown in equation [5.2]: - l.8 D(' - lax) D(.'X) D(rS + I,X)) [5.2] The method generates a 3D space (2 spatial dimensions..

plus scale) sparsely populated by scalar saliency values..

The above definition does not specify how the probabilities, p(d,s,x), are to be obtained from the À '.

image. In the standard embodiment, a circular sampling i, window is suggested. This is beneficial because it.., 15 enables a rotationally invariant saliency measure.

However, in the general case, any suitable single parameter-sampling window may be used.

Statement of Invention

The inventors of the present invention have recognized that the restriction in the prior art arrangements to a

single-parameter scaling function, for example the radius of a circle, biases the method towards isotropically 25 salient features. Furthermore, the method cannot measure local orientation. Also, sub-optimal scales are selected for features exhibiting anisotropy, for example, short sections of line-like features.

Thus, there exists a need in the field of the present

invention to provide an image transmission system, an image transmission unit and method of processing an image 5 that further improve salient icon selection techniques, wherein the abovementioned disadvantages may be alleviated. In accordance with a first aspect of the present 10 invention, there is provided a method for identifying a feature in an image, as claimed in Claim 1.

In accordance with a second aspect of the present...

invention, there is provided a method of image 15 transmission, as claimed in Claim 10..., .. À .. In accordance with a third aspect of the present .. invention, there is provided an image transmission À À. device, as claimed in Claim 11.: À...DTD: In accordance with a fourth aspect of the present invention, there is provided an image transmission unit, as claimed in Claim 14.

25 In accordance with a fifth aspect of the present invention, there is provided an image transmission system, as claimed in Claim 15.

In accordance with a sixth aspect of the present 30 invention, there is provided a storage medium storing processor-implementable instructions for controlling a processor to carry out any of the aforementioned method

steps of the first and/or second aspect of the present invention, as claimed in Claim 16.

Further aspects of the present invention are as defined 5 in the dependent Claims.

In summary, the inventive concepts of the present

invention, described below, overcome the limitations of the prior art approaches by providing N-parameter

10 sampling windows to account for anisotropic features/regions. In particular, the inventors propose to generalize the isotropic, single parameter sampling function to N 15 parameter sampling windows to account for anisotropic features/regions. In this manner, a better description

of the image, and therefore a feature in the image, is A.

determined. À 20 Brief Description of the Drawings

À À. FIG. 1 illustrates a series of local histograms of À..

intensity of an image, useful in understanding the context..

of the present invention; and.

À À FIG. 2 is a flowchart illustration of a known method for 25 determining icon salience.

Exemplary embodiments of the present invention will now be described, with reference to the accompanying drawings, in which: 30 FIG. 3 shows an elliptical function that provides an example of a plurality of parameter functions that can be used in identifying a feature in an image, in accordance with the preferred embodiment of the invention;

FIG. 4 shows a flowchart for identifying a feature using a plurality of parameters, in accordance with an enhancement to the preferred embodiment of the invention; FIG. 5 illustrates a device for identifying a feature in 5 an image, in accordance with an embodiment of the invention; FIG. 6 illustrates salient regions identified using the known isotropic scale saliency method of FIG. 2; FIG. 7 illustrates salient regions identified using the 10 anisotropic scale saliency method of FIG. 4, in accordance with an embodiment of the invention; FIG. 8 illustrates an original view of a cheetah; and FIG. 9 illustrates a stretched version of the cheetah of FIG. 8 to indicate how the method of FIG. 4 can be used 15 for matching across different views of an object.

Description of Preferred Embodiments À.

À À À The preferred embodiment of the present invention 20 overcomes the limitations of the aforementioned À Àe approaches by providing Nparameter sampling windows to À..

account for anisotropic features/regions.

À. :.. The preferred approach to resolve the aforementioned 25 problems is to use an ellipse 300 parameterised by two scale parameters 'S1' and 'S2' (one for each axis) and a rotation 'Stheta', as illustrated in FIG. 3. However, it is within the contemplation of the invention that any multiple parameter mechanism may benefit from the 30 inventive concepts described herein.

In this manner, a better description of the image, and

therefore a feature in the image, is determined.

Furthermore, this approach is able to deal with some of the effects of projective distortion, for example, 3D 5 effects such as perspective. A practical benefit of this approach is that the generalized Scale Saliency algorithm is able to select an improved set of features and scales in cases where the image is of a 3D scene. For example, in an orthographic view, a circle is imaged as a circle.

10 However, in another view, the circle might map onto an ellipse. A yet further benefit of the generalized concept is that orientation information can now be captured. 15 In an enhanced embodiment of the present invention, the multiple parameter function uses scale, axis ratio, and orientation to parameterise more accurately the ellipse.:.

In this enhanced embodiment, the WD value has only to be À calculated for one parameter instead of two. Indeed, a 20 better parameterisation might be to use scale (s), ratio.

a:. (r=sl/s2) and angle a (equivalent to Stheta), where scale is described in UK Patent Application GB-A-02367966....

This parameterisation has the useful property that the: À.

scale parameter is equivalent to the original isotropic 25 scale, to allow backward compatibility with the single parameter approach.

The preferred embodiment of the present invention is invariant to anisotropic scaling and shear; that is, the 30 full Affine set of transformations. The Affine set is a first order approximation to the full projective set of

transformations, as known to those skilled in the art.

Consequently, it will not be further described here.

The concept of the present invention can also be applied 5 to the arrangement of UK patent application 0112540.0, which was filed on 23 May 2001. UK application 0112540.0 is an arrangement for characterizing texture or a texture-like region in an image, and involves obtaining saliency values. The saliency values obtained can be 10 found using the multi-parameter approach of the present invention. In UK application 0112540.0, the single parameter definition of scale (s) would be replaced by the multi-parameter version described in the present application. So the scalar value s in UK application 15 0112540.0 would be replaced by the vector S used in the present application. So texture classification is one application of the multi-parameter approach of the A.

present invention. The text of UK application 0112540.0 À is hereby incorporated by reference.

À.e À À The preferred enhanced method of the present invention is À.

described below with reference to the flowchart 400 of..

Àe FIG. 4A and FIG. 4B. The flowchart focusses on...

À À. calculating the entropy for regions of pixels, 25 identifying peaks and applying a weighting function.

The enhanced process is carried out for each pixel in the image, where the pixel position is identified by 'x' and y' co-ordinates, as shown in step 402. Initially, the 30 rotation (angle) step 'a' of the elliptical function is set to zero, as in step 404. The ratio step (r=Sl/S2) is set to 'rmin/ratiodiv', as shown in step 406. The variables 'rmin' and 'ratiodiv' specify a number and

range of ratios to be tried. For example, the preferred embodiment of the present invention uses rmin = '5', ratiodiv = '20', which results in the ratio sampling from 0.25' to '1' in steps of '0.05'. The scale value S is 5 then set to a minimum (Smin), as in step 408, and as described in UK Patent Application GB-A-02367966.

The image sampler (IS) samples the local descriptor values at image location I(x,y) using the aforementioned 10 values for 'a', 'r' and 's', as shown in step 410. The preferred arrangement for generating these samples is also described in co-pending UK Patent Application GB-A-

02367966 filed by the same applicant.

15 A probability density P(d,s) value is set to an estimate of the local probability density function (PDF) from the IS samples, as shown in step 412. The PDF calculation À.

can be performed by any known mechanism, such as a À.

histogram technique, as described in UK Patent 20 Application GB-A02367966. The entropy (HD) for that sample is then calculated, for each of the aforementioned À.

values for 'a', 'r' and 's', as shown in step 414. The..

À - scale saliency (WD) for that sample is also calculated for..

the aforementioned values for 'a', 'r' and 's', as shown 25 in step 416.

The process then moves on to the next scale sample, as in step 418. If the next scale sample is not greater than the maximum scale sample in step 420, the above process 30 repeats from step 410 through step 418. The application of SmaX is described in UK Patent Application no. GB-A 02367966. If the next scale sample is greater than the

maximum scale sample SmaX' in step 420, the ratio step is incremented in step 422.

A determination is then made as to whether all ratios 5 have been sampled (i.e. whether the ratio value of S1 to S2 is '1'), in step 424. If the next ratio value is not greater than '1' in step 424, the above process repeats from step 408 through to step 422. That is, the scale value for the next ratio value is set to a minimum, and a 10 new set of scale values for the new ratio calculated.

If the next ratio value is greater than '1', in step 424, the angle value 'a' is incremented in step 426, in accordance with the selected angle step-size. In this 15 regard, the number of angle steps ('number_angles') may be selected in order to perform a predetermined number of À...:.

samples between O and n. A higher number of angle steps...

will give a more accurate result, but will of course be.

À.. slower... À À - For this new angle value, a determination is made as to..

A. whether the maximum angle 'n' has been sampled, in step...

428. If the maximum angle 'n' has not been sampled in step 428, the above process repeats from step 406 through 25 step 426. That is, the scale value and the ratio value are both set to a minimum, and new sets of scale values and ratio values are calculated for this new angle.

The flowchart 400 then moves on to FIG. 4B, once 30 calculations for each value of entropy (HD), local descriptor (IS), PDF (P(d,s)) and scale saliency

(WD(a,r,s) have been completed for each value of angle, ratio and scale (a,r,s). A filter, preferably a three tap averaging (smoothing) filter is applied to the scale saliency values (WD(a,r,s)), as shown in step 430. This 5 smoothing step is applied with respect to the scale parameter only, for all angles and ratios, and alleviates some of the potential noise problems.

Again, the angle step 'a' of the elliptical function is 10 set to zero in step 432. The ratio step (r= S1/S2) is set to 'rmin/ratiodiv', as shown in step 434. As mentioned above, the variables 'rmin' and 'ratiodiv' specify a number and range of ratios to be tried. The scale value S is then set to a minimum (Smin +1) in step 15 436.

. :'e'' À. A determination is then made at the selected scale value -

(in the first case Smin +1), to see if there is an entropy..

peak with respect to scale at this point, as shown in I.

20 step 438. If there is no entropy peak at this point in,., step 438, then the scale value is incremented in step '.

442. If there is an entropy peak at this point in step À 438, then YD (as specified in equation [6]) is calculated in step 440, and all values of YD are stored for later 25 processing. The scale value is then incremented in step 442. If there is not an entropy peak with respect to scale at this point in step 438, the process then moves to step 442 to increment the scale value...DTD: 30 A determination is then made to see whether the selected scale value is a maximum value, in step 444. If the

scale value is not at the maximum value in step 444, the above process repeats from step 438 to step 442, with a determination as to whether there is an entropy peak at the current scale value. Otherwise, if the scale value 5 is at the maximum value in step 444, the ratio (r) is incremented in step 446.

A determination is then made to see whether the current ratio (r) has reached a value greater than '1', in step 448. If the ratio (r) has not reached a value greater than '1' in step 448, the above process repeats with this incremented value of 'r' at step 436. Otherwise, if the ratio (r) has reached a value greater than '1' in step 448, the angle 'a' is increased in step 450, in the same 15 manner as step 426 above.

For this new angle value, a determination is made as to....:.

whether the maximum angle 'n' has been sampled, in step...

452. If the maximum angle 'n' has not been sampled in 20 step 452, the above process repeats from step 434 through..

step 448. That is, the scale value and the ratio value are both set to a minimum, and new sets of scale values À.

and ratio values are calculated for this new angle. If...

the maximum angle 'n' has been sampled in step 452, the 25 process ends by determining the feature (isotropic or anisotropic) of the image in step 454.

In essence, the flowchart of FIG. 4A and FIG. 4B describes replacement of the single parameter sampling 30 function with a multiple parameter version. In this case, it's a three-parameter version. The entropy is calculated at each value of each of the three parameters. However, in the preferred embodiment of the present invention, the scale

saliency (WD) measure is modified such that the partial derivative is taken with respect to 5 only the scale parameter. The inventors have recognised that the rotation angle and ratio cannot be used in this context, as they would measure unpredictability in the rotation angle and ratio. It is not viable to look for peaks over a rotation dimension, as that would bias the 10 method against round items, such as circles, which are not affected by rotation. It is inefficient to look for peaks with respect to ratio, as these do not add to the saliency decision. The peak detection is similarly modified to search for peaks in entropy with respect to 15 scale, but not rotation angle or ratio. The correct orientation is still found because the shape that causes the largest inter-scale saliency measure (WD) is the one that matches the feature shape....

À ÀÀ À. 20 Specifically, the equations can be modified to the elliptical anisotropic case by replacing the scalar s parameter with a vector, s= (a, r,s). The vector of À..

scales at which the entropy peaks, sp, becomes a matrix,....

À. Sp with three rows, one for each of the scale variables..

25 and as many columns as peaks at that position. The modified equations are as follows: D($ p, X)- 0 (SpllX) X WD(SP,X) [6]

($, X) - - |p(d, $7 X) kit P(/ $, X).4 [7] WD (S' X) rat x J L P(47 87 X) | dE.D [8] $ _ {s 7) 0} For a practical implementation, the discrete versions of 5 equations [7], [8], [9] are required: (87 X) - - Pd,s,x kid Pax dED [10].. Àe-.. WD (, X) - IP81.,X-P&, - 1'X I

2.- 1 dED... [11]. Sp _ {$ *D(s - 1,X) ('X) (I + 1,X)) 10 [12] À.

The invention has been described above in terms of a method. However, the invention also comprises an image transmission unit/device functioning in accordance with the invention.

An embodiment of such a device is illustrated in FIG. 5.

The device of FIG. 5 serves to rank pixels in an image by degrees of saliency. The device comprises a processor

500 providing a defining function 510 that defines a plurality of regions for each of a plurality of pixels in an image 505. In accordance with the preferred embodiment of the present invention, each of the regions 5 is defined over a different scale 's', at a ratio 'r' and angle 'a' for any one of the pixels at coordinates (x, y). The processor also includes a calculation function 530 for calculating an entropy value for each of the regions, for each permutation of the respective variables 10 a,r,s.

The processor also includes a peak identification function 560 for identifying any peak entropy regions.

The peak entropy regions are regions that include a peak 15 entropy value and a weighting function 570 for weighting the peak entropy value of each of the peak entropy regions by a weight value corresponding to a peak width estimate of the peak entropy value. This is described in UK patent application GB-A-02367966. Finally, the...

20 processor includes a feature identification function 590 for identifying a feature in an image based on the À.

determined peaks..

The focus of the preferred embodiment is to find À 25 interesting regions in an image in an unsupervised À.

manner. The weighted saliency measure supplies values about how interesting something is. In this regard, the present invention focuses on identifying a feature in an image, which to all intent and purpose is such an 30 interesting region in an image and the two expressions should be deemed synonymous. Once a rank ordered set of weighted saliency measures has been obtained, it is possible to then apply any conventional thresholding

technique to select a certain number of regions or points, for example, the 10% most salient points.

Alternatively, it is possible to apply clustering by any known technique to group salient points into salient 5 regions.

Also shown in FIG. 5 is an 'acquisition and frame structure' unit 508, which includes a camera for scanning the image. Such a unit is known to the person skilled in 10 the art, and hence is not described in further detail.

The PDF estimator 520 provides the probability density function, as described above with reference to the methods of the invention.

15 Referring now to FIG. 6, an example of the performance of the original isotropic scale saliency algorithm is illustrated. FIG. 7 illustrates an example of the performance of the modified anisotropic scale saliency '..

algorithm. Comparison of the two performances of FIG. 6 À 20 and FIG. 7 highlights that the anisotropic version correctly identifies the scales of the ellipses and the circle. In contrast, the isotropic version correctly À..

detects only the circle and finds numerous features along..

A. the ellipses. A..

Although the preferred embodiment of the present invention has been described with respect to a three-

parameter sampling window, it is within the contemplation of the invention that the inventive concepts can be 30 applied to any multiple- sampling window or alternative parameterisations. For example, instead of parameterising an ellipse using a major/minor axis ratio, a rotation, and a scale, it is possible to use two scales

and a rotation. As another example, any polygon with N sides can be parameterised using N-1 terms (for example the corner angles), a rotation and a scale. The WD term should then be modified to calculate the partial 5 derivative with respect to each of the parameters.

It is envisaged that the inventive concepts hereinbefore described apply to any video or image communication device. The video or image communication device may 10 select salient regions of an image for preferential transmission. Particularly where the transmission is over a radio link to or from a mobile or portable radio (PMR) or mobile telephone, the transmission may comprise the selected regions only. Alternatively, these regions 15 may be transmitted more frequently than other regions.

The video or image communication device may form part of a mobile or portable radio (PMR) or cellular telephone.

Àe As multimedia communication systems become commodetised À 20 in the future, technologies such as those offered by this invention will enable users to efficiently communicate ' À À - key features of an image, without having to pay for À.

additional and costly bandwidth in order to send the,.

À.. entire image itself. This invention could be...

À Àe 25 incorporated into any mobile image or video transmission device. It will be understood that the image transmission system, transmission unit and method for identifying a feature 30 within an image, as described above, provides at least the following advantages: (i) Extending the ability to find interesting regions of an image, as described in UK Patent Application GB-A

02367966, to anisotropic regions. In this regard, scale definition is extended to multiple parameters, thereby generalising the structures that can be identified.

(ii) If an ellipse is parameterised using scale, rotation 5 and a ratio of the major and minor axes, the method is invariant to anisotropic scaling and shear. As such, it is compatible with the full Affine set of transform-

ations. (iii) In particular, the method is also useful for 10 matching across different views of an object, as illustrated by the examples shown in FIG. 8 and FIG. 9.

FIG. 8 illustrates an original view of the Cheetah, and FIG. 9 illustrates a stretched version. It is noteworthy that the interesting regions (the spots) are still 15 correctly identified in the stretched version of FIG. 9, by utilising the aforementioned inventive concepts.

Method of the invention:.....

In summary, a method for identifying a feature in an À

20 image has been provided. The method includes the steps of defining a plurality of saliency values of an image or set of images over multiple parameters, and calculating À.

an entropy value for each of the saliency values over..

substantially each of the multiple parameters. The ^. À.

À Àe 25 method further includes the steps of determining one or more peak entropy values for the image or set of images based on the calculated entropy values; and identifying a feature of said image or set of images based on the determined peak entropy value.

Apparatus of the invention: Furthermore, an image transmission device has been described that includes a processor having the following

functions: a defining function for defining a plurality of saliency values of an image or set of images over multiple parameters; a calculation function for calculating entropy value for each of the saliency values 5 over substantially each of the multiple parameters; a peak entropy identification function for identifying any peak entropy value for the image or set of images based on said calculated entropy values; and a feature identification function identifying a feature in the 10 image or set of images based on the peak entropy value.

An image transmission unit adapted to perform the above method steps has also been provided. In addition, an image transmission system adapted to facilitate any of 15 the above method steps or incorporate the above image À. e transmission device has been provided. Also, a storage medium storing processor-implementable instructions for À controlling a processor to carry out any of the above..

method steps has been provided...

Àe Whilst the specific and preferred implementations of the À embodiments of the present invention are described above, À À.

it is clear that a skilled artisan could readily apply variations and modifications of such inventive concepts.

Thus, an image transmission system, an image transmission unit and method of identifying an image that further improve salient icon selection techniques have been provided, wherein the abovementioned disadvantages 30 associated with prior art arrangements have been

substantially alleviated.

Claims 1. A method (400) for identifying a feature in an image, comprising the following steps: 5 defining (410) a plurality of saliency values of an image or set of images over multiple parameters; calculating (414) an entropy value for each of said saliency values over substantially each of said multiple parameters; 10 determining (430, 432) one or more peak entropy values for said image or set of images based on said calculated entropy values; and identifying (454) a feature of said image or set of images based on said peak entropy value.

2. The method (400) for identifying a feature in an À...

image according to Claim 1, wherein the step of defining (410) includes the step of sampling a window of an image..

across multiple parameters such that an anisotropic À.

20 feature can be identified.. .

À:. 3. The method (400) for identifying a feature in an image according to Claim 1 or Claim 2, wherein the step of defining (410) includes the step of parameterising an 25 ellipse in order to calculate an entropy peak with respect to scale.

4. The method (400) for identifying a feature in an image according to Claim 3, wherein the step of 30 parameterising an ellipse includes using at least two scale parameters of said ellipse.

5. The method (400) for identifying a feature in an image according to Claim 4, wherein the at least two scale parameters include one or more of: a ratio of the ellipse axes (406) of said 5 ellipse, and/or a rotation (404), preferably angle (Stheta), of said ellipse, and/or a scale.

10 6. The method (400) for identifying a feature in an image according to Claim 1 or Claim 2, wherein the step of defining includes using a polygon having N sides that can be parameterised using N-1 terms.

À.....

15 7. The method (400) for identifying a feature in an image according to any preceding Claim, wherein said step À of calculating an entropy value includes measuring an...

inter-scale saliency of said image or set of images, the..

À. method further characterized by the step of: 20 calculating said interscale saliency measurement À by taking a partial derivative of a probability density. À.

function with respect to scale.

8. The method (400) for identifying a feature in an 25 image according to Claim 7 when dependent upon Claim 4, wherein said step of taking a partial derivative of said inter-scale saliency measurement precludes taking a partial derivative of said rotation angle.

30 9. The method (400) for identifying a feature in an image according to Claim 7 when dependent upon Claim 4, wherein said step of determining a peak entropy value

precludes taking a partial derivative of said rotation angle and/or ratio.

10. A method of image transmission, comprising 5 a method according to any previous Claim.

11. An image transmission device, comprising a processor (500) that includes the following functions: 10 a defining function (510) for defining a plurality of saliency values of an image or set of images over multiple parameters; a calculation function (530) for calculating entropy value for each of said . e. 15 saliency values over substantially each of said multiple parameters; À a peak entropy identification function..

(560) for identifying any peak entropy value for..

said image or set of images based on said 20 calculated entropy values; and À a feature identification function (590) À À.

identifying a feature in said image or set of images based on said peak entropy value.

25 12. The image transmission device according to Claim 11, further characterized by said plurality of parameters including scale 's', at ratio 'r', and angle 'a', wherein said entropy value is calculated for each permutation of the respective 30 parameters 's', 'a', 'r'.

13. The image transmission device according to Claim 11 or Claim 12, further characterized by said calculation function (530) performing a probability density function to determine entropy values for each of said saliency 5 values over substantially each of said multiple parameters. 14. An image transmission unit incorporating the image transmission device according to any of Claims 11 10 to 13 or adapted to perform the method of any of Claims 1 to 10.

15. An image transmission system adapted to facilitate the method of any of preceding claims 1 to 10.

15....:.

À 16. A storage medium storing processor-implementable À A..

instructions for controlling a processor to carry out the.

method any of claims 1 to 10...

20 17. A method (400) for identifying a feature in an.

image substantially as hereinbefore described with A..

reference to, and/or as illustrated by, FIG. 4A and FIG. 4B of the accompanying drawings.

25 18. An image transmission device substantially as hereinbefore described with reference to, and/or as illustrated by, FIG. 5 of the accompanying drawings.

Claims (1)

1, c: 3\.'À: c: Amended claims have been filed as followed Claims 1. A

method for identifying a feature in an image, comprising the following steps: 5 defining a plurality of saliency values of an image or set of images over multiple parameters; calculating an entropy value for each of said saliency values over substantially each of said multiple parameters; 10 determining one or more peak entropy values for said image or set of images based on said calculated entropy values; and identifying a feature of said image or set of images based on said peak entropy value.

2. The method for identifying a feature in an image according to Claim 1, wherein the step of defining includes the step of sampling a window of an image across multiple parameters such that an anisotropic feature can 20 be identified.

3. The method for identifying a feature in an image according to Claim 1 or Claim 2, wherein the step of defining includes the step of parameterising an ellipse 25 in order to calculate an entropy peak with respect to scale. 4. The method for identifying a feature in an image according to Claim 3, wherein the step of parameterising 30 an ellipse includes using at least two scale parameters of said ellipse.

d r À À 1 À

5. The method for identifying a feature in an image according to Claim 4, wherein the at least two scale parameters include one or more of: a ratio of the ellipse axes of said ellipse, 5 and/or a rotation, preferably angle (Stheta), of said ellipse, and/or a scale.

10 6. The method for identifying a feature in an image according to Claim 1 or Claim 2, wherein the step of defining includes using a polygon having N sides that can be parameterised using N-1 terms.

15 7. The method for identifying a feature in an image according to any preceding Claim, wherein said step of calculating an entropy value includes measuring an inter scale saliency of said image or set of images, the method further characterized by the step of: 20 calculating said inter-scale saliency measurement by taking a partial derivative of a probability density function with respect to scale.

8. The method for identifying a feature in an image 25 according to Claim 7 when dependent upon Claim 4, wherein said step of taking a partial derivative of said inter-

scale saliency measurement precludes taking a partial derivative of said rotation angle.

30 9. The method for identifying a feature in an image according to Claim 7 when dependent upon Claim 4, wherein said step of determining a peak entropy value precludes

. - : : :: À taking a partial derivative of said rotation angle and/or ratio. 10. A method of image transmission, comprising 5 a method according to any previous Claim...CLME: 11. An image transmission device, comprising a processor (500) that includes the following functions: 10 a defining function for defining a plurality of saliency values of an image or set of images over multiple parameters; a calculation function for calculating entropy value for each of said saliency values IS over substantially each of said multiple parameters; a peak entropy identification function (560) for identifying any peak entropy value for said image or set of images based on said 20 calculated entropy values; and a feature identification function identifying a feature in said image or set of images based on said peak entropy value.

25 12. The image transmission device according to Claim 11, further characterized by said plurality of parameters including scale 's', at ratio 'r', and angle 'a', wherein said entropy value is calculated for each permutation of the respective 30 parameters 's', 'a', 'r'.

i: t t t It . - t t À 8 t À; 8 1 8 8. 8

13. The image transmission device according to Claim 11 or Claim 12, further characterized by said calculation function performing a probability density function to determine entropy values for each of said saliency values 5 over substantially each of said multiple parameters.

14. A mobile or portable telephone incorporating the image transmission device according to any one of claims 11 to 13.

15. A storage medium storing processor-implementable instructions for controlling a processor to carry out the method any of claims 1 to 10.

15 16. A method for identifying a feature in an image substantially as hereinbefore described with reference to, and/or as illustrated by, FIG. 4A and FIG. 4B of the accompanying drawings.

20 17. An image transmission device substantially as hereinbefore described with reference to, and/or as illustrated by, FIG. 5 of the accompanying drawings.