WO2010001303A1

WO2010001303A1 - Texture regularity analysis for an image

Info

Publication number: WO2010001303A1
Application number: PCT/IB2009/052700
Authority: WO
Inventors: Chris Damkat; Paul M. Hofman
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2008-06-30
Filing date: 2009-06-24
Publication date: 2010-01-07

Abstract

An image is scanned using image blocks and a texture regularity vector set is determined for each image block. For each image block being processed, a group of texture regularity vector sets for a set of previously processed image blocks is included in a candidate set. Update texture regularity vector sets are then generated (207) from the group of texture regularity vector sets and these are added to the candidate set. For each vector set, a match measure is generated (211) between the image block and a set of candidate image blocks offset from the image block by vectors of the vector set. A texture regularity vector set is then selected (213) for the image block dependent on the match measures. The invention may allow computationally efficient and accurate determination of texture regularity vector sets that reflect the regularity of textures of the image.

Description

Texture regularity analysis for an image

FIELD OF THE INVENTION

The invention relates to determination of texture regularity vector sets for an image and in particular, but not exclusively, to determination of texture regularity vector sets for a frame of a video signal.

BACKGROUND OF THE INVENTION

Many images such as digital photos or frames of video sequences tend to comprise significant amounts of texture. A texture may be considered to be an area of the image which is substantially stationary and repetitive such that a fundamental image patch tends to be substantially repeated at a given spatial repetition interval.

In many applications, it is desirable to estimate the texture regularity and specifically to determine data that characterizes how a basic image patch is repeated to substantially generate the texture pattern (such as the periodicity(ies) or repetition interval(s) and direction(s)). A perfectly periodic texture can be described by the image patch and two vectors that describe the lattice of the texture. Furthermore, natural images often contain multiple textures that are typically only approximately periodic and which can be described as being locally periodic. For such textures, two regularity vectors may be used in each pixel region to describe the local texture. Known regularity estimators for estimating such regularity vectors tend to be based on determining an autocorrelation for the entire image and then identifying peaks in the autocorrelation. However, such systems tend to be suboptimal and have a number of associated disadvantages. For example, the process is computationally resource demanding and this typically prevents efficient real time applications from using the approach. Furthermore, the generated estimates tend to be based on assessment of the global regularity rather than the local regularity in specific texture regions as the autocorrelation is determined for the entire image.

Hence, an improved system for determining texture regularity vector sets for an image would be advantageous and in particular a system allowing increased flexibility, reduced complexity, facilitated implementation, improved accuracy, reduced computational resource usage and/or improved performance would be advantageous.

SUMMARY OF THE INVENTION Accordingly, the Invention seeks to preferably mitigate, alleviate or eliminate one or more of the above mentioned disadvantages singly or in any combination.

According to an aspect of the invention there is provided a method of determining texture regularity vector sets for an image, the method comprising iterating the steps of: selecting a selected image block; retrieving a group of texture regularity vector sets for a set of previously processed image blocks of the image; generating at least one update texture regularity vector set from the group of texture regularity vector sets; generating a candidate set of texture regularity vectors comprising the group of texture regularity vector sets and the at least one update texture regularity vector set; for each texture regularity vector set of the candidate set generating a match measure between the image block and a set of candidate image blocks being offset from the image block by vectors of the texture regularity vector set; and selecting a texture regularity vector set for the image block in response to the match measures.

The invention may allow a computationally efficient determination of texture regularity vector sets. The invention may in many cases allow improved determination of texture regularity vector sets and may for example allow a determination based on local texture characteristics rather than characteristics of the image as a whole.

The method may optionally comprise the optional step of segmenting the image into image segments in response to the texture regularity vector sets for the image blocks. The generated texture regularity vector sets may alternatively or additionally be used for image compression or encoding and/or for image noise filtering and/or resolution enhancing re-synthesis of texture and/or generation of shape from texture (e.g. three dimensional shape reconstruction from the texture) etc. An improved performance and/or reduced computational resource load can thus be achieved for many different image processing applications. The texture regularity vector sets may specifically be texture regularity vector pairs comprising two vectors. For most textures, the structure of the texture can be sufficiently accurately described by only two vectors.

The selection step may specifically select the texture regularity vector set for an image block as the texture regularity vector set of the candidate set which has the highest match measure. The selected image block may specifically be selected as the next image block of a scanning pattern for the image. The previously processed image blocks may include one or more previous image blocks in the scanning pattern.

In many embodiments, the number of previously processed image blocks for which texture regularity vector sets are included in the candidate set is less than ten and/or the number of update texture regularity vector sets is less than ten. Indeed, in some embodiments the number of previously processed image blocks for which texture regularity vector sets are included in the candidate set is less than five and/or the number of update texture regularity vector sets is less than five. By maintaining a low number of texture regularity vector sets in the candidate set, a low computational resource demand can be achieved. At the same time, as the selection for the candidate set is dependent on previously determined texture regularity vector sets, an accurate result can still be achieved and spatial consistency may be improved. Furthermore, the use of update sets allows a convergence of the process to the appropriate values. Each vector of a texture regularity vector set may typically correspond to a spatial offset between repetitions of a fundamental image patch of a texture.

In accordance with an optional feature of the invention, the set of previously processed image blocks comprises at least one image block adjacent to the selected image block. This may allow improved performance and in particular may provide a higher likelihood that the candidate set comprises suitable texture regularity vector sets for assessment.

In accordance with an optional feature of the invention, the match measure is determined as a function of a pixel value similarity measure for the selected image block and the set of candidate image blocks.

This may allow improved performance and/or facilitated implementation and/or operation. In particular, it may allow an efficient evaluation of the likelihood of the individual candidate texture regularity vector set corresponding to an offset between repetitions of the texture pattern. In accordance with an optional feature of the invention, a first match measure for a first texture regularity vector set of the candidate set is determined in response to a length of at least one vector of the first texture regularity vector set.

This may allow improved performance and/or facilitated implementation and/or operation. In particular, it may allow an improved selection of the texture regularity vector set from the candidate set. Specifically, a penalty value or compensation of a correlation measure may be made dependent on the vector lengths thereby allowing a bias towards vectors sets having the most likely or appropriate values.

In accordance with an optional feature of the invention, the first match measure comprises a penalty contribution which increases for reducing length.

This may allow improved performance and/or facilitated implementation and/or operation. In particular, it may allow an improved selection of the texture regularity vector set from the candidate set. Specifically, a penalty value or compensation may be increased for very low vector lengths thereby biasing against a selection of a vector set that corresponds to the same repetition of the fundamental image patch.

In accordance with an optional feature of the invention, the first match measure comprises a penalty contribution which increases for increasing length.

This may allow improved performance and/or facilitated implementation and/or operation. In particular, it may allow an improved selection of the texture regularity vector set from the candidate set. Specifically, a penalty value or compensation may be increased for high vector lengths thereby biasing against a selection of a vector set that does not correspond to the closest repetition of the fundamental image patch but to a second, third etc repetition.

In some embodiments, a texture scale input or estimate may be provided and the threshold may be determined in response to this.

In accordance with an optional feature of the invention, the match measure for a first texture regularity vector set of the candidate set is determined in response to an angle between vectors of the first texture regularity vector set.

This may allow improved performance and/or facilitated implementation and/or operation. In particular, it may allow an improved selection of the texture regularity vector set from the candidate set. Specifically, a penalty value or compensation may be increased for low angles thereby biasing against a selection of a vector set that have similar directions and accordingly may correspond to substantially the same repetition direction for the texture. Also, an increasing angle is indicative of a vector set that provides a better basis for linear combinations of the vectors to represent other repetitions.

In accordance with an optional feature of the invention, the match measure for a first texture regularity vector set of the candidate set is determined in response to a variance measure of at least one of the selected image block and an image block of the set of candidate image blocks. This may allow improved performance and/or facilitated implementation and/or operation. In particular, it may allow an improved selection of the texture regularity vector set from the candidate set. In particular, this may allow a compensation for the impact of contrast characteristics on the match measure thereby providing an improved texture regularity vector set determination.

In accordance with an optional feature of the invention, the iteration corresponds to a sequential processing of image blocks in accordance with a scanning pattern for the image; and the set of previously processed image blocks comprises at least one previous image block according to the scanning pattern. This may allow improved performance and/or facilitated implementation and/or operation.

In accordance with an optional feature of the invention, the iteration of steps corresponds to an iterated scanning of the image; and the set of previously processed image blocks comprises at least one image block of a previous scanning of the image. This may allow improved performance and/or facilitated implementation and/or operation. In particular, it may allow for a more accurate determination of the texture regularity vector sets and may specifically allow an improved convergence of the system towards the appropriate texture regularity vector sets.

In accordance with an optional feature of the invention, a different scanning pattern is used for at least two of the scans of the image.

This may allow improved performance and/or facilitated implementation and/or operation. In particular, it may allow for a more accurate determination of the texture regularity vector sets and may specifically allow an improved convergence of the system towards the appropriate texture regularity vector sets. In particular, it may allow the candidate set of one image scan to be based on different image blocks than for the previous image scan.

In accordance with an optional feature of the invention, the at least one update texture regularity vector set is generated from a first texture regularity vector set of the group of texture regularity vector sets by modifying only one vector of the first texture regularity vector set.

This may allow improved performance and/or facilitated implementation and/or operation. In particular, it may provide update texture regularity vector sets that are more likely to be close to the correct texture regularity vectors. The update texture regularity vector set may for example be generated by adding a random vector value to one vector of a texture regularity vector set while keeping the other vectors constant.

In accordance with an optional feature of the invention, a match measure for a texture regularity vector set of the candidate set is biased relative to a match measure for the at least one update texture regularity vector set.

This may allow improved performance and/or facilitated implementation and/or operation. In particular, it may allow a trade-off between accuracy, consistency and/or convergence speed to be flexibly and easily adjusted.

In accordance with an optional feature of the invention, the method further comprises the step of excluding a first texture regularity vector set of the group of texture regularity vector sets in response to the first texture regularity vector set comprising at least one vector having a length which is substantially a multiple of a length of at least one vector of a second texture regularity vector set of the group of texture regularity vector sets if a match measure of the second texture regularity vector set meets a criterion. This may allow improved performance and/or facilitated implementation and/or operation. In particular, it may increase the probability that a texture regularity vector set is selected that does not correspond to a first repetition.

The exclusion may for example be performed by excluding the first texture regularity vector set from the candidate set prior to determining match measures and/or may be done at the time of selection of the texture regularity vector set.

According to an aspect of the invention there is provided an apparatus for determining texture regularity vector sets for an image, the apparatus comprising a processor for iteratively performing the steps of: selecting a selected image block; retrieving a group of texture regularity vector sets for a set of previously processed image blocks of the image; generating at least one update texture regularity vector set from the group of texture regularity vector sets; generating a candidate set of texture regularity vectors comprising the group of texture regularity vector sets and the at least one update texture regularity vector set; for each texture regularity vector set of the candidate set generating a match measure between the image block and a set of candidate image blocks being offset from the image block by vectors of the texture regularity vector set; and selecting a texture regularity vector set for the image block in response to the match measures.

These and other aspects, features and advantages of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention will be described, by way of example only, with reference to the drawings, in which

Fig. 1 is an illustration of a scanning pattern for an image; Fig. 2 illustrates a method of determining texture regularity vector sets for an image in accordance with some embodiments of the invention;

Fig. 3 illustrates an example of an image block arrangement for an image;

Fig. 4 illustrates an example of penalty values for a method of determining texture regularity vector sets for an image in accordance with some embodiments of the invention;

Fig. 5 illustrates an example of an apparatus for determining texture regularity vector sets for an image in accordance with some embodiments of the invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS In the following, an approach for determining texture regularity vector sets for an image will be described wherein the texture regularity vector sets comprises two vectors that characterize the lattice structure of a texture. In the example, a texture regularity vector sets for an image block is thus a vector pair comprising two vectors that point to an estimated location of a first repetition of the image block. Specifically, an image is divided into image blocks, such as 8 by 8 pixel square image blocks, adjacent to each other. A texture regularity pair set is then determined for each image block in an iterative and recursive matter wherein the determination of the texture regularity vector pair for the current image block is based on the texture regularity vector pairs determined for previously processed image blocks. Specifically, the image is scanned in accordance with a scanning pattern, such as e.g. that illustrated in Fig. 1, and the processing of the current image block utilizes the results from the earlier image blocks of the scanning pattern.

Specifically, the described texture regularity determination and estimation is akin to a technique known as Three Dimensional Recursive Search (3DRS) which is used for determining motion estimation vectors between different images in a video sequence. Specifically, 3DRS is often used for motion compensated frame-rate conversion (e.g. doubling of a frame-rate) or motion compensated filtering. Further information of the 3DRS technique may be found in "True-motion estimation with 3-D recursive search block matching" by de Haan, G., et al. Circuits and Systems for Video Technology, IEEE Transactions on, 1993. 3(5): p. 368-379, 388.

Fig. 2 illustrates a method of determining texture regularity vector sets for an image in accordance with some embodiments of the invention. In the approach, a number of candidates for the texture regularity vector pair for an image block is first determined to include the texture regularity vector pairs that were found for some selected previously processed (close) image blocks. In addition, some possible update vectors are generated from these vector pairs by adding noise to one or more of the vectors. The update vector pairs are then included in the candidate set to result in a candidate set with a relatively low number of possible texture regularity vector pairs for the current image block (typically less then 10-20 sets). For each of the texture regularity vector pairs of the candidate set, the current image block is compared to the image area that is offset by an amount given by the vectors of the vector pair. A similarity measure is determined from the comparison and this may be modified in response to some penalty values to result in a match measure that is indicative of the likelihood that the image blocks correspond to texture repetitions of each other. The texture regularity vector pair having the highest match measure is then selected for the image block. The method then moves on to the next image block in accordance with the scanning pattern and repeats the process.

The method starts in step 201 wherein the scanning is initialized by the scanning pattern being retrieved. The method then proceeds to process a single image block at a time using the results of previous image blocks. It will be appreciated that for the first image block(s) no suitable results may be available for previously processed image blocks. Thus, the method may be initialized using predetermined starting texture regularity vector pairs, using texture regularity vector pairs from previously processed correlated images (e.g. a previous frame of a video signal) or from previous scans of the same image etc. For clarity, the following description of an image block will focus on the processing of an image block for which previously processed image blocks are available. Specifically, the image block may be an image block which is not in a border area of the image.

In step 203, the next image block is selected in accordance with the scanning pattern. For example, if the current scan direction is from left to right, the image block is selected as the block immediately to the right of the previously processed block. In the example, square image blocks with a size of e.g. 8 by 8 pixels are used but it will be appreciated that in other embodiments other sizes or shapes of image blocks may be used. Fig. 203 is followed by step 205 wherein a reference set of texture regularity vector pairs for a set of previously processed image blocks of the image are retrieved.

The reference set may specifically be selected to include the texture regularity vector pairs for a set of image blocks having a predetermined spatial relationship with the current image block. For example, the image blocks that are adjacent to the current image block and have already been processed by the scan may be included in the reference set. For example, as illustrated in Fig. 3, the current image block 301 may be processed in a current scan direction 303 from left to right (and from top to bottom as indicated in Fig. 1). Accordingly, the image block 305 immediately to the left and the image block 307 immediately above the current image block 301 have already been processed and thus a vector pair has been determined for each of these. Accordingly, the reference set of previously processed image blocks may include these two image blocks 305, 307 and thus the reference set of texture regularity vector pairs include the vector pairs of these two image blocks 305, 307. Step 205 is followed by step 207 wherein one or more update texture regularity vector pairs is generated from the set of texture regularity vector pairs. Specifically, one or more update vector sets may be generated from each of the reference vector sets by modifying the vectors thereof. For example, noise in the form of a random value may be added to the vector value to generate an update vector that provides some variation for the update vector pair relative to the reference vector pair.

In the specific example, each of the update vector pairs is generated by modifying only one vector of the corresponding reference vector pair. Specifically, a random noise value may be added to only one of the vectors whereas the other is kept unchanged. Although, it is possible to update both vectors with noise, the likelihood of finding a better vector set candidate in this way is likely to be lower since it is more likely that the appropriate texture regularity vector pair for the current image block includes a vector which corresponds to at least one of the vectors of an adjacent block. Accordingly, generating the update candidates by only changing one of the vectors results in an increased likelihood of finding a better vector set and in the speeding up of the convergence of the process. Step 207 is followed by step 209 wherein a candidate set of vector pairs is generated to include both the reference vector pairs and the update vector pairs. This candidate set thus includes both the vector pairs of the adjacent blocks as well as vector pairs that represent some random variations on these. Step 209 is followed by step 211 wherein the candidate vector pairs are evaluated. Specifically, for each of the candidate vector pairs, a match measure is generated reflecting a similarity between the current image block and a set of candidate image blocks of the image which is offset from the current image block by the vector set. Specifically, a location of a first candidate image block is determined by adding a vector of the vector pair to the location of the current image block. The pixel values of the image at that location are then compared to the pixel values of the image block and a pixel value similarity measure is determined. It will be appreciated that any suitable pixel value similarity measure may be used including for example a correlation measure, a sum of absolute differences measure or an average sum of the squared difference between pixel values.

The process may then be repeated for the second vector of the candidate vector pair and the two similarity measures may be combined into a single similarity measure (e.g. by a simple summation). Thus the similarity measure and the match measure is a combined match measure for both vectors of the texture regularity vector pairs.

In some embodiments, the pixel similarity measure may directly be used as the match measure. However, in many embodiments further compensation or penalties may further be considered as will be described later.

Step 211 is followed by step 213 wherein a texture regularity vector pair is selected for the image block. Specifically, the candidate vector pair which has the highest match measure is selected as the texture regularity vector pair for the current image block. Thus, the spatial offset that results in a closest match between the two image areas is selected as the texture regularity vector pair for the image block.

Step 213 is followed by step 215 wherein it is evaluated if there are any more image blocks left in the scanning process. If so, the method returns to step 203 and repeats the described approach for the next image object. The processing of this next image block may then use the determined vector pair for the current image block as one of the candidate vector pairs.

If no further image blocks are to be processed, the method proceeds to step 217 wherein the texture analysis in the specific example is used to segment the image into image segments based on the determined vector pairs. Specifically, image segments may be determined as contiguous regions for which the determined vector pairs are sufficiently similar. E.g. a segment may include all image blocks for which a difference between the vector pairs is less than a given value. The process may provide an accurate and efficient determination of local texture characteristics. Specifically, as only the potentially very small number of candidate vector pairs for each image block need to be evaluated (rather than e.g. all possible offsets), a low computational resource is required. This may specifically allow the approach to be used for many real-time applications.

Furthermore, despite the low number of candidate vectors, these are specifically selected to exploit the fact that there is often a high spatial consistency in the lattice of a texture. Furthermore, by including random variations for the candidates, the system may provide an accurate convergence and adaptation performance despite only evaluating few candidate vector pairs.

Specifically, the candidate set will include vector pairs that are correct if no change have occurred from the previous image block (and the vector pair was correct for the previous image block), as well as some vector pairs that may provide a better match if the texture regularity has changed or if the previous vector pair was not correct. This will typically allow the system to converge towards the correct value. For example, even if the scanning process is initialized with predetermined vector pairs, these will tend to converge towards the correct values for the subsequent image blocks as the update candidate vector pairs of the following image blocks are likely to provide a better match and thus to be selected. Thus, gradually the selected vector pairs will converge towards the appropriate values.

As mentioned previously, the step of determining the match measure may include various compensations or penalties to reflect different characteristics and preferences. Specifically, vector based penalty values may be introduced to the match measure. These vector penalties may facilitate or enable the use of the approach for texture regularity determination within a single image. Furthermore, the vector penalties may facilitate the generation of two independent vectors necessary for the description of the periodic structure of the texture.

Specifically, the match measure for a texture regularity vector pair may be determined in response to a length of at least one vector of the texture regularity vector pair. Thus, the match measure may be biased dependent on the length of one or both vectors of the vector pair.

In many embodiments, the match measure may be reduced for reducing length if the length is below a threshold. Thus, an increase of a penalty value may be introduced if the length of one or both of the vectors is below a given value thereby biasing the selection of the vector pair away from vector pairs that have very short lengths. This may specifically seek to reduce the probability that the detected match is indeed the same repetition (i.e. the same instance of the basic texture patch) rather than a different repetition of the image area in the texture. Alternatively or additionally, the match measure may be reduced for increasing length for the length being above a threshold. Thus, an increase of a penalty value may be introduced if the length of one or both of the vectors is above a given value thereby biasing the selection of vector pair away from vector pairs that have very long lengths. This may specifically seek to reduce the probability that the detected match is not the first texture repetition but rather a second or third etc repetition.

Thus, in the specific example, it is desired to penalize the self-matching of the block and therefor very short vectors are penalized. Furthermore, in a periodic texture many matches can be found (e.g. at two times, three times the period etc) and it is desired that only the shortest match vector describing the fundamental period is identified. Therefore, longer vectors may be penalized.

As a specific example, the following length dependent penalties may be used:

ncn (tλ - Ct ■ p^~Wσ + R . I l / I l

where t is the regularity vector, a and β are parameters controlling the strength of the penalties, and σ sets the size of the area to be penalized around the zero vector.

Thus, in the example, match measure comprises both a penalty contribution which increases for reducing length and a penalty contribution which increases for increasing length. By adjusting the penalty contributions and the relative weights thereof, a bias may thus be introduced to the match measure which biases away from the selection of both very small and very long vectors.

In some embodiments, the negative bias of longer vectors may be more substantial. For example, in some embodiments, vector pairs may be excluded if one of the vectors has a length which is a multiple of a length of a vector of another vector pair that has been found to have a match measure sufficient to indicate a match (e.g. a value above a given threshold). Specifically, if the shorter vector pair has a match measure that meets a criterion indicative of the current image block having a match at this location; it is likely that the longer vector pair merely corresponds to a further repetition of the image block. Thus, even if the longer vector pair results in a somewhat better match, it is likely that it does not correspond to the first repetition of the image block but rather a second, third etc repetition. As the method is aimed at finding the vector pair describing the first repetition, the longer vector pair should be excluded in preference of the shorter pair.

It will be appreciated that due to noise and variations, vector pairs corresponding to different repetitions may not have lengths that are exact multiples of each other and therefore it is sufficient for the exclusion that the lengths are substantially a multiple of each other. Typically, it will be sufficient if the length of the longer vector is within a multiple of the shorter vector length with an uncertainty interval of ± 10% of the shorter vector length. It will be appreciated that the exclusion of a vector pair may be performed in any suitable way. For example, the vector pair may e.g. be deleted from the candidate set prior to the determination of the match measure, a very high penalty value may be added to the match measure and/or the vector pair may be excluded from the selection based on the match measures. In some embodiments, the match measure for a texture regularity vector pair may be determined in response to an angle between the vectors of the texture regularity vector pair. Thus, the match measure may be biased dependent on the angle between the vectors of the vector pair.

Specifically, as the vector pairs comprise two vectors, ti and t₂, used to describe the periodic structure of the texture, (known as the lattice), the determined vectors should be linearly independent thereby allowing other offsets to be described as linear combinations of the vectors and preventing that the two vectors of the pair correspond to different repetitions of the pattern in the same direction. Therefore, in the method of Fig. 2, the match measure may include a penalty contribution that increases for decreasing values of the angle between the vectors. Specifically, the following penalty contribution may be used:

pen_anglM,k) = ω ^• cos(2 ^■ (Zt₁ - Zt₂))

where ω may be used to set a strength of the penalty and ti and t₂ are the two vectors of the vector pair.

In the specific example, the different penalty values are combined to result in a single penalty value for the vector pair. Specifically, for a vector pair comprising two vectors (ti, t₂) the penalty value may be given as: pen{t_γ,t₂) = PeH₁ Jt₁) + pen, At₂) + pen_a At₁J₂) .

Fig. 4 illustrates an example of such a penalty.

In the specific example, the penalty value determined in this way is simply added to the similarity value (e.g. after appropriate weighting or scaling) to generate the match measure.

In some embodiments, the match measure may furthermore take into account a variance measure of at least one of the selected image block and the candidate block. Specifically, the similarity measure determined by e.g. a correlation measurement or sum of absolute differences measures may depend on the contrast and pixel value variation of the image block. However, although the contrast/ pixel variation level may be the same for all vector pairs of the candidate set, a penalty value being added thereto may have different impact depending on the absolute value of the similarity value and accordingly the penalty value and/or similarity measure may be scaled in accordance with the variance measure. It will be appreciated that the variance measure may be any suitable measure of the variation of pixel values, such as for example the variance of the pixel values, an average pixel difference measure etc.

The impact of the penalty value may in many scenarios also be dependent on the scale of the texture which is being evaluated. Therefore, in some embodiments, another algorithm may be used to provide a rough estimate of the repetition size and the penalty values or the similarity measure may be adjusted accordingly. As another example, the method may be arranged to adjust one of these values in response to a scale indication provided by a manual user input.

In some embodiments, the match measure is furthermore different for the different types of vector pairs in the candidate set. Specifically, a bias may be introduced to the reference vector pairs relative to the update vector pairs. For example, the equation or function used to determine a match measure may be different for update vector pairs and reference vector pairs. In the specific example, a penalty value may be introduced to either the update data pairs or the reference data pairs. Such a bias may be used to influence the operation of the method and can specifically be used to provide a different weighting of the vector pairs from previously processed image blocks and the variations represented by the update vector pairs. Thus, such a bias may for example be used to control the tradeoff between the convergence speed and the accuracy of the method. Specifically by weighting the reference vector pairs higher, a higher spatial consistency between vector pairs of adjacent image blocks is achieved whereas a higher weighting of the update vector pairs may result in a faster convergence speed. In some embodiments, the image is scanned a plurality of times with the method described with reference to Fig. 2 being performed for each scan of the image. Furthermore, when scanning the image, the candidate set of vector pairs may not only include vector pairs of image blocks that have already been processed in the current scan but may also include image blocks that were processed in a previous scan of the image. Also, when performing a multiple scan, the scanning patterns of subsequent scans are chosen to be different. For example, a first scan may be initiated in a left to right direction and from the top towards the bottom of the image. The subsequent scan of the image will then be from the bottom to the top and with the horizontal direction for each line being the opposite of the previous scan. This may allow the candidate set to include image blocks of the current scan which is taken from a different spatial direction than for the previous scan. Accordingly an improved convergence and accuracy, especially around edges of image segments, can be achieved.

E.g. in the example of Fig. 3, the first scan of the image may for image block 301 be in the direction indicated by the arrow 303. Accordingly, the candidate set will include the vector pairs from image blocks 305 and 307. As there are no previous scans, no other reference vector pairs are included. At the subsequent scan of the image, the image block 301 is scanned in the opposite direction of the arrow 303 (and from the bottom towards the top). Accordingly, image blocks 307 and 309 have already been processed for the current scan and accordingly the vector pairs determined for these two image blocks 307, 309 are included in the candidate set. Furthermore, although image blocks 305 and 307 have not yet been processed for the current scan, the vector pairs of these image blocks 305, 307 determined in the previous scan will be included in the candidate set. Furthermore, for each of the reference vector pairs included in the candidate set, an update vector pair is generated. Thus, at the second scan of the image, the candidate set for the image block 301 will contain eight vector pairs.

Hence, in operation the described method may process a single two- dimensional image and may without any prior knowledge or information determine texture regularity vector pairs for the image blocks. As each new image block is processed, the update vector pairs allow an adaptation and convergence towards the correct values. Furthermore, the reference vector pairs (and the fact that the update vector pairs are based on these) ensure that the generated results are spatially consistent. Furthermore, by repeating the scanning of an image using results from the previous scan and/or a different scan pattern, even the vector pairs of the initial image blocks will convert towards the correct values. Typically, a scanning of the image of two to four times will be sufficient to result in accurate vector pair determination for all areas of the image. The described approach is extremely computationally efficient and may for example enable texture estimation and processing in many real time applications. For example, based on the generated texture regularity vector pairs, the image may be segmented. As another example, noise reduction may be achieved by averaging images over different repetitions of the texture as indicated by the vector pairs. Furthermore, the generated texture regularity vector pairs are based on local characteristics and evaluation rather than on a global auto-correlation and may therefore provide more accurate local results. For example, it may allow the method to identify different textures in the same image. Furthermore, it can adapt to slight variations of regularity within textures as occur in near-regular textures e.g. under perspective transformation. The ability to track slight variations due to perspective may enable 3D shape from texture reconstruction, commonly known as shape-from-texture.

Fig. 5 illustrates an example of an image processing system which may be arranged to execute the method previously described. Specifically, the image processing system may comprise an image store 501 wherein the image to be processed may be stored. The image may for example be a frame of a video signal or may be an individual image such as a digital photo. The image store 501 is coupled to a processor 503 which is further coupled to system memory 505. The system memory 505 may contain executable code which when executed by the processor 503 results in the processor performing the described method of Fig. 2. Thus, the process of 503 may retrieve the executable code from the system memory 505 and the image data from the image store 501. The processor 503 is furthermore coupled to a result store 507 in which it can store the determined texture regularity vector pairs.

It will be appreciated that the method can be implemented in many other ways. For example, the method may be executed by an embedded digital signal processor of an image or video device. As another example, the method may be executed by an application specific integrated circuit.

Thus, it will be appreciated that the invention can be implemented in any suitable form including hardware, software, firmware or any combination of these. The invention may optionally be implemented at least partly as computer software running on one or more data processors and/or digital signal processors. The elements and components of an embodiment of the invention may be physically, functionally and logically implemented in any suitable way. Indeed the functionality may be implemented in a single unit, in a plurality of units or as part of other functional units. As such, the invention may be implemented in a single unit or may be physically and functionally distributed between different units and processors.

Although the present invention has been described in connection with some embodiments, it is not intended to be limited to the specific form set forth herein. Rather, the scope of the present invention is limited only by the accompanying claims. Additionally, although a feature may appear to be described in connection with particular embodiments, one skilled in the art would recognize that various features of the described embodiments may be combined in accordance with the invention. In the claims, the term comprising does not exclude the presence of other elements or steps. Furthermore, although individually listed, a plurality of means, elements or method steps may be implemented by e.g. a single unit or processor. Additionally, although individual features may be included in different claims, these may possibly be advantageously combined, and the inclusion in different claims does not imply that a combination of features is not feasible and/or advantageous. Also the inclusion of a feature in one category of claims does not imply a limitation to this category but rather indicates that the feature is equally applicable to other claim categories as appropriate. Furthermore, the order of features in the claims do not imply any specific order in which the features must be worked and in particular the order of individual steps in a method claim does not imply that the steps must be performed in this order. Rather, the steps may be performed in any suitable order. In addition, singular references do not exclude a plurality. Thus references to "a", "an", "first", "second" etc do not preclude a plurality. Reference signs in the claims are provided merely as a clarifying example shall not be construed as limiting the scope of the claims in any way.

Claims

CLAIMS:

1. A method of determining texture regularity vector sets for an image, the method comprising iterating the steps of: selecting (203) a selected image block; retrieving (205) a group of texture regularity vector sets for a set of previously processed image blocks of the image; generating (207) at least one update texture regularity vector set from the group of texture regularity vector sets; generating (209) a candidate set of texture regularity vectors comprising the group of texture regularity vector sets and the at least one update texture regularity vector set; for each texture regularity vector set of the candidate set generating (211) a match measure between the image block and a set of candidate image blocks being offset from the image block by vectors of the texture regularity vector set; and selecting (213) a texture regularity vector set for the image block in response to the match measures.

2. The method of claim 1 wherein the set of previously processed image blocks comprises at least one image block adjacent to the selected image block.

3. The method of claim 1 wherein the match measure is determined as a function of a pixel value similarity measure for the selected image block and the set of candidate image blocks.

4. The method of claim 1 wherein a first match measure for a first texture regularity vector set of the candidate set is determined in response to a length of at least one vector of the first texture regularity vector set.

5. The method of claim 4 wherein the first match measure comprises a penalty contribution which increases for reducing length.

6. The method of claim 4 wherein the first match measure comprises a penalty contribution which increases for increasing length.

7. The method of claim 1 wherein the match measure for a first texture regularity vector set of the candidate set is determined in response to an angle between vectors of the first texture regularity vector set.

8. The method of claim 1 wherein the match measure for a first texture regularity vector set of the candidate set is determined in response to a variance measure of at least one of the selected image block and an image block of the set of candidate image blocks.

9. The method of claim 1 wherein the iteration corresponds to a sequential processing of image blocks in accordance with a scanning pattern for the image; and the set of previously processed image blocks comprises at least one previous image block according to the scanning pattern.

10. The method of claim 1 wherein the iteration of steps corresponds to an iterated scanning of the image; and the set of previously processed image blocks comprises at least one image block of a previous scanning of the image.

11. The method of claim 10 wherein a different scanning pattern is used for at least two of the scans of the image.

12. The method of claim 1 wherein the at least one update texture regularity vector set is generated from a first texture regularity vector set of the group of texture regularity vector sets by modifying only one vector of the first texture regularity vector set.

13. The method of claim wherein a match measure for a texture regularity vector set of the candidate set is biased relative to a match measure for the at least one update texture regularity vector set.

14. The method of claim 1 further comprising the step of excluding a first texture regularity vector set of the group of texture regularity vector sets in response to the first texture regularity vector set comprising at least one vector having a length which is substantially a multiple of a length of at least one vector of a second texture regularity vector set of the group of texture regularity vector sets if a match measure of the second texture regularity vector set meets a criterion.

15. An apparatus for determining texture regularity vector sets for an image, the apparatus comprising a processor (503) for iteratively performing the steps of: selecting a selected image block; retrieving a group of texture regularity vector sets for a set of previously processed image blocks of the image; generating at least one update texture regularity vector set from the group of texture regularity vector sets; generating a candidate set of texture regularity vectors comprising the group of texture regularity vector sets and the at least one update texture regularity vector set; for each texture regularity vector set of the candidate set generating a match measure between the image block and a set of candidate image blocks being offset from the image block by vectors of the texture regularity vector set; and selecting a texture regularity vector set for the image block in response to the match measures.