FR3056800A1 - METHOD OF CHARACTERIZING AN IMAGE - Google Patents

Abstract

The present invention relates to a method for characterizing an image by means of at least one computer means comprising at least one memory and at least one processor for executing the method, said method comprising at least one step of constructing an image. efficient binary descriptor requiring less computing resources, said constructing step being based on criterion of selection of tests using a modified Shannon entropy function.

Description

TECHNICAL FIELD OF THE INVENTION

The present invention relates to image processing and more specifically to the description of objects present in an image or a scene.

io TECHNOLOGICAL BACKGROUND OF THE INVENTION

Image processing brings together image processing, analysis and characterization techniques. A good interpretation or characterization of the information contained in an image depends on the quality of the techniques used in each phase of the processing, in particular in the analysis step. The latter deals with the physical characteristics of the image, such as texture to help in the identification and extraction of the information contained in the image. The purpose of analyzing the characteristics of an image is to determine signatures or detect patterns allowing better characterization of the image.

Determining these signatures, generally called descriptors, is therefore a crucial step in the image characterization phase.

Several methods of constructing or determining the descriptors are known in image processing, each being more or less precise or more or less involving the use of significant computer resources.

Each of the methods includes at least one step of generation and selection of test functions which will make it possible to construct the descriptor.

The quality of a descriptor thus depends, in particular, on the detection and selection criteria defined by each method.

GENERAL DESCRIPTION OF THE INVENTION

The present invention aims to overcome certain drawbacks of the prior art by proposing a method for characterizing an effective image and limiting the necessary computer resources.

This object is achieved by a method for characterizing an image, obtained by an image capture device, implemented in computer means comprising at least one memory and at least one processor for executing said method which comprises:

• the definition of at least one database set comprising, on the one hand, at least one database = {// ", ..., V / _a} ^ N learning _has thumbnails from Wf of a sample of test images and, on the other hand, at least one database = [Wf, ..., Wfî _d } of description of N _d imagettes W · ¹ from a sample of the image to characterize, said databases being stored in said memory;

• the construction of a binary descriptor, this construction comprising:

the generation, by means of an algorithm executed on said processor, of a set P _t = {ti, of N _t tests tj, each of these tests being a function for comparing the characteristics of at least two pixels originating from each of the images included in the databases and d £, each test tj corresponding to an image W _L ^a or W · ¹ , a binary result x _i; · Included in {0,1};

- the offline selection of optimum tests on said learning database;

the construction of said binary descriptor being characterized in that the offline selection of the optimum tests tf, best describing at least a given part of the image and / or the image, is based on the search for the maximum of the entropy function H _m (t) of modified Shannon representing an amount of information associated with the probability of success of a test, said entropy function H _m (t) being weighted by the probability that said information associated with said test is kept if the database d is changed

According to another particular feature, the modified entropy function H _m (t) representing a quantity of measured information is defined by:

= ~ [Px _t ^lo § (Pxt) + ( ¹ - Pxt) M ¹ - PXt)] Pye ¹⁰ Px _t = P (N = 1) designating the probability of success of a test t, and p _yt the probability that the information associated with the t test is kept if the learning database is changed, X _t = t (W) being a Bernoulli random variable and W a random variable from the learning database

According to another particular feature, the construction of the binary descriptor also comprises the online selection, by application to said description database D ^, of the optimum tests obtained in the offline selection, the binary descriptor being defined by the set of values binaries x _i; · Obtained after said online selection and at least part of the optimum tests tf.

According to another particularity, the implementation in the offline selection, on computer means, of the criterion for selecting the optimum tests includes the selection of at least one invariant or robust test to any geometric transformation of the thumbnails of a database , thus ensuring good optimization of the characterization process.

According to another particularity, the online selection of optimum tests

N retained after the offline selection, consists in estimating the similarities between said thumbnails of the description database after applying at least one geometric transformation on said thumbnails.

According to another particular feature, the process of estimating the similarities between at least two patches Wf ¹ and V / 2 ^d of the description database DÛ comprises at least the calculation of a distance dH of Hamming between the respective result sets x _± = = l, N _t ) and x ₂ = of said thumbnails, obtained by applying at least one series of N _t optimum tests on said thumbnails, stored in at least one memory.

According to another feature, the calculation of the distance _H between two sets of x results _± and x ₂ two thumbnails, defined by d ^ x ^ xR = ® * 2; puts ^implement simultaneously reducing the number of set of results and the number of optimum tests, and the conservation of invariant or robust tests to at least one geometric transformation of the thumbnails, the descriptor being composed of the most robust sets of results and tests remaining.

DESCRIPTION OF ILLUSTRATIVE FIGURES

Other particularities and advantages of the present invention will appear more clearly on reading the description below, made with reference to the appended drawings, in which:

- Figures 1a and 1b respectively represent the diagram of the image characterization process and the step of construction of the binary descriptor, according to one embodiment;

- Figure 2 shows the diagram of a "ROC curve" characterizing the performance of the binary descriptor obtained after the offline selection step;

- Figures 3a and 3b show the area diagrams under the "ROC curve" as a function of the angle of rotation of the Leuven and Bikes image samples respectively.

DESCRIPTION OF THE PREFERRED EMBODIMENTS OF THE INVENTION

The present invention relates to a method for characterizing an image or a scene. Such a characterization makes it possible to detect and highlight objects or elements of the image, such as for example, a target to be detected or monitored or a defect to be identified, etc.

The invention makes it possible in particular to establish links between phases of two successive images of a video stream (for example for augmented reality).

In certain embodiments, the method (1, FIG. 1a) for characterizing an image, obtained by an image capture device, is implemented in computer means comprising at least one memory and at least one processor for carry out said process which comprises:

• the definition (E1) of at least one set of databases comprising, on the one hand, at least one learning = database of N _a Wf imagettes from a sample of test images and, d on the other hand, at least one database = [Wf, ..., wfî _d } of description of N _d thumbnails W · ¹ from a sample of the image to be characterized, said databases being stored in said memory;

• the construction (E2, Figure 1a and 1b) of a binary descriptor, this construction comprising:

the generation (E21), by means of an algorithm executed on said processor, of a set P _t = (u, of N _t tests tj, each of these tests being a function for comparing the characteristics of at least two pixels from each of the thumbnails included in the databases and D $, each test tj matching a thumbnail Wf or wf, a binary result x _i; · included in {0,1};

îo - the selection (E22) offline of optimum tests on said learning database;

the construction of said binary descriptor being characterized in that the offline selection of the optimum tests tf, best describing at least a given part of the image and / or the image, is based on the search for the maximum of the entropy function H _m (t) of modified Shannon representing an amount of information associated with the probability of success of a test, said entropy function H _m (t) being weighted by the probability that said information associated with said test is kept if the database d learning is changed.

The process of characterizing an image or a scene, includes in part the description of each thumbnail or patch of the image and / or of a point or a set of points included in said thumbnail or said patch . A test can include, for example and without limitation, the comparison of two pixel intensity values around a point that one wishes to characterize in a given thumbnail or patch, for example of size 32x32 . Two tests thus differ in the pixel intensity values that they compare within a patch. The application of each test on each patch allows the binarization of the image, that is to say the description of each point of the image in terms of binary digits included in the set {0,1}. For example and in a nonlimiting manner, if one wishes to describe a particular region of an image, one can create patches of different or identical sizes, around the region which one wishes to describe. Tests are then generated to measure the intensities of a set of pixels within each patch. If it emerges from the comparison of two pixels p1 and p2, included in a test, that the pixel p1 is larger than the pixel p2, said test returns the value 1, otherwise the value returned is 0. The application of all testing a given thumbnail or any given region of the image thus makes it possible to define a set of results or binary values {1, 0, ..., 0, 1, 0} which makes it possible to characterize said thumbnail or said image region. The result of a test can take the value 1 or 0, it follows a binomial law of Bernoulli, in which when the result is 1 we say that the test is a success and if not, that is to say if the result is zero, it is said to be a failure.

The construction (E2) of the binary descriptor generally comprises, in addition to the offline filtering or selection (E22), an online filtering or selection (E23) defined by application of optimum tests, obtained in the offline selection (E22), on said description database The binary descriptor is defined by the set of binary values x _i; · Obtained after said selection (E23) online and at least part of the optimum tests tf. However, the present invention relates more particularly to the offline selection phase (E22).

The descriptor, consisting of all the tests and all the binary results, constructed in the offline selection step (E22) is generally called a global descriptor and is concerned with the description or characterization of the patch or the image as a whole. This phase can thus contain a high number of test functions generated, randomly, by means of an algorithm. In practice, a number of the order of 10,000 tests can for example be chosen to carry out the characterization of the patch or of the image. Offline filtering or selection (E22) generally consists of keeping only tests with a large binomial variance, said variance, which characterizes the dispersion of test results around an average value, being the selection criterion such as for example in the so-called BOLD method. The greater the variance of a test, the better it discriminates, that is to say, it makes it possible to better distinguish the differences between two distinct regions of the image or patch. For the filtering of tests in the offline selection step (E22) we preferably choose the entropy function H _m (t) defined above as selection criteria which, unlike the selection criterion based on variance, makes it possible to considerably reduce the number of tests necessary for the characterization of the thumbnail or of the image. In fact, the weighting of the entropy function by the probability that the information associated with a test is preserved makes it possible to retain only the tests whose result will remain invariant if the database is changed. We are thus guaranteed to select optimum tests. In addition, the tests thus chosen by this method will be inclined to be more robust to geometric transformations, for example rotations, within the patches and therefore not to be rejected by the online part.

The effectiveness of a descriptor is, in general, evaluated by means of a “ROC curve” (from the English Receiver Operating Characteristic, which is a measure of the performance of any system or model whose objective is to categorize elements of a given set, for example the binary descriptor. The "OCR curve" is represented as a curve which gives the rate of true positives as a function of the rate of false positives of the tests which make up the binary descriptor. A true positive is a result which verifies the criterion fixed by a test, this criterion possibly being, for example, a threshold of pixel value to be increased. On the other hand, a false positive is a result declared positive, that is to say meeting the criterion fixed by a test where it is actually negative, that is to say not meeting the criterion fixed by said test. The closer the curve is to the axis of true positives, the more efficient it is. Figure 2 shows, for example, the performance of the descriptor constructed in after the offline selection steps (E22) and online (E23) which we compare with that of the BOLD descriptor using variance as the selection criterion for tests. As can be seen, the descriptor of the present application best characterizes, with less testing, the sample of images on which the performance of the two descriptors was evaluated. For example, it takes twice as many tests in the BOLD descriptor, using tests selected on the criterion of variance, to reproduce the same performance as the descriptor of the present application which uses 256 tests. This saves computing time on the computer and also in data storage capacity on the memories.

In certain embodiments, the modified Shannon entropy function H _m (t) representing a quantity of measured information, is preferably defined by:

= ~ [Px _t ^lo § (Pxt) + ( ¹ - Pxt) ^lo g (l - PXt)] Pye 15 p _Xt = p (X _t = 1) designating the probability of success of a test t, and p _yt the probability that the information associated with the t-test will be kept if the learning database is changed, X _t = t (W) being a Bernoulli random variable and W a random variable from the learning database D %. The quantity p _yt is the weight making it possible to find the optimum tests. The optimization of the function defined above consists in finding the tests for which said function has a maximum value.

In certain embodiments, the implementation in the selection (E22) offline, on the computer means, of the criterion for selecting the optimum tests comprises the selection of at least one invariant or robust test to any geometric transformation of the thumbnails of a database, thus ensuring good optimization of the characterization process.

ίο

In certain embodiments, the selection (E23) online of the optimum tests tf, retained after the selection (E22) offline, consists in estimating the similarities between said thumbnails of the description database D $ after application of at minus a geometric transformation on said thumbnails.

The second step (E23) of construction of the binary descriptor, is based on the optimum tests tf selected in the first step, that is to say in the selection step (E22) offline. The descriptor constructed in the online filtering or selection step (E23), generally called local descriptor, allows an effective characterization of a given region of the patch or of the image. The second step (E23) makes it possible to reduce the number of tests preselected in the filtering or selection step (E22) offline.

A geometric transformation can be, for example and without limitation, a rotation of the patch or of the image. The tests which are invariant, during a geometric transformation, are said to be robust and kept to form with the set of results the local descriptor.

During the construction of the descriptor, after the online selection step (E23), a higher number of robust tests are retained in comparison with a BOLD type method. In fact, in a BOLD type method, robust tests may already be rejected in the offline selection phase, which is not the case with the offline selection method (E22) of the present invention, because the test selection criterion defined by the entropy function H _m (t) makes it possible to better choose the tests likely to be robust to certain geometric transformations. Thus, as the most robust tests allow to obtain the optimal information for the description of the thumbnail or of the image, it is therefore possible to have results similar to those obtained in an approach similar to that of BOLD, and this, with a number of tests reduced by half in the descriptor of the present invention (FIG. 2).

In some embodiments, the process of estimating similarities between at least two patches Wf and W ₂ of the description database includes at least the calculation of a Hamming distance d _H between the respective result sets x _± = [χ · ^, i = l, N _t } and x ₂ = ⁵ [ ^χ 2, ί> ί = TN _t } of said thumbnails, obtained by applying at least a series of N _t optimum tests on said thumbnails, stored in at least one memory.

In certain embodiments, the calculation of the distance d _H , between two sets of results x _± and x ₂ of two patches, defined by d _H (xi> ^x 2) = Zfji * i, i ® * 2 ,; ^and evaluated by means of an algorithm executed on the processor of at least one computer means, said algorithm being able to be for example and without limitation the “popcount” algorithm which consists in counting the number of bits at 1 in a word or a series of bits.

Said calculation simultaneously implements the reduction of the number of set of results and the number of optimum tests, and the conservation of invariant or robust tests to at least one geometric transformation of the thumbnails, the descriptor being composed of the sets of results and of the tests the most robust remaining.

The Hamming distance d _H between the result sets x _± and x ₂ is the total number of times the results x _{± i} and x _2i differ. The symbol "Φ" represents the logical operator "or exclusive". If the variable x _{± i} differs from the variable x _2i then the operation x _{± i} θ x _2i gives 1, otherwise the result is 0. So if the Hamming distance d _H is zero it means that the patches are similar, the N _t test series is considered robust and therefore retained. Otherwise the tests are rejected.

The performance of the binary descriptor obtained in the online filtering or selection step (E23) is evaluated by calculating, for example, the area under the "ROC curve" obtained on a sample of images, after a geometric transformation, for example a rotation. The area is then represented according to the angle of rotation. The larger the area under the curve, the more efficient the descriptor. In FIGS. 3a and 3b are shown the curves of the areas under the "ROC curves" as a function of the angles of rotation of two image samples ranging from 5 to 30 °. As can be seen, the descriptor of the present application best characterizes the two image samples (Leuven and Bikes) in comparison to the BOLD model or descriptor, and this in general, for angles between 15 and 30 °.

The present application describes various technical characteristics and advantages with reference to the figures and / or to various embodiments. Those skilled in the art will understand that the technical characteristics of a given embodiment can in fact be combined with characteristics of another embodiment unless the reverse is explicitly mentioned or it is obvious that these characteristics are incompatible or that the combination does not provide a solution to at least one of the technical problems mentioned in the present application. In addition, the technical characteristics described in a given embodiment can be isolated from the other characteristics of this mode unless the reverse is explicitly mentioned.

It should be obvious to those skilled in the art that the present invention allows embodiments in many other specific forms without departing from the scope of the invention as claimed. Therefore, the present embodiments should be considered by way of illustration, but may be modified in the field defined by the scope of the appended claims, and the invention should not be limited to the details given above.

Claims (7)

1. Method (1) for characterizing an image, obtained by an image capture device, implemented in means 5 computers comprising at least one memory and at least one processor for executing said method which comprises: • the definition (E1) of at least one set of databases comprising, on the one hand, at least one database D% = [Wf, ..., Wp} _a } for learning N _a thumbnails Wf from a sample of test images and, on the other hand, at least one database = [Wf, ..., Wfî _d } of description of N _d Wf thumbnails from a sample of image to be characterized, said databases being stored in said memory; • the construction (E2) of a binary descriptor, this construction 15 including: the generation (E21), by means of an algorithm executed on said processor, of a set P _t = {t _x , of N _t tests tj, each of these tests being a function for comparing the characteristics of at least two pixels from Each of the thumbnails included in the databases and d £, each test tj corresponding to a thumbnail Wf or a binary result x _i; · Included in {0,1}; - the selection (E22) offline of optimum tests on said 25 learning database; the construction of said binary descriptor being characterized in that the offline selection of the optimum tests tf, best describing at least a given part of the image and / or the image, is based on the search for the maximum of the entropy function H _m (t) modified Shannon representing a quantity 30 of information associated with the probability of success of a test, said entropy function H _m (t) being weighted by the probability that said information associated with said test is retained if the training database is changed. 2. Method (1) for characterizing an image according to claim 5 1, characterized in that the modified entropy function H _m (t) representing a quantity of measured information is defined by: = ~ [Px _t log (Px _t ) + (1 - P _Xt ) log (l - Pxt)] Py _e p _Xt = p (X _t = 1) designating the probability of success of a test t, and p _yt the probability that the information associated with the t test will be kept if the learning database is changed, X _t = t (W) being a random io variable of Bernoulli and W a random variable of the learning database 3. Method (1) for characterizing an image according to claim 1, characterized in that the construction of the binary descriptor also comprises the selection (E23) online, by application on said base 15 description data D $, optimum tests obtained in the selection (E22) offline, the binary descriptor being defined by the set of binary values x _i; · Obtained after said selection (E23) online and at least part of the optimum tests tf. 4. Method (1) for characterizing an image according to one of the 20 claims 1 to 2, characterized in that the implementation in the selection (E22) offline, on the computer means, of the selection criterion for the optimum tests comprises the selection of at least one invariant or robust test to any geometric transformation thumbnails of a database, thus ensuring good optimization of the 25 characterization process. 5. Method (1) for characterizing an image according to one of claims 2 and 4, characterized in that the selection (E23) online of the optimum tests tf, retained after the selection (E22) offline, consists in estimate the similarities between said imagettes of the description database D $, after application of at least one geometric transformation on said imagettes. 6. Method (1) for characterizing an image according to claim 5 4, characterized in that the process of estimating similarities between at least two patches Wf and W ₂ of the description database comprises at least the calculation of a distance d _H of Hamming between the respective result sets x _± = [x _ld , i = l, N _t j and x ₂ = [x _{2 /} i, i = 1, N _t } of said thumbnails, obtained by applying at least one series of io N _t optimum tests on said thumbnails, stored in at least one memory. 7. Method (1) for characterizing an image according to claim 5, characterized in that the calculation of the distance d _H between two sets of results x _± and x ₂ of two thumbnails, defined by £ / _η (χ _1λ χ ₂₎ = ¹⁵ Σ ^ ι ^ ι, ίΦ ^χ 2, ί, ^sets simultaneously work to reduce the number of overall results and optimum number of tests, and conservation of invariant tests or robust to at least one transformation geometric thumbnails, the descriptor being composed of the most robust result sets and tests remaining. 1/3