MXPA01002841A - Method and device for displaying or searching for object in image and computer-readable storage medium - Google Patents

Abstract

A method for displaying an object appearing in an image by processing a signal corresponding to a still or video image comprising a step of deriving a curvature scale space (CSS) display of the outline of an object by smoothing the outline of the object, a step of deriving at least one additional parameter reflecting the distribution of the shape or size of a version obtained by smoothing the original curve, and a step of correlating the CSS display with the additional parameter serving as a shape descriptor of the object.

Description

COMPUTED SYSTEM, PROGRAM CO PUTARI ZAPO, APPARATUS, METHOD AND STORAGE THAT CAN BE READ BY COMPUTER TO REPRESENT AND EXPLORE A OBJECT IN AN IMAGE Technical Field The present invention relates to the representation of an object appearing in a still or video image, such as an image stored in a multimedia database, especially for search purposes, and with a method and apparatus for scanning. an object that uses such representation.

BACKGROUND OF THE INVENTION In applications such as image or video libraries, it is desired to have an efficient representation and storage of the sketch or configuration of objects or parts of objects that appear in still or video images. A well-known technique for indexing and shape-based recovery uses the representation of Curvature Scale Space (CSS). Details of the representation of CSS can be found in the documents "Robus t and Efficient Shape Indexing through Curvature Scale Space" Proc. British Machine Vision conference, pp. 53-62, Edinburgh, UK, 1996 and "Indexing an Image Datábase by Shape Content using Curvature Scale Space" Proc. IEE Colloquium on Intelligent Databases, London 1996, both by F. Mokhtarian, S.? Bbasi and J. Kittler, the contents of which are incorporated herein by reference. The CSS representation uses a curvature function for the sketch of the object, starting from an arbitrary point in the sketch. The curvature function is studied as the sketch configuration is wrapped by a series of deformations that smooth the configuration. More specifically, the crossing of curves of the derivative of the curvature function with vo 1 u s ona with a family of Gaussian filters is computed. Curve junctions are plotted on a graph, known as the Curvature Scale Space, where the x-axis is the normalized arc length of the curve and the y-axis is the evolution parameter, specifically, the filter parameter applied. The strokes in the graph form the characteristic loops of the sketch. Each convex or concave part of the object sketch corresponds to a loop in the CSS image. The coordinates of the peaks of the most prominent loops in the CSS image are used as a representation of the bos that j or ~. To explore objects in stored images "in a database that matches the configuration of an input object, the CSS representation of an input configuration is calculated.The similarity between an input configuration and stored forms is determined by comparing the position and height of the peaks in the respective CSS images using a correspondence algorithm It is also known from the first document mentioned above that it uses two additional parameters, the circularity and the eccentricity of the original configuration, which reject the process configurations of correspondence with significantly different circularity and eccentricity parameters A problem with representation as described in the above is that exact recovery is sometimes deficient, especially for curves that have a small number of concave cavities or convex cavities. particular, the representation Ion can not distinguish between several convex curves. One aspect of the present invention is to introduce an additional means to describe the configuration of the "prototype contour configuration". The prototype contour configuration is defined herein preferably as: 1) the original configuration, if there are no convex cavities or concave cavities in the contour (ie, there are no peaks in the CSS image), or 2) the outline of the configuration after smoothing equivalent to the highest peak in the CSS image. Note, that the prototype contour configuration is always convex. For example, the configuration of the prototype contour can be described by means of the variants based on region moments such as ib >; e in the document "Visual Pattern Recognition by Moments Invariants", IEEE Transaction on Information Theory, Vol. IT-8, 179-187, 1962 by M.K. Hu, the contents of which are incorporated herein by reference or use the Fourier descriptors as described in the document "On Image Analysis by the Methods of Moments", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 1- 0, No. 4, July 1988, by Cho-Huak The, the contents of which are incorporated herein for reference, or parameters such as eccentricity, circularity, etc. In the known method mentioned in the above, the eccentricity and circularity is only used in relation to the original configuration. In the present it is used in relation to "a prototype configuration", which is different for curves that have at least one CSS peak. Another difference is that the known method of eccentricity and circularity are used to reject certain configurations of similarity correspondence, and in the present they are used (in addition to the CSS peaks) to derive the value of the similarity measure. Finally, the additional parameters used in the process of correspondence to the current invariants, Fourier descriptors and Moments of Z e r n i ke are extended. As a result of the invention, accurate recovery can be improved.

Description of the Invention A method for representing an object appearing * in "a still or video image, processing signals corresponding to the image set forth in claim 1, the method comprising deriving a curvature scale space representation (CSS) of the object sketch smoothing the object sketch, deriving at least one additional parameter that reflects the configuration or distribution of mass from trn to smoothed version of the original curve, and associating the CSS representation and the additional parameter as a configuration descriptor of the object In a method set forth in claim 2, an additional parameter relates to "the smoothed sketch corresponding to a peak in the CSS image. In a method set forth in claim 3, an additional parameter relates to the smoothed sketch that corresponds to the highest peak in the CSS image. In a method set forth in claim 4, an additional parameter corresponding to the eccentricity of the sketch.

In a method established in rei indication 5, an additional parameter corresponds to the circularity of the sketch. In a method set forth in claim 6, at least one additional parameter uses a representation based on the region. In a method set forth in claim 7, an additional parameter is a region moment invariant. In a method set forth in claim 8, an additional parameter is based on Fourier descriptors. In a method set forth in claim 9, an additional parameter is based on Zernike moments of the region enclosed by the sketch. A method for representing a plurality of objects appearing in a still or video image, processing signals corresponding to the As set forth in claim 10, the method comprises, for each object sketch, determining whether there are significant changes in curvature in the object sketch, and, if there are significant changes in curvature of the object sketch, then deriving a configuration descriptor using a method as claimed in any one of claims 1 to 9 and, if there are no significant changes in curvature of the object sketch, then derive a configuration descriptor including at least the additional parameter that reflects the configuration of the object sketch. In a method set forth in claim 11, the additional parameter for an object sketch that does not have significant changes in curvature is based on region moment invariants, Fourier descriptors or Zernike moments of the sketch. A method for scanning an object in a still or video image by processing signals corresponding to images set forth in claim 12, the method comprises entering a query in the form of a two-dimensional sketch, deriving a descriptor from the sketch using a method such as is claimed in any of claims 1 to 11, and comparing the query descriptor with each descriptor for stored objects using a matching procedure that uses the CSS values and the additional parameters to derive a similarity measure, and select and display by at least one result corresponding to an image containing an object for which the comparison indicates a degree of similarity between the query and object. In a method set forth in claim 13, the similarity measure is based on M where M = a * GP-S + CSS-S where GP-S is the measure of similarity between the additional parameters of the compared object sketches and CSS -S is the measurement of similarity between the CSS values for the compared object sketches, it is already a constant. In a method set forth in claim 14, a depends on the number and height of the CSS peaks. In a method set forth in claim 15, a = 1 where there are no CSS peaks associated with either the sketch and a = 0 where at least one sketch has a CSS peak. A method for scanning an object in a still or video image by processing signals corresponding to images set in the re-indication 16, the method comprises calculating a similarity measure between two object sketches using a CSS representation of the sketches and additional parameters that reflect the configuration of mass distribution within the original sketch or a smoothed version of the bos j or j. An apparatus set forth in claim 17 is adapted to implement a method as claimed in any of claims 1 to 16. A computerized program set forth in claim 18 implements a method as claimed in any of the rei indications 1 to 16. A The computerized system set forth in claim 19 is programmed to operate in accordance with a method as claimed in any of claims 1 to 16. A computer readable storage medium set forth in claim 20 stores steps of processing that can be executed by computer to implement "a method as claimed in any of claims 1 to 16.

A method for representing objects in still images or video set in the rei indication 21 is described with reference to the accompanying drawings. A method for scanning objects in still images or video set forth in claim 22 is described with reference to the accompanying drawings. A computerized system set forth in claim 23 is described with reference to the accompanying drawings.

Brief Description of the Drawings Fig. 1 is a block diagram of a video database system; Figure 2 is a drawing of a sketch of an object; and Figure 3 is a CSS representation of the sketch of Figure 2 BEST MODE FOR CARRYING OUT THE INVENTION First Mode Figure 1 shows a computerized video database system according to one embodiment of the invention. The system includes a control unit 2 in the form of a computer, a display unit 4 in the form of a monitor, a pointing device 6 in the shape of a mouse, an image database 8 that includes still images and stored video and a descriptor database 10 that. stores descriptors of objects or parts of objects that appear, in images stored in the image database 8. A descriptor for the configuration of each object of interest that appears in "an image in the image database is derived by means of the control unit 2 and stored in the descriptor database 10. The control unit 2 derives the descriptors that operate under the control of a suitable program imp 1 by using a method as described below: First, for a given object sketch, a CSS representation of the sketch is derived.This is done using the known method as described in one of the aforementioned documents, more specifically, the sketch is expressed by a representation of? = { (x (u), ue [0, 1].}. where u is a length parameter of arc no rma 1 iz ado.

The sketch is smoothed together with one? with a central Gaussian ID module g (u, s), and the crossing of curvature curves of the envelope curve are examined as changes s. The crossing of curves is identified using the following expression for the curve: s) ~. { XÁu, a? + YÁu, s? where X (u, s) = x (u) * g (u, s) Y (M, s) = y (u) * g (u, s) Xu (U, 0-) -? (U) * g (U> s) xu¡, (u < s) =? (U) * gu? L < > s) In the above, * represents convolusion and subscribers represent derivatives. The number of changes of curves crossing curves as changes s, and when s is high enough? It is a convex curve with no crossing of curves. Curve crossing points (u, s) are plotted on a graph, known as the CSS image space. This results in a plurality of characteristic curves of the original sketch. The peaks of the characteristic curves are identified and the corresponding coordinates are extracted and stored. In general terms, this gives a set of pairs of coordinates n [(xl, yl), (x2, y 2), ... (xn, yn)], where n is the number of peaks, and xi is the position of arc length of the peak of ith and yi is the high peak. These peak coordinates constitute the representation of CSS. In addition to the CSS representation, additional parameters are associated with the configuration to produce the configuration descriptor. In this mode, the additional parameters are the eccentricity and circularity of the "prototype region" for the configuration, where the "prototype region" of the configuration is the contour of the configuration after the final smoothing stage, that is, in the Equivalent point for the highest peak value s. Other values of s can be selected for the prototype region. This results in a configuration descriptor for an S configuration in the form:. { EPR, CPR, PEAKS} , where EPR represents the eccentricity of the prototype region, CPR represents the circularity of the prototype region, and PEAKS represents the CSS representation.

A method for scanning an object in an image according to an embodiment of the invention will now be described. In the present, the system descriptor database 10 of Figure 1 stores the derived configuration descriptors according to the method described in the above. The user initiates a search by drawing an object sketch on the screen using the pointing device. The control unit 2 then derives a configuration descriptor from the input sketch in the manner described in the above. The control unit then performs a correspondence comparison with each configuration descriptor stored in the database. Consider that the input sketch, the SI configuration, is being compared with a stored S2 configuration, SI and S2 that are respective descriptors: SI:. { EPR1, CPR1, PEAKS1} S2:. { EPR2, CPR2, PEAKS2} Where EPR means Eccentricity of the prototype region and CPR means Circularity of the prototype region, and PEAKS means the set of peak coordinates in the CSS image (the set may be empty). The measurement of similarity between two configurations is computed as follows. M = a * abs ((EPR2-EPR1) / (EPR2 + EPR1)) + h * abs ((CPR2-CPR1) / ((C PR2 + C PR1)) + SM (PE AKS 1, PEAKS2) Where a and b are two coefficients and SM is the standard similarity measure defined in the two sets of peaks [1] / and a b s represents the absolute value. SM is calculated using a known matching algorithm such as that described in the aforementioned documents that can be used. This correspondence procedure is briefly described below. Given the two closed contour configurations, the image curve? I and the model curve ? m and its ^ respective sets of peaks . { (xil, yil), (xi2, yi2),. .., (xin, yin)} Y . { (xml, yml), (xm2, ym2),. . ., (xmn, ymn)} the measurement of. Similarity is calculated. The similarity measure is defined as a total cost of peak matching in the model within the peaks in the image. The correlation that decreases the total cost is determined using dynamic programming. The algorithm agrees to calculate the peaks from the model to the peaks from the image and calculates the cost of each correspondence. Each peak model can correspond to only one model peak. Part of the model and / or image peak may follow without correspondence, and there is an additional penalty cost for each peak without correspondence. Two peaks can be matched if the horizontal distance is less than 0.2. The cost of a correspondence is the length of the straight line between the two agreed peaks. The cost of a peak without correspondence is its height. In more detail, the algorithm works by creating and extending a tree-like structure, where the nodes correspond to the agreed peaks: 1. Create the start of the node consisting of the longest maximum of the image (xik, yik) and the longest maximum of the model (xir, yir) 2. For each peak model that remains which is within 80 percent of the largest maximum of the image peaks creates an additional start node. 3. Start the cost of each start node created in 1 and 2 for the absolute difference of the y coordinate of the image and the model peaks bound by its node. 4. For each initiation node in 3, compute the CSS-change alpha parameter, defined as the difference in x coordinates (horizontal) of the model and the image peaks agreed on its start node. The change parameter will be different for each node. 5. For each start node, create a list of model peaks and a list of image peaks. The list of sustained information whose peaks are already to be agreed upon. for each start, the node mark peaks agreed on this node as "agreed", and all other peaks "not matched". 6. Expand resources and value a lower cost node (start from each node created in steps ~ 1-6 and continue with its dependent nodes) until the condition at point 8 is filled. To expand a node use the following procedure: 7. Expand a node: If there is at least one image and a peak model left without correspondence: select the longest scale image curve of maximum CSS which is not agreed (xip, yip) • Apply the start node change parameter (computed in step 4) to create a map of the maximum selected to the CSS image models - now the selected peak has the coordinates (xip-alpha, yip) • Locate the nearest model curve peak that is not matched (xms, yms). If the horizontal distance between the two peaks is less than 0.2 (e say: | xip-alpha-xms | <; 0.2), match two peaks and define the cost of correspondence as the length of the straight line between the two peaks. Add the cost of the correspondence to the total cost of that node. Remove the agreed peaks from the respective lists by marking them as "agreed". If the horizontal distance between the two peaks is greater than 0.2, the image peak (can not be matched). In that case, add your height yip to the total cost and remove only the peak (xip, yip) from the image peak list, marking it as "agreed". Otherwise (there are only image peaks or there are only peaks left without correspondence): Define the cost of the correspondence as the height of the image without higher sponge or model peak and remove that peak from the list. 8. If after expanding a node in 7 there are no peaks without correspondence in the image and model lists, the matching procedure is determined. The cost of this node is the measurement of similarity between the image curve and the mole it. Otherwise go to point 7 and expand the lowest cost node. The above procedure is repeated with the image curve peaks and the model curve peaks exchanged. The final matching value is the lower of the two. The previous stages are repeated for each model in the data hase. The similarity measurements result from the correspondence comparisons that are ordered and the objects corresponding to the descriptors that have similarity measurements that indicate the closest correspondence (that is, the -measures of lowest similarity) then are displayed in a deployment unit 4 for one user. - The number of objects that can be displayed can be selected by the user. In an alternative implementation, different parameters can be used to describe the configuration of the "prototype region". For example, three Fourier coefficients of the curve can be used. The similarity measure can be defined as follows: M = a * EUC (Fl, F2) + SM (PE AKS 1, PE AKS 2) Where EUC is a distance "Euclidean between vectors Fl and F2 formed from the three coefficients Fourier's main model and image configuration, a is a constant, and SM represents the similarity measure for the CSS peaks, calculated using a method essentially as described above.

Industrial Applicability A system according to the invention can, for example, be provided in an image library. Alternatively, the databases can be placed away from the system control unit, connected to the control unit by means of a temporary link such as a telephone line or through a network such as the Internet. The image and descriptor databases may be provided, for example, in permanent storage or in a portable data storage medium such as CD-ROMs or DVDs.

The system components as described can be provided * in a software or hardware form. Although the invention has been described in the form of a computerized system, it can implode in other ways, for example, using a dedicated chip. Specific examples have been given of methods for representing a 2D configuration of an object and of methods for calculating values that represent similarities between two configurations although any suitable method can be used. The invention can also be used, for example, to match images of objects for verification purposes, or for filtering.

Claims (20)

1. CLAIMS 1. A method to represent an object that appears in a still or video image, processing signals that correspond to the image, the method involves deriving a representation of curvature scale space (CSS) from the object sketch smoothing out the sketch of object, deriving at least one additional parameter that reflects the configuration or mass distribution of a smoothed version of the original curve, and associating the 'CSS representation and the additional parameter as an object configuration descriptor.
2. 2. A method as claimed in claim 1, wherein an additional parameter relates to the smoothed sketch that corresponds to a peak in the CSS image.
3. 3. A method as claimed in claim 2, wherein an additional parameter is related to the smoothed sketch that corresponds to the high peak in the CSS image.
4. 4. A method as claimed in any of claims 1 to 3, wherein an additional parameter corresponds to the eccentricity of the sketch.
5. 5. A method as claimed in any of the rei indications 1 to 4, wherein an additional parameter corresponds to the circularity of the bos that j o.
6. 6. A method as claimed in any of claims 1 to 5, wherein at least one additional parameter uses a region-based representation.
7. 7. A method as claimed in claim 6, wherein an additional parameter is a region moment invariant.
8. 8. A method as claimed in claim 6 or claim 7, wherein an additional parameter is based on Fourier descriptors.
9. 9. A method as claimed in claim 6, wherein an additional parameter is based on Zernike moments of the region enclosed by the sketch.
10. 10. A method to represent a plurality of objects that appear in a fixed image? video, processing signals that correspond to the images, the method comprises, for each object sketch, determine if there are significant changes in curvature in the object sketch and, if there are significant changes in curvature of the object sketch, then derive a descriptor of configuration using a method as claimed in any of claims 1 to 9 and, if there are no significant changes in curvature of the object sketch, then derive a configuration descriptor including at least one additional parameter reflecting the confi gu " r ation of the object sketch.
11. 11. A method as claimed in claim 10, wherein the additional parameter for an object sketch that has no significant change in curvature is based on region moment variants, Fourier descriptors or Zernike moments.
12. 12. A method for scanning an object in a still or video image, processing signals that correspond to images, the method comprises entering a query in the form of a two-dimensional sketch, deriving a descriptor from the sketch using a method as claimed in any of claims 1 to 11, and comparing the query descriptor with each descriptor for stored objects using a matching procedure using the CSS values and the additional parameters to derive a similarity measure, and selecting and displaying at least one result that corresponds to an image that contains an object for which the comparison indicates a degree of similarity between the query and the object.
13. 13. A method as claimed in claim 12, wherein the similarity measurement is based on M where M = a * GP-S + CSS-S where GP-S is the similarity measure between the additional parameters of the object sketches compared and CSS-S is the measurement of similarity between CSS values for the compared object sketches, it is already a constant.
14. 14. A method as claimed in claim 13, wherein a depends on the number and height of the CSS peaks.
15. 15. A method as claimed in claim 13, or claim 14 wherein a = 1 where there are no CSS peaks associated with the sketch and a = 0 when at least one sketch has a CSS peak.
16. 16. A method for scanning an object in a still or video image by processing signals that correspond to images, the method involves calculating a similarity measurement between two object sketches using a CSS representation of the sketches and additional parameters that reflect the configuration of or the mass distribution within the original sketch or the smoothed version of the sketch.
17. 17. An apparatus adapted to implement a method as claimed in any one of the claims is 1 to 16.
18. 18. A computerized program for implementing a method as claimed in any of claims 1 to 16.
19. 19. A computerized system programmed to operate according to a method as claimed in any of claims 1 to 16.
20. 20. A storage medium that can be read by computer that stores process steps that are executed by computer to implement a method as claimed in any of the rei indications 1 to 16.