EP1543444A2 - Procede et dispositif de mesure de similarite entre images - Google Patents
Procede et dispositif de mesure de similarite entre imagesInfo
- Publication number
- EP1543444A2 EP1543444A2 EP03780266A EP03780266A EP1543444A2 EP 1543444 A2 EP1543444 A2 EP 1543444A2 EP 03780266 A EP03780266 A EP 03780266A EP 03780266 A EP03780266 A EP 03780266A EP 1543444 A2 EP1543444 A2 EP 1543444A2
- Authority
- EP
- European Patent Office
- Prior art keywords
- class
- images
- segments
- image
- histogram
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/5854—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using shape and object relationship
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/469—Contour-based spatial representations, e.g. vector-coding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
- G06V10/507—Summing image-intensity values; Histogram projection analysis
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99933—Query processing, i.e. searching
- Y10S707/99936—Pattern matching access
Definitions
- the invention relates to a method and device for measuring similarity between images.
- the field of application is that of calculating the similarity between images.
- This similarity can be used when searching for images by similarity. It can also be used to estimate groups of close images according to the similarity criterion used. This is for example the construction of video summaries. This consists of a search for similar images in a database made up of images from a video sequence. The goal is to reduce the number of these images by keeping only one representative for a group of images declared to be similar. It is also about indexing, which consists in selecting index images based on their similarity to other images. or simply to search for specific images in the video sequence, from query images.
- MPEG-7 and the algorithms for processing these attributes for the measurement of similarity do not allow, for certain types of sequences, to detect similar images satisfactorily, insofar as they do not take into account either image as a whole, or the details in the image.
- the similarity measure With a view to grouping images extracted from a video, the similarity measure must be able to reflect the fact that several images correspond to the same scene, characterized by a unit of place. Information characterizing only the details of the image is not sufficient to fulfill this task. On the other hand, information characterizing the image only in its entirety may prove to be unsuitable in the case of partial modification of the scene decoration, for example by modification of the background of the scene.
- the invention aims to overcome the aforementioned drawbacks. It relates to a method of measuring similarity between images, characterized in that it performs, for each image, the following steps:
- the method is characterized in that it also calculates a histogram corresponding to the distribution of the segments around the center of gravity of each class. According to a particular embodiment, to calculate this histogram, it performs a calculation of the standard deviation of the distances from the mediums of the segments of a class to the center of gravity of the class considered.
- the comparison of the histograms consists in a subtraction of the ordinates, class by class and in a sum, over all the classes, of the values obtained for each class.
- the histograms are coded according to the MPEG-7 standard.
- the invention also relates to a method for clustering images of a database, characterized in that the grouping is carried out according to similarity measures according to the method described above to give groups (clusters) of images.
- the invention also relates to a method for creating video summaries, characterized in that it selects at least one of the images of at least one group calculated according to the preceding method.
- the invention also relates to a video indexing method, characterized in that it selects at least one of the images of at least one group calculated according to the preceding method, as the indexing image.
- the invention also relates to a device for measuring similarity between images, characterized in that it comprises a circuit for processing and calculating histograms receiving digital data defining these images to perform, for each of them, the following operations :
- the main advantage of the invention is that it implements high-performance algorithms for characterizing an image and measuring the similarity between images from these algorithms, thanks to the joint use of attributes based on contour orientation making it possible to characterize the number of segments. , their size as well as their distribution according to the orientation.
- the global description of the image is obtained from the histograms of the orientations of the segments in the image, therefore without taking account of the position of these segments in the image, and the local description is obtained from a measurement of centers of gravity of segment classes, which corresponds to the relative positions of the different segments of the same class.
- the method performs segmentation of the image.
- Object does not exist here, however. This involves determining segments in the image which are approximations of the actual contours in the image, for example from gradient measurements, regardless of whether or not they belong to a particular object in the image. 'picture.
- FIG. 1 represents a flow chart defining the main steps of the method for calculating attribute histograms characterizing an image.
- a first phase which is a preprocessing of the image consists in detecting contours in the image to obtain a contour map and in segmenting these contours to obtain a segment map.
- the second phase performs an attribute calculation for the detection of similarity.
- An image to be processed is transmitted at a first step referenced 1 on the flow diagram of FIG. 1.
- This step of the method performs a calculation of gradients in the image.
- a gradient detection filter is applied to the image to give a gradient measurement at each point.
- a vertical gradient map and a horizontal gradient map are thus calculated.
- the gradient norm obtained from these maps, the square root of the sum of the squares of the horizontal and vertical gradient values assigned to a pixel, is also used.
- Step 2 performs a pixel selection from the gradient values calculated in the previous step, compared with thresholds. This selection is refined by that of the points of greatest contrast in a given neighborhood window.
- the goal is to remove thick contours by selecting, in the vicinity of a pixel, the pixels having the strongest gradients, by considering the values of the horizontal and vertical gradients. It is also a question of favoring the neighboring pixels of a pixel already selected as an outline. The isolated contour points are eliminated.
- a binary map of contours each pixel of the image is labeled as contour or not.
- a connectivity test is carried out on the binary contour map in order to gather the neighboring pixels, thereby forming contour lines. To do this, a labeling process of related components is implemented in the next step 3.
- Two pixels are declared related if there is an unbroken path of contour pixels connecting these two pixels. Pixel contours are thus "chain” so as to obtain a line of continuity. Related pixels are labeled, with each pixel touching another pixel labeled with the same label. We obtain, for each label, a list of related components, this list being the coordinates of the different pixels of a line of continuity relating to a label.
- the next step 4 of the preprocessing process is a polygonal approximation of these chained contours in order to approach each contour line by a set of connected segments.
- FIG. 2 represents a method of polygonal approximation of a chained contour.
- the point e3 is sought, giving the maximum distance between the points of the contour and the segment formed by the ends of this contour e1 and e2. If this distance is greater than a threshold, the contour is approached by two segments [e1, e3] and [e3, e2]. The process is repeated until the distances from the point to the segment are less than the fixed threshold.
- the second phase consists of a calculation and an exploitation of the attributes relating to this segment map.
- Step 5 calculates, for each segment of the segment map, the angle between its direction and the horizontal.
- the angle obtained between 0 and 180 degrees, is then quantified to obtain a reduced number of categories or classes of angles.
- the angles obtained are listed in 36 classes, the quantization step being 5 degrees or, in other words, one class corresponds to a width of 5 degrees.
- Histograms relating to three attributes are now calculated in step 6, using the segment orientation information obtained previously. - a histogram of the number of segments according to the orientation.
- the different classes are represented on the abscissa and the occurrences on the ordinate.
- the ordinates correspond to the sum of the lengths of the segments of the class, for a given class.
- the center of gravity G, of class Q is obtained by calculating the barycenter of the media M (s of the segments Sj.
- the ordinates H (i) correspond to the standard deviation of the Euclidean distances d (G i M (s j )) between the barycenter or center of gravity of class C, considered and the center of each segment of the class:
- card is the cardinal function returning the number of segments of the class considered.
- the distribution is put in the form of a histogram, that is to say of vectors of values, thus allowing its exploitation within the framework of the MPEG7 standard which defines the coding of such histograms.
- the histograms are normalized, for example from the maximum values found.
- the three histograms of the attributes characterizing the image are used for the similarity measurements, step 7.
- Three histograms are calculated for a first image which is for example a request image and three other histograms for a second image which is an image in a database of data.
- the measurement of similarity between the two images Ii and, that is to say the calculation of the distance between these images can then be carried out by simple subtraction of the ordinates, class by class, for each type of histogram Jtf of size
- An indicator of similarity SCT ⁇ / j is for example the sum of the values obtained by these subtractions, for all the histograms:
- the similarity measure can be improved by comparing the class also to classes close to the histogram of the other image, with a weighting factor.
- the comparison of histograms uses for example a metric of cross quadratic type making it possible to get rid of the small variations of overall orientation between two successive images, for example during a weak rotation of the camera.
- FIG. 3 An example of searching for similar images is given in FIG. 3, showing, at the top left, the image considered, its associated contour and segment map, as well as the images recognized as most similar in a base of 150 images from two different sequences.
- the main characteristic of these histograms is therefore to provide both a global and local description of the images. It is thus possible, thanks to the global description, to differentiate globally structured images, for example images of cities characterized by orientations of horizontal and vertical segments, of countryside images characterized by orientations of more random segments. But it is also possible, thanks to the local description, to differentiate locally structured images, a part of the image is for example devoted to a building, another to a wood, of less structured images, for example a house in middle of the forest.
- the orientations of the exploited segments can be the angles formed with other reference lines than the horizontal.
- the dimensions of the classes can be less than or greater than 5 degrees, depending on the available computing power or time or the desired measurement quality.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Library & Information Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
- Collating Specific Patterns (AREA)
Abstract
Description
Claims
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
FR0211950 | 2002-09-27 | ||
FR0211950A FR2845186B1 (fr) | 2002-09-27 | 2002-09-27 | Procede et dispositif de mesure de similarite entre images |
PCT/FR2003/050053 WO2004029833A2 (fr) | 2002-09-27 | 2003-09-12 | Procede et dispositif de mesure de similarite entre images |
Publications (1)
Publication Number | Publication Date |
---|---|
EP1543444A2 true EP1543444A2 (fr) | 2005-06-22 |
Family
ID=31985273
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP03780266A Withdrawn EP1543444A2 (fr) | 2002-09-27 | 2003-09-12 | Procede et dispositif de mesure de similarite entre images |
Country Status (5)
Country | Link |
---|---|
US (1) | US7203358B2 (fr) |
EP (1) | EP1543444A2 (fr) |
AU (1) | AU2003288359A1 (fr) |
FR (1) | FR2845186B1 (fr) |
WO (1) | WO2004029833A2 (fr) |
Families Citing this family (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
FR2885719B1 (fr) * | 2005-05-10 | 2007-12-07 | Thomson Licensing Sa | Procede et dispositif de suivi d'objets dans une sequence d'images |
AT504213B1 (de) * | 2006-09-22 | 2008-04-15 | Ipac Improve Process Analytics | Verfahren zum ähnlichkeitsvergleich von gegenständen |
FR2907239B1 (fr) * | 2006-10-11 | 2009-01-09 | Spikenet Technology | Procede de recherche et de reconnaissance rapides d'une image numerique representative d'au moins un motif graphique dans une banque d'images numeriques |
JP2008269471A (ja) * | 2007-04-24 | 2008-11-06 | Sony Corp | 類似画像判定装置と類似画像判定方法およびプログラムと記録媒体 |
GB2454213A (en) * | 2007-10-31 | 2009-05-06 | Sony Corp | Analyzing a Plurality of Stored Images to Allow Searching |
US8396325B1 (en) | 2009-04-27 | 2013-03-12 | Google Inc. | Image enhancement through discrete patch optimization |
US8611695B1 (en) * | 2009-04-27 | 2013-12-17 | Google Inc. | Large scale patch search |
US8391634B1 (en) * | 2009-04-28 | 2013-03-05 | Google Inc. | Illumination estimation for images |
US8385662B1 (en) | 2009-04-30 | 2013-02-26 | Google Inc. | Principal component analysis based seed generation for clustering analysis |
US8798393B2 (en) | 2010-12-01 | 2014-08-05 | Google Inc. | Removing illumination variation from images |
US8938119B1 (en) | 2012-05-01 | 2015-01-20 | Google Inc. | Facade illumination removal |
CN107223344A (zh) * | 2017-01-24 | 2017-09-29 | 深圳大学 | 一种静态视频摘要的生成方法及装置 |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5592572A (en) * | 1993-11-05 | 1997-01-07 | The United States Of America As Represented By The Department Of Health And Human Services | Automated portrait/landscape mode detection on a binary image |
US5668891A (en) * | 1995-01-06 | 1997-09-16 | Xerox Corporation | Methods for determining font attributes of characters |
US8107015B1 (en) * | 1996-06-07 | 2012-01-31 | Virage, Incorporated | Key frame selection |
US6108444A (en) * | 1997-09-29 | 2000-08-22 | Xerox Corporation | Method of grouping handwritten word segments in handwritten document images |
US6130706A (en) * | 1998-03-25 | 2000-10-10 | Lucent Technologies Inc. | Process for determining vehicle dynamics |
US6674915B1 (en) * | 1999-10-07 | 2004-01-06 | Sony Corporation | Descriptors adjustment when using steerable pyramid to extract features for content based search |
-
2002
- 2002-09-27 FR FR0211950A patent/FR2845186B1/fr not_active Expired - Fee Related
-
2003
- 2003-09-12 AU AU2003288359A patent/AU2003288359A1/en not_active Abandoned
- 2003-09-12 US US10/528,729 patent/US7203358B2/en not_active Expired - Lifetime
- 2003-09-12 EP EP03780266A patent/EP1543444A2/fr not_active Withdrawn
- 2003-09-12 WO PCT/FR2003/050053 patent/WO2004029833A2/fr not_active Application Discontinuation
Non-Patent Citations (1)
Title |
---|
SOO-JUN PARK (ETRI ET AL: "Core Experiments on MPEG-7 edge histogram descriptors", 52. MPEG MEETING; 31-05-2000 - 02-06-2000; GENÃVE; (MOTION PICTUREEXPERT GROUP OR ISO/IEC JTC1/SC29/WG11),, no. M5984, 26 May 2000 (2000-05-26), XP030035158, ISSN: 0000-0293 * |
Also Published As
Publication number | Publication date |
---|---|
AU2003288359A1 (en) | 2004-04-19 |
AU2003288359A8 (en) | 2004-04-19 |
WO2004029833A3 (fr) | 2004-07-15 |
US7203358B2 (en) | 2007-04-10 |
FR2845186A1 (fr) | 2004-04-02 |
FR2845186B1 (fr) | 2004-11-05 |
WO2004029833A2 (fr) | 2004-04-08 |
US20060023944A1 (en) | 2006-02-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7715621B2 (en) | Method and apparatus for representing a group of images | |
JP5711387B2 (ja) | 映像を比較する方法および装置 | |
CN102176208B (zh) | 基于三维空时特征的鲁棒视频指纹方法 | |
CN106250431B (zh) | 一种基于分类服装的颜色特征提取方法及服装检索系统 | |
EP2316082A1 (fr) | Procede d'identification d'un objet dans une archive video | |
CN102609548A (zh) | 一种基于运动目标的视频内容检索方法及系统 | |
EP1543444A2 (fr) | Procede et dispositif de mesure de similarite entre images | |
EP2930659B1 (fr) | Procédé de détection de points d'intérêt dans une image numérique | |
Upcroft et al. | Lighting invariant urban street classification | |
CN111182364B (zh) | 一种短视频版权检测方法及系统 | |
CN113886632B (zh) | 一种基于动态规划的视频检索匹配方法 | |
CN104299234B (zh) | 视频数据中雨场去除的方法和系统 | |
CN114743139A (zh) | 视频场景检索方法、装置、电子设备及可读存储介质 | |
CN104410867A (zh) | 改进的视频镜头检测方法 | |
CN117292176A (zh) | 一种架空输电线路关键部件及缺陷的检测方法 | |
CN110830734B (zh) | 一种突变和渐变镜头切换识别方法及系统 | |
Ceroni et al. | Mining exoticism from visual content with fusion-based deep neural networks | |
FR2774493A1 (fr) | Procede de segmentation spatiale d'une image en objets visuels et application | |
Asghar et al. | Pakvehicle-reid: a multi-perspective benchmark for vehicle re-identification in unconstrained urban road environment | |
Chu et al. | Travel video scene detection by search | |
CN117874289B (zh) | 一种摄像头回放查找的方法、缺陷检测装置和存储介质 | |
Jiang et al. | Improving the anti-occlusion ability of correlation filter-based trackers via segmentation | |
Suárez et al. | Image patch similarity through a meta-learning metric based approach | |
Venkatesan et al. | Supervised and unsupervised learning approaches for tracking moving vehicles | |
Tapu et al. | Multiresolution median filtering based video temporal segmentation |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20050317 |
|
AK | Designated contracting states |
Kind code of ref document: A2 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PT RO SE SI SK TR |
|
AX | Request for extension of the european patent |
Extension state: AL LT LV MK |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: THOMSON LICENSING |
|
DAX | Request for extension of the european patent (deleted) | ||
RBV | Designated contracting states (corrected) |
Designated state(s): DE FR GB |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: OISEL, LIONEL Inventor name: LE CLERC, FRANCOIS Inventor name: CHUPEAU, BERTRAND |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: THOMSON LICENSING |
|
17Q | First examination report despatched |
Effective date: 20140313 |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: THOMSON LICENSING DTV |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTG | Intention to grant announced |
Effective date: 20180705 |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: INTERDIGITAL MADISON PATENT HOLDINGS |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN |
|
18D | Application deemed to be withdrawn |
Effective date: 20181116 |