WO2010146495A1 - Procédé et appareil de sélection d'une image représentative - Google Patents

Procédé et appareil de sélection d'une image représentative Download PDF

Info

Publication number
WO2010146495A1
WO2010146495A1 PCT/IB2010/052534 IB2010052534W WO2010146495A1 WO 2010146495 A1 WO2010146495 A1 WO 2010146495A1 IB 2010052534 W IB2010052534 W IB 2010052534W WO 2010146495 A1 WO2010146495 A1 WO 2010146495A1
Authority
WO
WIPO (PCT)
Prior art keywords
images
selecting
cluster
image
clustering
Prior art date
Application number
PCT/IB2010/052534
Other languages
English (en)
Inventor
Marc Andre Peters
Pedro Fonseca
Original Assignee
Koninklijke Philips Electronics N.V.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips Electronics N.V. filed Critical Koninklijke Philips Electronics N.V.
Priority to JP2012514579A priority Critical patent/JP2012530287A/ja
Priority to CN2010800266823A priority patent/CN102460433A/zh
Priority to EP10728337A priority patent/EP2443569A1/fr
Priority to RU2012101280/08A priority patent/RU2012101280A/ru
Priority to US13/377,841 priority patent/US20120082378A1/en
Publication of WO2010146495A1 publication Critical patent/WO2010146495A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually

Definitions

  • the present invention relates to a method and apparatus for selecting at least one representative image from a plurality of images.
  • the present invention seeks to provide a technique for obtaining from amongst a vast number of images a representative image of a group of images.
  • a method of selecting at least one representative image from the plurality of images comprising the steps of: dividing a plurality of images into clusters according to a predetermined characteristic of the content of the plurality of images; selecting at least one of the clusters based on the number of images in each of the clusters; and selecting at least one image from the selected at least one cluster as the representative image.
  • apparatus for selecting at least one representative image from the plurality of images comprising: a divider for dividing a plurality of images into clusters according to a predetermined characteristic of the content of the plurality of images; a selector for selecting at least one of the clusters based on the number of images in each of the clusters and for selecting at least one image from the selected at least one cluster as the representative image.
  • images are divided into clusters. This may be achieved according to similarity, time, event or even a folder where they are located.
  • a cluster is selected and at least one image is selected from the selected cluster. This may be a single image or a set of images which best represents the entire group of images.
  • the step of selecting at least one cluster comprises the step of: selecting the cluster having the largest number of images. The idea is that the more important a certain element in a group of images is
  • the more images of that element will exist in the collection.
  • the more images there are of a specific object the easier it will be for the user to recognize it and associate it with a specific event, time period or group of images. This enables the representative image to be selected from the cluster which is most likely to contain the most important objects and therefore to best represent the plurality of images.
  • a cluster may further be selected by selecting the cluster having the least amount of variation in the predetermined characteristic. This assures that the images in the selected cluster are even more alike than in the other clusters.
  • the step of selecting at least one image from the selected at least one cluster as a representative image comprises the step of: selecting the image closest to a centroid of the selected at least one cluster.
  • This representative image is therefore selected as the image closest to the centroid of the cluster which is a representation (in terms of features) of, for example, the average of the images within the cluster.
  • This provides a representative image having strong association for the user with the specific cluster.
  • the image may be randomly selected.
  • the plurality of images may be divided into clusters by clustering images having similar characteristics, for example, visually similar such that the clusters contained related or images having similar content.
  • the plurality of images may be divided into clusters by clustering the images captured at a time within a predetermined time interval.
  • the images can be divided into a cluster of images captured on a certain day or within a vacation period.
  • the images may be clustered such that the time difference between the consecutive images within a cluster is no more than a certain relatively small threshold (e.g. 2 up to 10 minutes).
  • a certain relatively small threshold e.g. 2 up to 10 minutes.
  • clustering images that are visually similar may be preceded by the step of: clustering images captured at time within a predetermined time interval; and the step of clustering images that are visually similar comprises the step of: clustering images of the cluster of images captured at time within a predetermined time interval that are visually similar.
  • time information as a first clustering step prevents images that are semantically unrelated but visually very similar being clustered together. For example, using visual clustering only, two images of the sea captured during two different holiday trips may be clustered together.
  • the images may be clustered by extracting at least one feature from each of said plurality of images; determining the distance between at least one extracted feature of each of the plurality of images; and clustering images having a distance below a predetermined threshold.
  • the at least one feature may comprise one of luminance; colour information; colour distribution features; texture features.
  • the step of selecting at least one image from the selected at least one cluster as a representative image may comprise the steps of: determining the presence of at least one face within each of said images of said selected at least one cluster; determining the ratio of the number of images which contain at least one face to the number of images that contain no face; and selecting an image having a face if said ratio is greater than or equal to 1 or selecting an image without a face if said ratio is less than to 1.
  • the presence of a person, i.e. a face, within an image can provide a good basis for selecting a representative image. If most of the images in the cluster do not contain faces, the most representative image should preferably also not contain faces. Likewise, if most of the images in the cluster do contain faces, the most representative image should preferably also contain a face. As a result face detection can help identify the image or images that best represent the plurality of images.
  • Figure 1 is a simplified schematic of apparatus for selecting an image according to an embodiment of the present invention.
  • Figure 2 is a flowchart of a method of selecting an image according to an embodiment of the present invention.
  • the apparatus 100 comprises an input terminal 101 connected to a storage means 103.
  • the storage means 103 is illustrated here as external to the apparatus 100, in an alternative embodiment, the storage means 103 may be integral with the apparatus.
  • the storage means 103 may be a memory device of a computer system, such as a ROM/RAM drive, CD, a memory device of a camera or like device connected to the apparatus 100, or remote server. It may be accessed via a wired or wireless connection and/or accessed via a wider network such as the Internet.
  • the storage means 103 stores a plurality of images. Images stored on a remote server, for example, may be uploaded and temporarily stored in a local storage means (not shown here) of the apparatus 100.
  • the input terminal 101 of the apparatus 100 is connected to the input of a divider 105 of the apparatus 100.
  • the output of the divider 105 is connected to the input of a selector 107 of the apparatus 100.
  • the output of the selector 107 is connected to an output terminal 109 of the apparatus 100.
  • the output terminal 109 is connected to a display device 111 or the like. Operation of the apparatus will now be described with reference to Figure 2.
  • a plurality of images are retrieved from the storage means 103 and are provide to the divider 105 via the input terminal 101 of the apparatus 100.
  • the plurality of images are divided into a plurality of clusters based upon a predetermined characteristic, step 201.
  • the images may be divided into clusters based on time the images were captured, metadata associated with an image or, alternatively, their visual properties. Further, metadata such as GPS data, or high level features such as recognition of faces or objects may be used as a basis to cluster images.
  • the captured images are analyzed using known content analysis algorithms.
  • this may be achieved by extracting low-level features, such as luminance; colour information like hue and MPEG 7 dominant colour; colour distribution features like MPEG 7 colour layout and colour structure; and texture features like edges.
  • the distance between each extracted feature is determined.
  • the degree of similarity between the images is the determined distance. Therefore, images are clustered having a determined distance which is less than a predetermined threshold, resulting in clusters of images that are visually very similar. This may be achieved by comparing the distance of one feature or a combination of features in clustering the plurality of images.
  • the features may be combined by a simple summation and the elements of the summation may be weighted.
  • These clusters are provide to the selector 107 and at least one cluster is selected, step 203, based upon the number of images in a cluster.
  • the cluster having the largest number of images is selected. This cluster will have the largest amount of similar images and as such is more likely to contain an important or popular object/scene.
  • the cluster having the least amount of (visual) variation within the cluster is selected. This assures that the images in the selected cluster are even more alike than in the other clusters.
  • the selector 107 selects at least one image from the selected cluster that best represents the images of the plurality of the images (the entire group of images), step 205. In an embodiment, the image which best represents the entire group of images is selected as the image closest to the centroid.
  • the centroid is a virtual representation, in terms of features, of the average of the cluster.
  • the image which best represents the entire group of images may be selected on the basis of a particular desired feature, for example, quality of the image such as sharpness/blur contrast or, the presence of a face in which eyes are open or the person is smiling etc.
  • the plurality of images may be clustered in step 201, by making use of Exchangeable Image File (EXIF) date information if available.
  • EXIF Exchangeable Image File
  • the images are grouped based on the time the images were captured. For example, a group of images can be created such that the time difference between the consecutive images is no more than a certain relatively small threshold (e.g. 2 up to 10 minutes) i.e. images captured within a predetermined time interval. Such images are captured around the same time and are likely to be images of the same object, scene or event.
  • a certain relatively small threshold e.g. 2 up to 10 minutes
  • This clustering may be achieved with a higher threshold than normally, i.e., each individual cluster can allow for more visual variability, since the time information already assures that the images are related. In this way the visual clustering algorithm uses the previous cluster (based on time) as input rather than all the separate images enabling the visual clustering algorithm to operate faster and more efficiently.
  • time information as a first clustering step prevents images that are semantically unrelated but visually very similar being clustered together. For example, using visual clustering only, two images of the sea captured during two different holiday trips may be clustered together.
  • the most representative image or images may be selected on the basis of whether or not the images contain a face. If most of the images in the cluster do not contain faces, the most representative image(s) should preferably also not contain faces. Likewise, if most of the images in the cluster do contain faces, the most representative image(s) should preferably also contain a face. For example if one has a trip with many sceneries (landscapes, cityscapes, etc), but one evening the user captures many images of his/her child doing something funny, the largest cluster is likely to be the one with the child. However, the user probably identifies the set of images much more with the location and scenery, and a representative image selected from the scenery would therefore be more appropriate. On the other hand, if the set is for example images captured at a birthday party, an image of the celebrating person(s) would most likely be a correct representative image for the event. Face detection can thus help identify the image or images that best represent the entire group of images.
  • the selected representative image can then be used for browsing a large collection of images, for example, a timeline can be used to represent a collection of thousands of images captured over the years. If a given time period is represented by a selected image that best represented the time period (according the embodiments above), browsing the whole collection can be as simple as browsing the representative images. If a user wants to see more of a specific time period, the interval can be split into smaller intervals with again selecting a representative image for each interval.
  • Using (EXIF) date information and clustering the image as described above enables the user to automatically detect where there are image capturing "peaks" in a collection, i.e., points in time where a user captured relatively many images. These peaks typically correspond to special events, like holidays, or birthdays or a day at the zoo. Where a timeline would, ordinarily take all images into account, using only the peaks the collection is summarized to the events that took place over the years. With an image or images that are representative for each event, providing an ideal summary of a collection. One can select all events, or for example only peaks that span multiple days. In the first case one day events are included, like birthdays and daytrips, while in the latter case only multiple days' events are displayed, like holidays.
  • the same method can also be used to select a given amount of images to represent the group. Rather than taking only one image from the largest cluster, one can take one image per cluster for the n largest clusters where n is the desired number of representatives.
  • 'Means' as will be apparent to a person skilled in the art, are meant to include any hardware (such as separate or integrated circuits or electronic elements) or software (such as programs or parts of programs) which reproduce in operation or are designed to reproduce a specified function, be it solely or in conjunction with other functions, be it in isolation or in co-operation with other elements.
  • the invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the apparatus claim enumerating several means, several of these means can be embodied by one and the same item of hardware.
  • 'Computer program product' is to be understood to mean any software product stored on a computer-readable medium, such as a floppy disk, downloadable via a network, such as the Internet, or marketable in any other manner.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Library & Information Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Image Analysis (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Processing Or Creating Images (AREA)

Abstract

L'invention porte sur un procédé de sélection d'au moins une image représentative à partir d'une pluralité d'images, le procédé comprenant les étapes consistant à : diviser (201) la pluralité d'images en groupes selon une caractéristique prédéterminée du contenu de la pluralité d'images, sélectionner (203) au moins l'un des groupes sur la base du nombre d'images dans chacun des groupes, et sélectionner (205) au moins une image à partir de l'au moins un groupe sélectionné en tant qu'image représentative.
PCT/IB2010/052534 2009-06-15 2010-06-08 Procédé et appareil de sélection d'une image représentative WO2010146495A1 (fr)

Priority Applications (5)

Application Number Priority Date Filing Date Title
JP2012514579A JP2012530287A (ja) 2009-06-15 2010-06-08 代表的な画像を選択するための方法及び装置
CN2010800266823A CN102460433A (zh) 2009-06-15 2010-06-08 选择代表图像的方法和设备
EP10728337A EP2443569A1 (fr) 2009-06-15 2010-06-08 Procédé et appareil de sélection d'une image représentative
RU2012101280/08A RU2012101280A (ru) 2009-06-15 2010-06-08 Способ и устройство для выбора типичного изображения
US13/377,841 US20120082378A1 (en) 2009-06-15 2010-06-08 method and apparatus for selecting a representative image

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP09162685 2009-06-15
EP09162685.3 2009-06-15

Publications (1)

Publication Number Publication Date
WO2010146495A1 true WO2010146495A1 (fr) 2010-12-23

Family

ID=42335256

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2010/052534 WO2010146495A1 (fr) 2009-06-15 2010-06-08 Procédé et appareil de sélection d'une image représentative

Country Status (6)

Country Link
US (1) US20120082378A1 (fr)
EP (1) EP2443569A1 (fr)
JP (1) JP2012530287A (fr)
CN (1) CN102460433A (fr)
RU (1) RU2012101280A (fr)
WO (1) WO2010146495A1 (fr)

Families Citing this family (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8639028B2 (en) * 2006-03-30 2014-01-28 Adobe Systems Incorporated Automatic stacking based on time proximity and visual similarity
US8724910B1 (en) * 2010-08-31 2014-05-13 Google Inc. Selection of representative images
US20120213404A1 (en) 2011-02-18 2012-08-23 Google Inc. Automatic event recognition and cross-user photo clustering
US8914483B1 (en) 2011-03-17 2014-12-16 Google Inc. System and method for event management and information sharing
US8891883B2 (en) 2012-05-15 2014-11-18 Google Inc. Summarizing a photo album in a social network system
US9391792B2 (en) 2012-06-27 2016-07-12 Google Inc. System and method for event content stream
US9418370B2 (en) 2012-10-23 2016-08-16 Google Inc. Obtaining event reviews
US9311310B2 (en) * 2012-10-26 2016-04-12 Google Inc. System and method for grouping related photographs
US8897556B2 (en) 2012-12-17 2014-11-25 Adobe Systems Incorporated Photo chapters organization
US8983150B2 (en) 2012-12-17 2015-03-17 Adobe Systems Incorporated Photo importance determination
JP6280382B2 (ja) * 2013-03-08 2018-02-14 キヤノン株式会社 画像処理装置および画像処理方法
US9070048B2 (en) * 2013-10-17 2015-06-30 Adobe Systems Incorporated Method and apparatus for automatically identifying a representative image for an image group
CN106462568A (zh) 2014-02-13 2017-02-22 河谷控股Ip有限责任公司 全局视觉词汇、系统和方法
US9858295B2 (en) 2014-06-24 2018-01-02 Google Llc Ranking and selecting images for display from a set of images
US9721186B2 (en) 2015-03-05 2017-08-01 Nant Holdings Ip, Llc Global signatures for large-scale image recognition
CN105138962A (zh) * 2015-07-28 2015-12-09 小米科技有限责任公司 图像显示方法及装置
WO2017058826A1 (fr) 2015-09-28 2017-04-06 Google Inc. Partage d'images et d'albums d'images sur un réseau de communication
CN105404863B (zh) * 2015-11-13 2018-11-02 小米科技有限责任公司 人物特征识别方法及系统
CN107016004A (zh) * 2016-01-28 2017-08-04 百度在线网络技术（北京）有限公司 图像处理方法及装置
US11048744B1 (en) * 2016-12-29 2021-06-29 Shutterstock, Inc. Computer architecture for weighting search results by stylistic preferences
WO2018212815A1 (fr) 2017-05-17 2018-11-22 Google Llc Partage d'image automatique avec des utilisateurs désignés sur un réseau de communication
CN110741334B (zh) * 2017-06-19 2024-03-05 索尼公司 显示控制设备、显示控制方法和显示控制程序
KR102035531B1 (ko) 2017-09-26 2019-10-24 네이버웹툰 주식회사 대표 이미지 생성
CN110290426B (zh) * 2019-06-24 2022-04-19 腾讯科技（深圳）有限公司 展示资源的方法、装置、设备及存储介质
CN110403582B (zh) * 2019-07-23 2021-12-03 宏人仁医医疗器械设备(东莞)有限公司 一种用于分析脉波波形品质的方法
US11656881B2 (en) * 2021-10-21 2023-05-23 Abbyy Development Inc. Detecting repetitive patterns of user interface actions

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030009469A1 (en) * 2001-03-09 2003-01-09 Microsoft Corporation Managing media objects in a database
WO2003038680A2 (fr) * 2001-10-31 2003-05-08 Hewlett-Packard Company Procede et systeme d'acces a un ensemble d'images dans une base de donnees
WO2006096384A1 (fr) 2005-03-04 2006-09-14 Eastman Kodak Company Groupage additif d'images depourvues d'informations temporelles

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10162020A (ja) * 1996-12-03 1998-06-19 Ricoh Co Ltd 画像データベースのブラウジング方法
US6393427B1 (en) * 1999-03-22 2002-05-21 Nec Usa, Inc. Personalized navigation trees
JP4418400B2 (ja) * 2005-05-20 2010-02-17 オリンパスメディカルシステムズ株式会社 画像表示装置
JP2007094990A (ja) * 2005-09-30 2007-04-12 Fujifilm Corp 画像分類装置および方法並びにプログラム
JP2007188427A (ja) * 2006-01-16 2007-07-26 Nippon Telegr & Teleph Corp <Ntt> 話題画像選出方法及び装置及プログラム
US7869658B2 (en) * 2006-10-06 2011-01-11 Eastman Kodak Company Representative image selection based on hierarchical clustering
JP4375442B2 (ja) * 2007-06-04 2009-12-02 ソニー株式会社 画像管理装置、画像管理方法および画像管理プログラム

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030009469A1 (en) * 2001-03-09 2003-01-09 Microsoft Corporation Managing media objects in a database
WO2003038680A2 (fr) * 2001-10-31 2003-05-08 Hewlett-Packard Company Procede et systeme d'acces a un ensemble d'images dans une base de donnees
WO2006096384A1 (fr) 2005-03-04 2006-09-14 Eastman Kodak Company Groupage additif d'images depourvues d'informations temporelles

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
A. GRAHAM ET AL.: "Time as Essence for Photo Browsing Through Personal Digital Libraries", JCDL 2002. PROC. OF THE 2ND ACM/IEEE-CS JOINT CONFERENCE ON DIGITAL LIBRARIES, vol. 2, 14 July 2002 (2002-07-14), pages 326 - 335
GRAHAM A ET AL: "Time as essence for photo browsing through personal digital libraries", JCDL 2002. PROCEEDINGS OF THE SECOND ACM/IEEE-CS JOINT CONFERENCE ON DIGITAL LIBRARIES. PORTLAND, OR, JULY 14 - 18, 2002; [PROCEEDINGS ACM/IEEE-CS JOINT CONFERENCE ON DIGITAL LIBRARIES], NEW YORK, NY : ACM, US LNKD- DOI:10.1145/544220.544301, vol. CONF. 2, 14 July 2002 (2002-07-14), pages 326 - 335, XP002383768, ISBN: 978-1-58113-513-8 *

Also Published As

Publication number Publication date
JP2012530287A (ja) 2012-11-29
US20120082378A1 (en) 2012-04-05
EP2443569A1 (fr) 2012-04-25
RU2012101280A (ru) 2013-07-27
CN102460433A (zh) 2012-05-16

Similar Documents

Publication Publication Date Title
US20120082378A1 (en) method and apparatus for selecting a representative image
US20220004573A1 (en) Method for creating view-based representations from multimedia collections
TWI338265B (en) System, apparatus, method and program for processing image
US8306331B2 (en) Image processing apparatus and method, and program
EP2402867B1 (fr) Procédé implémenté par ordinateur, produit de programme informatique et système informatique pour le traitement d&#39;images
US8594440B2 (en) Automatic creation of a scalable relevance ordered representation of an image collection
JP4337064B2 (ja) 情報処理装置、情報処理方法、およびプログラム
TWI278757B (en) Presenting a collection of media objects
Chen et al. Tiling slideshow
EP2224372A2 (fr) Regroupement d&#39;images par emplacement
JP2012507189A (ja) コンテンツベース・フィルタリング及びテーマベース・クラスタリングを用いたページ内へのイメージの配置
JP4643735B1 (ja) 電子機器及び映像処理方法
KR20100052676A (ko) 컨텐츠 앨범화 장치 및 그 방법
WO2006073299A1 (fr) Procede et appareil permettant de regrouper des photos numeriques en fonction de la situation, et systeme et procede de creation d&#39;albums associes
EP2304617A2 (fr) Procede et appareil permettant de generer une collection d images
CA2753978A1 (fr) Videos groupees en fonction d&#39;un lieu
JP2006203574A (ja) 画像表示装置
EP2033139A1 (fr) Utilisation de l&#39;arrière-plan pour une recherche dans des collections d&#39;images
CN102177703A (zh) 用于生成要被显示而同时伴随有音频的多个图像的序列的方法和设备
US20140029854A1 (en) Metadata supersets for matching images
JP5878523B2 (ja) コンテンツ加工装置とその集積回路、方法、およびプログラム
US20110137964A1 (en) File System Manager Using Tagging Organization
KR100790865B1 (ko) 상황기반 디지털 사진 클러스터링 방법 및 장치와 이를이용한 디지털 사진 앨범 시스템 및 방법
JP2009217828A (ja) 画像検索装置
JP2006079460A (ja) 電子アルバム表示システム、電子アルバム表示方法、電子アルバム表示プログラム、画像分類装置、画像分類方法、及び画像分類プログラム

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 201080026682.3

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 10728337

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2010728337

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2012514579

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 13377841

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 100/CHENP/2012

Country of ref document: IN

WWE Wipo information: entry into national phase

Ref document number: 2012101280

Country of ref document: RU