WO2007130688A2 - Dispositif informatique mobile à capacité d'imagerie - Google Patents

Dispositif informatique mobile à capacité d'imagerie Download PDF

Info

Publication number
WO2007130688A2
WO2007130688A2 PCT/US2007/011120 US2007011120W WO2007130688A2 WO 2007130688 A2 WO2007130688 A2 WO 2007130688A2 US 2007011120 W US2007011120 W US 2007011120W WO 2007130688 A2 WO2007130688 A2 WO 2007130688A2
Authority
WO
WIPO (PCT)
Prior art keywords
computing device
visual features
image
features
item
Prior art date
Application number
PCT/US2007/011120
Other languages
English (en)
Other versions
WO2007130688A3 (fr
Inventor
Enrico Di Bernardo
Mario Munich
Paolo Pirjanian
William Gross
Original Assignee
Evolution Robotics, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Evolution Robotics, Inc. filed Critical Evolution Robotics, Inc.
Publication of WO2007130688A2 publication Critical patent/WO2007130688A2/fr
Publication of WO2007130688A3 publication Critical patent/WO2007130688A3/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/77—Determining position or orientation of objects or cameras using statistical methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/94—Hardware or software architectures specially adapted for image or video understanding
    • G06V10/95—Hardware or software architectures specially adapted for image or video understanding structured as a network, e.g. client-server architectures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30204—Marker
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]

Definitions

  • the invention generally relates to a mobile computing device configured to generate and transmit visual features of images.
  • the invention relates to a mobile phone or other communication device configured to capture images, extract visual features from those images, and transmit those visual features to a server where objects depicted in the image are recognized.
  • a significant percentage of mobile phones are configured with pinhole cameras with which the user can capture digital images and video.
  • the resolution and quality of the images is generally sufficient to recognize objects depicted in those images using various techniques known to those skilled in the art.
  • the process of object recognition is, however, performed by a remote server instead of the phone due to processing and memory limitations.
  • the images — or a portion of the image — are captured by the phone and transmitted to a remote server for identification based on, for example, the presence of a universal product code (UPC) shown in the image.
  • UPC universal product code
  • This system requires that image data be transmitted by the phone over a cellular network. Transmission of the raw image data can be especially burdensome where the image contains several mega pixels.
  • the process is susceptible to signal noise, transmission errors, and disruption if the connection between the phone and the base station is dropped.
  • privacy laws in some countries prohibit image from being relayed to a server without the user explicit authorization.
  • the phone is configured to capture an image and decode a universal product code (UPC) depicted in the image.
  • UPC universal product code
  • the decoded number associated with the UPC may then be transmitted back to the server for processing. While this approach is useful in limited situations in which the object has computer- readable indicia, such as system is unable to actually recognize the object based on its visual appearance more generally.
  • the invention features a system and method for using a mobile computing device in combination with a distributed network to perform object recognition and initiate responsive action.
  • the invention is a method of performing object recognition with a camera phone in a cellular communications network, the method comprising: using the mobile computing device to extract one or more visual features from an image captured by the mobile computing device; transmitting the extracted features from the mobile computing device to a remote server; using the server to compare the one or more visual features with a database of visual features representing known objects; and identifying an item in the image as one of the known objects based on a match between the extracted features and the visual features of the plurality of known objects.
  • the mobile computing device is a mobile phone although various other devices may be suitable including digital cameras, personal computers, personal digital assistant (PDAs), digital cameras with wireless network capability, portable media players, and global positioning systems (GPSs), for example.
  • the mobile device is configured to transmit the visual features to the object server recognition server via a cellular communications network, although a packet- switched network such as a local area network (LAN), metropolitan area network (MAN), and/or the Internet may also being employed.
  • the visual features extracted from the image are preferably scale-invariant features.
  • Suitable feature detector/descriptor types include, but are not limited to, scale-invariant feature transform (SIFT) features and Speeded Up Robust Features (SURF) in which the visual features can be characterized by a vector comprising image gradient data, image scale data, feature orientation data, and geometric location data.
  • SIFT scale-invariant feature transform
  • SURF Speeded Up Robust Features
  • the server may take any of a number of actions.
  • the server is configured to return information to the mobile computing device or take action on behalf of the user of the mobile computing device. If the identified item is a product offered for sale, for example, the server may transmit product information with the price and availability as well as a hyperlink with which the user can purchase the product.
  • the invention also includes a system for performing object recognition with the mobile computing device and object recognition server.
  • the mobile computing device preferably includes a camera adapted to capture a digital photograph or video of an item, a feature extractor adapted to extract one or more visual features from the image, and a transmitter adapted to send the one or more visual features from the mobile computing device to the server.
  • the server preferably includes a database of visual features associated with a plurality of known objects and an object recognition processor configured to identify the item based on the one or more visual features and the visual features associated with the plurality of known objects.
  • the mobile computing device is preferably a mobile phone and the server is configured to return product information to the phone or take other predetermined action.
  • FIG. 1 is a functional block diagram of a distributed network with, in accordance with the preferred embodiment of the present invention
  • FIG. 2 is a functional block diagram of a mobile computing device, an information service provider, and an object recognition server, in accordance with the preferred embodiment of the present invention
  • FIG. 3 is a flowchart of a process of recognizing a product using a mobile computing device in a distributed network, in accordance with the preferred embodiment of the present invention
  • FIG. 4 is a flowchart of the method of extracting scale-invariant visual features, in accordance with the preferred embodiment of the present invention
  • FIG. 5 is a flowchart of the method of implementing object recognition using scale- invariant visual features, in accordance with the preferred embodiment of the present invention.
  • FIG. 1 Illustrated in FIG. 1 is a network for implementing the object recognition system 100 and method of requesting information or otherwise acquiring information about a product or other item using a mobile communication device with minimal user interaction.
  • the network includes a plurality of mobile computing devices with imaging capability, which are diagrammatically represented by users 1 10-112.
  • the mobile computing devices in the preferred embodiment are cellular phones with built- in digital cameras, although various other forms of devices may be employed including personal computers, personal digital assistants (PDAs), digital cameras with wireless network capability, portable media players, global positioning systems (GPS), and the like, as well as some non-portable devices.
  • PDAs personal digital assistants
  • GPS global positioning systems
  • the cellular phones are operatively coupled to a cellular network 103 adapted to provide access to a data communications network such as the Internet 104 via a cellular service provider 102.
  • handsets are enabled with the 802.1 1 protocol (or Wi-Max, Bluetooth, or other means of wireless data communication) to access the Internet 104 or other packet-switched network more directly through an access point 105.
  • the network further includes an information service provider 106 adapted to provide information about products, for example, based on objects that are depicted in digital photographs or snapshots taken by the users 1 10-1 12 and identified by the object recognition server 120.
  • users 1 10-1 12 access the information service provider 106 for purposes of purchasing the product or item, getting additional information on a product or item, or receiving coupons or discounts, for example.
  • the functions described herein are performed by programs including executable code or computer readable instructions running on one or more handsets and/or general-purpose computers.
  • the computers can include one or more central processing units for executing program code, volatile memory, such as random access memory (RAM) for temporarily storing data and data structures during program execution, non-volatile memory, such as a hard disc drive, flash memory, or optical drive, for storing programs and data, including databases, and a network interface for accessing an intranet and/or the Internet.
  • RAM random access memory
  • non-volatile memory such as a hard disc drive, flash memory, or optical drive
  • programs and data including databases
  • a network interface for accessing an intranet and/or the Internet.
  • the functions described herein can also be implemented using special purpose computers, application-specific integrated circuit (ASIC), state machines, and/or hardwired electronic circuits.
  • ASIC application-specific integrated circuit
  • state machines and/or hardwired electronic circuits.
  • the example processes described herein do not necessarily have to be performed in the
  • FIG. 2 Illustrated in FIG. 2 is a functional block diagram of the object recognition system including a mobile computing device 110, information service provider 106, and object recognition server 120.
  • the mobile computing device 1 preferably a cellular phone, includes a user interface 202 with a keypad 203, audio input/output 204, and a liquid crystal display (LCD) for displaying images and corresponding product information; a digital camera 208; and a feature extractor 252 for generating visual features used to identify objects depicted in the images acquired by the camera 208.
  • the user may initiate a search for more product information by snapping a picture of the item.
  • the acquired image is passed to the feature extractor 252 which generates a plurality of visual features that characterize the object(s) depicted therein.
  • the feature extractor 252 in the preferred embodiment generates a plurality of scale-invariant visual features with a scale-invariant feature transform (SIFT) processor 254 discuss in.more detail below.
  • SIFT visual features generally represent the orientation, form, and contrast of lines, edges, corners, textures, and like elements of an image, but do not include computer readable indicia like UPC codes.
  • the visual features are then transmitted to the object recognition server 120 which employs a pattern recognition processor or module 256 to identify the one or more items depicted in the photograph by comparing the extracted visual features with the visual features of known objects retained in the feature descriptor database 272.
  • the one or more identified items are then used to retrieve instructions to initiate a predetermined action or send additional information from the product info database 280 of the information service provider 106 or from a remote part of the network via a uniform resource locator (URL).
  • the additional information is returned to the user where it may be displayed as a product page 206 via the user interface 202.
  • a product page 206 as used herein refers to a combination of text, graphical, and/or hyperlinks that present product information selected by or tailored for the user based on the image captured by the camera.
  • the categories of product information that can be returned to the user 110 include web pages or documents with various forms of content (or hyperlinks to content) including purchasing information, text, graphics, audio, video, and various other forms resources.
  • the purchasing information may further include one or more pictures of the product, the product's prices, quantity, availability, location, physical or functional specifications (dimensions, weight, model, part #, universal product code (UPC) code), pictures, video, product reviews, and/or a URL of a website or other link to such information.
  • the object recognition 120 may also be configured to take a prescribed action associated with the identified objection including, for example, entering the user/sender into a sweepstakes specified in the image, allowing the user to cast a vote or a preference in an election or survey indicated in the image, sending samples of the product depicted to user's residence, or calling the user back to provide more information in person or with a recorded message.
  • the product page 206 may also include a thumbnail picture 21 1 of the identified item 210 to enable the user to verify the matching object identified by the server 120.
  • the object recognition server 120 identifies a plurality of known objects or product pages associated with the pictured item 210
  • the user may be provided a list including a brief description of the each of the objects from which the user may select. If the user snaps a photograph of a can of COCA-COLA (TM). the list returned by the server 120 may include the price and product information for a 6-pack and 12-pack of COCA-COLA (TM).
  • the server may return a product page 21 1 including a menu from which the user may select, the menu enabling the user to select price information, the name and address of stores where the product can be purchased, maps locating the stores, news and reviews about the item, and like information.
  • the product page 206 enables the user to execute an order for the product.
  • the user may order the identified product by clicking on a single "purchase" button once the user has logged in to the vendor's website by providing identifying information including an account name and personal identification number (PIN), for example.
  • PIN personal identification number
  • the order may be automatically charged against a user-designated financial account (e.g., credit card or phone bill) and shipped to a pre-designated address based on information previously entered by the user.
  • the user may identify himself by entering biometric information such as a fingerprint or into the mobile device, thereby allowing the purchase to be automatically charged to the phone service provider bill.
  • Illustrated in FIG. 3 is a flowchart of a process of invoking object recognition and returning product information for a user based on a photograph taken with a mobile computing device, e.g., a camera phone handset.
  • the user snaps a photograph of an object, i.e., the item of interest.
  • the item may be, for example, a product, place, advertisement, or the like.
  • the product may also be displayed in a store or depicted in a catalog or a magazine, for example.
  • the image captured by the handset may be in any of a number of raster formats including bit map, JPEG, or GIF or in video formats including MPEG and AVI.
  • the item depicted in the photograph may include the entire item or a portion of the item sufficient to extract visual features with which to identify the item. Thereafter, the user may select a preprogrammed camera mode that initiates the following operations to solicit more information about the item.
  • the image is transferred to a first process, preferably the phone's SIFT processor, which typically extracts 304 a plurality of visual features used to characterize the object depicted in the photograph.
  • SIFT is able to consistently identify visual features with which objects can be identified independent of differences in image size, image lighting conditions, position and orientation of the features in the images and camera viewpoint.
  • the number and quality of extracted visual features will vary depending on the image, but frequently varies between 50 and 2000 features per VGA resolution (640 x 480) image. The process of extracting visual features is discussed in more detail below in context of FIG. 4.
  • the visual features are transmitted 306 to the information service provider 106 with instructions to provide additional information or take other action regarding the item.
  • the instructions may further specify the type of information requested by the user or the ID of the application which is generating the request.
  • the application running on the device may allow the user to specify what type of information the users is interested in — pricing, reviews, samples, coupons, store names and locations, etc. — in regards to the item photographed.
  • the information service provider 106 forwards the plurality of visual features to a second process or module, preferably the object recognition processor 120.
  • the object recognition processor 120 compares each of the features from the captured image with the features of known objects retained in the feature descriptor database 272. A match is identified if the Euclidian distance between the extracted features and features of known objects satisfies a determined threshold, or the distance is minimal compared to the other known objects. If one or more objects are identified by matching 308 the visual features with those of the database, the identities of the objects are returned to the information service provider 106 and the associated product information retrieved 310. Thereafter, the associated product information is transmitted 312 to the user's handset. As stated, the user can respond 314 to the product information by, for example, purchasing the identified product using an executable instruction included in the product page 206.
  • FIG. 4 Illustrated in FIG. 4 is a flowchart of the method of extracting scale-invariant visual features from a digital photograph in the preferred embodiment.
  • Visual features preferably SIFT-based visual features
  • DoG Difference-of-Gaussian
  • a Difference-of-Gaussian image represents a band-pass filtered image produced by subtracting a first copy of the image blurred with a first Gaussian kernel from a second copy of the image blurred with a second Gaussian kernel. This process is repeated for multiple frequency bands in order to accentuate objects and object features independent of their size and resolution.
  • Each of the DoG images is inspected by the SIFT processor 254 to identify the pixel extrema including minima and maxima. To be selected; an extremum must possess the highest or lowest pixel intensity among the eight adjacent pixels in the same DoG image as well as the nine adjacent pixels in the two adjacent DoG images having the closest related band-pass filtering, i.e., the adjacent DoG images having the next highest scale and the next lowest scale if present.
  • the identified extrema which may be referred to herein as image "keypoints," are associated with the center point of visual features.
  • an improved estimate of the location of each extremum within a DoG image may be determined through interpolation using a 3- dimensional quadratic function, for example, to improve feature matching and stability.
  • the local image properties are used to assign an orientation to each of the keypoints.
  • the orientation is derived from an orientation histogram formed from gradient orientations at all points within a circular window or region around the keypoint.
  • the peak in the orientation histogram which corresponds to a dominant direction of the gradients local to a keypoint, is assigned to be the feature's orientation.
  • the SIFT processor 254 of the feature extractor 252 With the orientation of each keypoint assigned, the SIFT processor 254 of the feature extractor 252 generates 408 a feature descriptor to characterize the image data in a region surrounding each identified keypoint at its respective orientation.
  • the surrounding region within the associated DoG image is subdivided into an Mx M array of subfields aligned with the keypoint's assigned orientation.
  • Each subf ⁇ eld in turn is characterized by an orientation histogram having a plurality of bins, each bin representing the sum of the image's gradient magnitudes having an orientation within a particular angular range and present within the associated subf ⁇ eld.
  • the feature descriptor includes a 128 byte array corresponding to a 4 x 4 array of subfields with each subfield including eight bins corresponding to an angular width of 45 degrees.
  • the feature descriptor in the preferred embodiment further includes an identifier of the associated image, the scale of the DoG image in which the associated keypoint was identified, the orientation of the feature, and the geometric location of the keypoint in the associated DoG image.
  • SURF Speeded Up Robust Features
  • the SURF technique uses a Hessian matrix composed of box filters that operator on points of the image to determine the location of keypoints as well as the scale of the image data at which the keypoint is an extremum in scale space.
  • the box filters approximate Gaussian second order derivative filters.
  • An orientation is assigned to the feature based on Gaussian- weighted, Haar-wavelet responses in the horizontal and vertical directions. A square aligned with the assigned orientation is centered about the point for purposes of generating a a feature descriptor.
  • Haar-wavelet responses are generated at multiple points for orthogonal directions in each of 4 x 4 sub-regions that make up the square.
  • SURF is taught in: Herbert Bay, Tinne Tuytelaars, Luc Van Gool, "SURF: Speeded Up Robust Features", Proceedings of the ninth European Conference on Computer Vision, May 2006, which is hereby incorporated by reference herein.
  • Exemplary feature detectors include the salient region detector that maximizes the entropy within the region, which was proposed by Kadir and Brady; and the edge- based region detector proposed by June et al; and various aff ⁇ ne-invariant feature detectors known to those skilled in the art.
  • Exemplary feature descriptors include Gaussian derivatives, moment invariants, complex features, steerable filters, and phase-based local features known to those skilled in the art.
  • FIG. 5 Illustrated in FIG. 5 is a flowchart of the method of identifying one or more known objects that match the visual features extracted from the photograph, as referenced in step 308 of FIG. 3.
  • each of the extracted feature descriptors of the user's photograph is compared to the feature descriptors 272 of the known objects to identify 502 matching features.
  • Two features match when the Euclidian distance between their respective SIFT feature descriptors is below some threshold.
  • These matching features referred to here as nearest neighbors, may be identified in any number of ways including a linear search ("brute force search").
  • the pattern recognition module 256 identifies a nearest- neighbor using a Best-Bin-First search in which the vector components of a feature descriptor are used to search a binary tree composed from each of the feature descriptors of the other images to be searched.
  • a Best-Bin-First search is generally less accurate than the linear search, the Best-Bin-First search provides substantially the same results with significant computational savings.
  • a counter associated with the particular known object containing the nearest neighbor is incremented to effectively enter a "vote" to ascribe similarity between the image of the product and the known object with respect to the particular feature.
  • the voting is performed in a 5 dimensional space where the dimensions are image identifier or number, and the relative scale, rotation, and translation of the two matching features.
  • the known objects that accumulate a number of "votes" in excess of a predetermined threshold (or dynamically determined threshold) are selected for subsequent processing as described below while the known objects with an insufficient number of votes are removed from further consideration.
  • the pattern recognition module 256 determines 504 the geometric consistency between the sets of matching features common to the photograph of the item 210 and each image of a known object selected for subsequent processing.
  • a combination of features referred to as "feature patterns” that are common to the DoG images associated with the photograph and known object are aligned using an affine transformation, which maps the coordinates of features of the item's image to the coordinates of the corresponding features in the image of the known object. If the feature patterns are associated with the same underlying item 210 or product, the feature descriptors characterizing the product will geometrically align with only minor difference in the respective feature coordinates.
  • the degree to which the images of the known object and the item 210 match (or fail to match) can be quantified in terms of a "residual error" computed 506 for each affine transform comparison.
  • a small error signifies a close alignment between the feature patterns which may be due to the fact that the same underlying object is being depicted in the two images.
  • a large error generally indicates that the feature patterns do not align, although common feature descriptors match individually by coincidence.
  • the object recognition server 120 may be employed to: (1 ) identify and locate images including a known brand, design, or logo; (2) identify and locate several instances of the same image or substantially similar versions of an image that may include minor editing, the forms of editing including, for instance, cropping, re-sampling, and modification or removal of copyright information; (3) identify and locate all images in a database of images or video sequences, for example, that contain a user specified visual pattern, even if the original graphic has been distorted by: changes in scale, rotations (in-plane and out-of-plane), translations, aff ⁇ ne geometric transformation, changes in brightness, changes in color, changes in gamma, compression artifacts, noise from various sources, lens distortion from an imaging process, cropping, changes in lighting, and occlusions that may obstruct portions of an object to be recognized (4) look-up works of art (paintings, statues, buildings, monuments) for reviews, historical information, and tour guide information, (5) look-up building addresses

Abstract

Système et procédé de reconnaissance d'objets au moyen d'un dispositif informatique mobile (110) et d'un serveur (120). Le dispositif de calcul mobile, de préférence un téléphone à appareil de photo, est conçu pour prendre (202) des clichés ou une séquence vidéo numériques, extraire (304) des caractéristiques visuelles des données d'image et transmettre (306) ces caractéristiques visuelles à un serveur via un réseau cellulaire ou Internet, notamment. A la réception, le serveur (120) compare les caractéristiques extraites aux caractéristiques d'une pluralité d'objets connus dans le but d'identifier (308) un ou plusieurs articles décrits dans les données d'image. Selon l'article identifié, le serveur peut exécuter une ou plusieurs actions prédéterminées, dont la transmission (312) d'informations produit au téléphone mobile. Dans ces informations produit peuvent être spécifiés le prix, la quantité, la disponibilité ou encore l'emplacement de l'article identifié.
PCT/US2007/011120 2006-05-10 2007-05-08 Dispositif informatique mobile à capacité d'imagerie WO2007130688A2 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US79962106P 2006-05-10 2006-05-10
US60/799,621 2006-05-10

Publications (2)

Publication Number Publication Date
WO2007130688A2 true WO2007130688A2 (fr) 2007-11-15
WO2007130688A3 WO2007130688A3 (fr) 2008-11-06

Family

ID=38668400

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2007/011120 WO2007130688A2 (fr) 2006-05-10 2007-05-08 Dispositif informatique mobile à capacité d'imagerie

Country Status (1)

Country Link
WO (1) WO2007130688A2 (fr)

Cited By (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009113992A1 (fr) * 2008-03-11 2009-09-17 Sony Ericsson Mobile Communications Ab Systèmes et procédés d'insertion de publicité pour caméras numériques à reconnaissance d'objets
EP2122535A1 (fr) * 2007-01-25 2009-11-25 Sony Electronics Inc. Programmes video portatifs
US20100122283A1 (en) * 2008-11-12 2010-05-13 Alcatel-Lucent Usa Inc. Targeted advertising via mobile enhanced reality
WO2010055399A1 (fr) * 2008-11-12 2010-05-20 Nokia Corporation Procédé et appareil pour représenter et identifier des descripteurs de caractéristiques à l'aide d'un histogramme compressé de gradients
WO2011151477A1 (fr) * 2010-06-01 2011-12-08 Milenium Espacio Soft, S.A. Procédé de reconnaissance d'objets
WO2012019794A1 (fr) * 2010-08-10 2012-02-16 Technische Universität München Procédé de localisation visuel
CN102572450A (zh) * 2012-01-10 2012-07-11 中国传媒大学 基于sift特征与grnn网络的立体视频颜色校正方法
WO2013033442A1 (fr) 2011-08-30 2013-03-07 Digimarc Corporation Procédés et agencements d'identification d'objets
CN103164856A (zh) * 2013-03-07 2013-06-19 南京工业大学 一种基于稠密sift流的视频复制粘贴盲检测方法
JP2013125506A (ja) * 2011-12-16 2013-06-24 Pasuko:Kk 撮影対象検索システム
EP2635997A2 (fr) * 2010-11-04 2013-09-11 Digimarc Corporation Procédés et systèmes basés sur des smartphones
US8687891B2 (en) 2009-11-19 2014-04-01 Stanford University Method and apparatus for tracking and recognition with rotation invariant feature descriptors
US8740085B2 (en) 2012-02-10 2014-06-03 Honeywell International Inc. System having imaging assembly for use in output of image data
US8818024B2 (en) 2009-03-12 2014-08-26 Nokia Corporation Method, apparatus, and computer program product for object tracking
CN104036245A (zh) * 2014-06-10 2014-09-10 电子科技大学 一种基于在线特征点匹配的生物特征识别方法
WO2015027226A1 (fr) * 2013-08-23 2015-02-26 Nantmobile, Llc Gestion de contenu basée sur la reconnaissance, systèmes et procédés
CN104637055A (zh) * 2015-01-30 2015-05-20 天津科技大学 一种基于小尺度特征点的高精度图像匹配方法
CN105513038A (zh) * 2014-10-20 2016-04-20 网易（杭州）网络有限公司 图像匹配方法及手机应用测试平台
EP3031032A1 (fr) * 2012-08-09 2016-06-15 WINK-APP Ltd. Procédé et système pour relier des objets imprimés avec un contenu électronique
CN105809690A (zh) * 2016-03-09 2016-07-27 联想(北京)有限公司 一种数据处理方法、装置及电子设备
CN105959696A (zh) * 2016-04-28 2016-09-21 成都三零凯天通信实业有限公司 一种基于sift特征算法的视频内容安全监管方法
WO2017018602A1 (fr) * 2015-07-30 2017-02-02 엘지전자 주식회사 Terminal mobile et procédé de commande correspondant
CN106407989A (zh) * 2016-09-07 2017-02-15 厦门大学 基于快速自动密度聚类的可变尺度红外小目标的检测方法
EP3118801A4 (fr) * 2014-03-11 2017-09-13 Research And Innovation Co., Ltd. Système d'utilisation d'informations d'achat, procédé d'utilisation d'informations d'achat et programme
GB2548316A (en) * 2015-12-01 2017-09-20 Zaptobuy Ltd Methods and systems for identifying an object in a video image
EP2192525A3 (fr) * 2008-12-01 2017-12-13 Electronics and Telecommunications Research Institute Appareil pour fournir des contenants numériques et son procédé
US10185976B2 (en) * 2014-07-23 2019-01-22 Target Brands Inc. Shopping systems, user interfaces and methods
WO2019157582A1 (fr) * 2018-02-14 2019-08-22 Hummig Ednilson Guimaraes Plateforme de localisation d'objets
US10432601B2 (en) 2012-02-24 2019-10-01 Nant Holdings Ip, Llc Content activation via interaction-based authentication, systems and method
US10650442B2 (en) 2012-01-13 2020-05-12 Amro SHIHADAH Systems and methods for presentation and analysis of media content
CN111881923A (zh) * 2020-07-28 2020-11-03 民生科技有限责任公司 一种基于特征匹配的票据要素提取方法

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040199430A1 (en) * 2003-03-26 2004-10-07 Victor Hsieh Online intelligent multilingual comparison-shop agents for wireless networks
US20070179921A1 (en) * 2006-01-27 2007-08-02 Microsoft Corporation Object instance recognition using feature symbol triplets

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040199430A1 (en) * 2003-03-26 2004-10-07 Victor Hsieh Online intelligent multilingual comparison-shop agents for wireless networks
US20070179921A1 (en) * 2006-01-27 2007-08-02 Microsoft Corporation Object instance recognition using feature symbol triplets

Cited By (53)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2122535A4 (fr) * 2007-01-25 2010-08-04 Sony Electronics Inc Programmes video portatifs
EP2122535A1 (fr) * 2007-01-25 2009-11-25 Sony Electronics Inc. Programmes video portatifs
WO2009113992A1 (fr) * 2008-03-11 2009-09-17 Sony Ericsson Mobile Communications Ab Systèmes et procédés d'insertion de publicité pour caméras numériques à reconnaissance d'objets
US8098881B2 (en) 2008-03-11 2012-01-17 Sony Ericsson Mobile Communications Ab Advertisement insertion systems and methods for digital cameras based on object recognition
RU2505856C2 (ru) * 2008-11-12 2014-01-27 Нокиа Корпорейшн Способ и устройство для представления и идентификации дескрипторов признаков с использованием сжатой гистограммы градиентов
US20100122283A1 (en) * 2008-11-12 2010-05-13 Alcatel-Lucent Usa Inc. Targeted advertising via mobile enhanced reality
US9710492B2 (en) 2008-11-12 2017-07-18 Nokia Technologies Oy Method and apparatus for representing and identifying feature descriptors utilizing a compressed histogram of gradients
WO2010055399A1 (fr) * 2008-11-12 2010-05-20 Nokia Corporation Procédé et appareil pour représenter et identifier des descripteurs de caractéristiques à l'aide d'un histogramme compressé de gradients
KR101323439B1 (ko) * 2008-11-12 2013-10-29 보드 오브 트러스티스 오브 더 리랜드 스탠포드 주니어 유니버시티 특징 디스크립터를 표현하고 식별하는 방법, 장치 및 컴퓨터 판독가능 저장 매체
US20100226575A1 (en) * 2008-11-12 2010-09-09 Nokia Corporation Method and apparatus for representing and identifying feature descriptions utilizing a compressed histogram of gradients
EP2192525A3 (fr) * 2008-12-01 2017-12-13 Electronics and Telecommunications Research Institute Appareil pour fournir des contenants numériques et son procédé
US8818024B2 (en) 2009-03-12 2014-08-26 Nokia Corporation Method, apparatus, and computer program product for object tracking
US8687891B2 (en) 2009-11-19 2014-04-01 Stanford University Method and apparatus for tracking and recognition with rotation invariant feature descriptors
WO2011151477A1 (fr) * 2010-06-01 2011-12-08 Milenium Espacio Soft, S.A. Procédé de reconnaissance d'objets
US10229136B2 (en) 2010-08-10 2019-03-12 Navvis Gmbh Visual localization method
US9946733B2 (en) 2010-08-10 2018-04-17 Navvis Gmbh Visual localization method
US10585938B2 (en) 2010-08-10 2020-03-10 Navvis Gmbh Visual localization method
WO2012019794A1 (fr) * 2010-08-10 2012-02-16 Technische Universität München Procédé de localisation visuel
US11803586B2 (en) 2010-08-10 2023-10-31 Navvis Gmbh Visual localization method
US9641981B2 (en) 2010-08-10 2017-05-02 Navvis Gmbh Visual localization method
US10956489B2 (en) 2010-08-10 2021-03-23 Navvis Gmbh Visual localization method
EP2635997A4 (fr) * 2010-11-04 2015-01-07 Digimarc Corp Procédés et systèmes basés sur des smartphones
EP2635997A2 (fr) * 2010-11-04 2013-09-11 Digimarc Corporation Procédés et systèmes basés sur des smartphones
WO2013033442A1 (fr) 2011-08-30 2013-03-07 Digimarc Corporation Procédés et agencements d'identification d'objets
JP2013125506A (ja) * 2011-12-16 2013-06-24 Pasuko:Kk 撮影対象検索システム
CN102572450A (zh) * 2012-01-10 2012-07-11 中国传媒大学 基于sift特征与grnn网络的立体视频颜色校正方法
US10650442B2 (en) 2012-01-13 2020-05-12 Amro SHIHADAH Systems and methods for presentation and analysis of media content
US8740085B2 (en) 2012-02-10 2014-06-03 Honeywell International Inc. System having imaging assembly for use in output of image data
US11503007B2 (en) 2012-02-24 2022-11-15 Nant Holdings Ip, Llc Content activation via interaction-based authentication, systems and method
US10432601B2 (en) 2012-02-24 2019-10-01 Nant Holdings Ip, Llc Content activation via interaction-based authentication, systems and method
US10841292B2 (en) 2012-02-24 2020-11-17 Nant Holdings Ip, Llc Content activation via interaction-based authentication, systems and method
EP3031032A4 (fr) * 2012-08-09 2017-03-29 WINK-APP Ltd. Procédé et système pour relier des objets imprimés avec un contenu électronique
US9916499B2 (en) 2012-08-09 2018-03-13 Wink-App Ltd. Method and system for linking printed objects with electronic content
EP3031032A1 (fr) * 2012-08-09 2016-06-15 WINK-APP Ltd. Procédé et système pour relier des objets imprimés avec un contenu électronique
CN103164856A (zh) * 2013-03-07 2013-06-19 南京工业大学 一种基于稠密sift流的视频复制粘贴盲检测方法
US11042607B2 (en) 2013-08-23 2021-06-22 Nant Holdings Ip, Llc Recognition-based content management, systems and methods
WO2015027226A1 (fr) * 2013-08-23 2015-02-26 Nantmobile, Llc Gestion de contenu basée sur la reconnaissance, systèmes et procédés
EP3118801A4 (fr) * 2014-03-11 2017-09-13 Research And Innovation Co., Ltd. Système d'utilisation d'informations d'achat, procédé d'utilisation d'informations d'achat et programme
US11263673B2 (en) 2014-03-11 2022-03-01 Research And Innovation Co., Ltd. Purchase information utilization system, purchase information utilization method, and program
US11769182B2 (en) 2014-03-11 2023-09-26 Research And Innovation Co., Ltd. Purchase information utilization system, purchase information utilization method, and program
CN104036245B (zh) * 2014-06-10 2018-04-06 电子科技大学 一种基于在线特征点匹配的生物特征识别方法
CN104036245A (zh) * 2014-06-10 2014-09-10 电子科技大学 一种基于在线特征点匹配的生物特征识别方法
US10185976B2 (en) * 2014-07-23 2019-01-22 Target Brands Inc. Shopping systems, user interfaces and methods
CN105513038B (zh) * 2014-10-20 2019-04-09 网易（杭州）网络有限公司 图像匹配方法及手机应用测试平台
CN105513038A (zh) * 2014-10-20 2016-04-20 网易（杭州）网络有限公司 图像匹配方法及手机应用测试平台
CN104637055A (zh) * 2015-01-30 2015-05-20 天津科技大学 一种基于小尺度特征点的高精度图像匹配方法
WO2017018602A1 (fr) * 2015-07-30 2017-02-02 엘지전자 주식회사 Terminal mobile et procédé de commande correspondant
GB2548316A (en) * 2015-12-01 2017-09-20 Zaptobuy Ltd Methods and systems for identifying an object in a video image
CN105809690A (zh) * 2016-03-09 2016-07-27 联想(北京)有限公司 一种数据处理方法、装置及电子设备
CN105959696A (zh) * 2016-04-28 2016-09-21 成都三零凯天通信实业有限公司 一种基于sift特征算法的视频内容安全监管方法
CN106407989A (zh) * 2016-09-07 2017-02-15 厦门大学 基于快速自动密度聚类的可变尺度红外小目标的检测方法
WO2019157582A1 (fr) * 2018-02-14 2019-08-22 Hummig Ednilson Guimaraes Plateforme de localisation d'objets
CN111881923A (zh) * 2020-07-28 2020-11-03 民生科技有限责任公司 一种基于特征匹配的票据要素提取方法

Also Published As

Publication number Publication date
WO2007130688A3 (fr) 2008-11-06

Similar Documents

Publication Publication Date Title
WO2007130688A2 (fr) Dispositif informatique mobile à capacité d'imagerie
US9317769B2 (en) Image capture and identification system and process
JP5427859B2 (ja) 画像取込及び識別のためのシステム
US20090141986A1 (en) Image Capture and Identification System and Process
US8355533B2 (en) Method for providing photographed image-related information to user, and mobile system therefor
JP2013167973A (ja) 検索装置、検索方法、検索プログラム、及びそのプログラムを記憶するコンピュータ読取可能な記録媒体
JP5967036B2 (ja) 画像検索システム、情報処理装置及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 07809049

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 07809049

Country of ref document: EP

Kind code of ref document: A2