WO2008094951A1 - Système et procédé d'édition d'image - Google Patents

Système et procédé d'édition d'image Download PDF

Info

Publication number
WO2008094951A1
WO2008094951A1 PCT/US2008/052367 US2008052367W WO2008094951A1 WO 2008094951 A1 WO2008094951 A1 WO 2008094951A1 US 2008052367 W US2008052367 W US 2008052367W WO 2008094951 A1 WO2008094951 A1 WO 2008094951A1
Authority
WO
WIPO (PCT)
Prior art keywords
interest
image
points
display
crop area
Prior art date
Application number
PCT/US2008/052367
Other languages
English (en)
Inventor
Andrew Gavin
Scott Shumaker
Ben Stragnell
Original Assignee
Flektor, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Flektor, Inc. filed Critical Flektor, Inc.
Publication of WO2008094951A1 publication Critical patent/WO2008094951A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/60—Editing figures and text; Combining figures or text
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/387—Composing, repositioning or otherwise geometrically modifying originals
    • H04N1/3872—Repositioning or masking
    • H04N1/3873—Repositioning or masking defined only by a limited number of coordinate points or parameters, e.g. corners, centre; for trimming

Definitions

  • the present invention relates to an image editing system, and more particularly, to a system for detecting and editing points of interest on images and manipulating those images using the points of interest for specific applications such as cropping, animation, and navigation..
  • FIG. 1 is a block diagram of a web-based video editing system according to a first embodiment of the present invention.
  • FIG. Ia is a block diagram of a web-based video editing system according to a second embodiment of the present invention.
  • FIG. 2 is a block diagram of one embodiment of an image editing system of the web- based video editing system of FIG. 1.
  • FIG. 3 is a flowchart showing a method of operation for the detection subsystem.
  • the subsystem receives data pertaining to a particular image in the system.
  • FIG. 3a shows a sample image containing points of interest. Ln this example, the image contains two figures with recognizable human faces.
  • FIG. 4 shows a flowchart for a method of operation to automatically crop an image using data for predetermined points of interest in the image.
  • FIG. 5a is a visual representation of the method by which image data in a landscape orientation is processed for photo-cropping.
  • FIG. 5b is a visual representation of a method by which image data in a portrait orientation is processed for photo-cropping.
  • FIG. 5c is a visual representation of a method by which image data is processed for photo-cropping.
  • FIG. 1 is a block diagram of a web-based video editing system according to a first embodiment of the present invention.
  • the editing system includes one or more communication devices 1 10 each having a graphical user interface 115, a server 120 having a connection manager
  • the communication devices include, but are not limited to, a personal computer, a mobile telephone, a PDA, or any other communication device configured to operate as a client computer to the server.
  • the network to which the server and devices are coupled may be a wireless or a wireline network and may range in size from a local area network to a wide area network to the Internet. A dedicated open socket connection exists between the connection manager and the client computers.
  • one or more client computers are configured to transmit information to and receive information from the server, hi some embodiments, each of the client computers is configured to send a query for information and the server is configured to respond to the query by sending the requested information to the client computer. In some embodiments, one or more of the client computers is configured to transmit commands to the server and the server is configured to perform functions in response to the command.
  • each of the client computers is configured with an application for displaying multimedia on the graphical user interface of the client computer.
  • the application may be Adobe Flash® or any other application capable of displaying multimedia.
  • connection manager is configured to determine the condition of the server and perform asynchronous messaging to one or more of the client computers over the dedicated open socket connection.
  • the content of the messages is indicative of the state of the server.
  • the server is configured to receive requests from one or more of the client computers and perform functions in response to the received requests.
  • the server performs any number of functions typically performed in the server of a web-based video editing system.
  • the server also provides an image editing system for the web-based video editing system.
  • FIG. Ia is a block diagram of some embodiments of the web-based video editing system of FIG. 1. hi these embodiments, the system does not include a connection manager 130a for communication between the client computer 110a and the server 120a. Otherwise, client computer 110a is an instance of client computer 110, graphical user interface 115a is an instance of grpahical user interface 115, server 120a is an instance of server 120, image editing system 140a is an instance of image editing system 140, and internet 150a is an instance of internet 150. All the components in FIG. Ia are thus identical to those in FIG. 1, and they are all configured to operate as described in FIG. 1.
  • the combination of data representing specified portions of an image is sometimes referred to as a point of interest.
  • These points of interest may take the shape of a square, rectangle, or other quadrilateral, a circle, an ellipse, or any other closed two-dimensional shape.
  • a user operates the client computer 110 to manage and manipulate points of interest on a computer image. The user can add, reposition, and resize points of interest through the graphical user interface 115.
  • the server 120 automatically detects points of interest by using algorithms such as face detection and feature recognition.
  • the server-generated points of interest are also presented to the user in a graphical fashion, allowing the user to adjust their size and position and remove false positives or undesired interest points in an identical fashion to the points added by the user. Once these points of interest have been saved, they are stored in a database on the server along with the image and used appropriately as the image is displayed in different contexts.
  • users can choose to present the image in an animated fashion, and the client computer will use the points of interest to intelligently pan around the image and focus on parts of the image.
  • the points of interest become 'hot spots' suitable for web-based navigation.
  • FIG. 2 is a block diagram of one embodiment of an image-editing system 140 of the web-based video editing system of FIG. 1.
  • the image editing system includes a detection subsystem 210.
  • the detection subsystem includes a processor 212, memory
  • the memory 214 and computer code product including instruction stored in the memory and adapted to cause the processor, and thereby the detection subsystem, to receive and process user point of interest detection and selection requests.
  • the memory also stores information indicative of the user selection requests.
  • the memory may be any type of read- write memory, including, but not limited to, random access memory.
  • the user input received by the detection subsystem includes the identity of the user, the image to be cropped, the size to which the image is to be cropped, and the points of interest edited by the user and/or generated by the server and stored on the server in an earlier stage.
  • Each point of interest consists of a rectangular region of the image that encompasses the point of interest, although in some embodiments this point could be a circle (position and radius) or any other closed two-dimensional shape.
  • the data is be stored in the editing system server (not shown), the detection subsystem memory 214, or at a remote location connected to the network of FIG. 1.
  • the data is provided by the web-based video editing system or is data generated by the user.
  • the data may include uncropped images, cropped images, meta data for cropping the images, and points of interest.
  • the system includes an animation subsystem configured to examine the points of interest and present an intelligent fly through of the image.
  • the system includes a navigation subsystem that allows a user to annotate the points of interest with text boxes and hyperlinks.
  • FIG. 3 is a flowchart showing a method of operation for the detection subsystem.
  • the subsystem receives 310 data pertaining to a particular image in the system. This data is a pointer to a file on the server, a pointer to a file stored in a remote location connected to the server by the network in FIG. 1 , the actual data for the image, or any other data that would allow the subsystem access to the image data.
  • the subsystem then processes 320 the image using a facial recognition detection algorithm to determine all of the faces in the image and creates points of interest for each of the detected faces.
  • the subsystem either transmits (not shown) the data to the image editing system, or the subsystem transmits 330 the data to the client computer.
  • the client computer displays the image alone with markers indicating each of the automatically selected points of interest.
  • the system receives 340 input from the user indicating any changes to the selected points of interest that he desires. For example, the user may remove some of the preselected points of interest, or the user may add additional points of interest.
  • the additional points of interest may include any particular features of the image in addition to the automatically detected faces that the user wishes to have included in the cropped image.
  • the subsystem transmits 350 the data for the points of interest to the image editing system.
  • FIG. 3a shows a sample image containing points of interest. In this example, the image 300 contains two figures 310 with recognizable human faces.
  • the detection subsystem When the detection subsystem processes this image, it automatically applies points of interest 320 to these faces.
  • the image also contains points of interest 330 added by the user so that a pet 340 and a painting 350 are also marked as being important in the image and thus represent visuals within in the image that the user desires to include in the final cropped image.
  • FIG. 4 shows a flowchart for a method of operation to automatically crop an image using data for predetermined points of interest in the image.
  • the image editing system receives 410 data pertaining to points of interest either automatically generated or generated by the user and data pertaining to the image to be cropped.
  • the system further receives 420 data pertaining to the size and orientation to which the image is to be cropped. Based on this data, the system determines 430 whether to crop the image along its height or along its width.
  • the system analyzes every valid configuration of the crop rectangle by beginning 440 at one side of the image.
  • the system examines 450 the locations of the points of interest in relation to the position of the crop rectangle, and assigns 460 a score to the current crop rectangle configuration.
  • the system In assigning the score, the system tries to maximize the area of points of interest displayed inside the crop rectangle, to avoid partially cropping a point of interest and cutting it off.
  • the system also adjusts the score to display the points of interest in accordance with basic photographic rules, such as the rule of thirds, centering, framing, balance, rotation such that all faces in the image are vertical, use of the golden-ratio, etc.
  • basic photographic rules such as the rule of thirds, centering, framing, balance, rotation such that all faces in the image are vertical, use of the golden-ratio, etc.
  • the photographic rules contribute just enough to the score to allow the system to choose between crop configurations that would otherwise be tied.
  • the system determines 470 whether it has reached the other end of the image. If the system has not reached the other end, it moves 480 the crop area by one pixel, analyzes 450 the current position, and assigns 460 it a score. If instead, the system determines 470 that it has reached the end of the image, it selects 490 and stores the crop configuration with the highest score. The system may then crop the image accordingly.
  • the system also allows the user to preview the cropped image overlaid on the original image and adjust the cropped region.
  • FIG. 5a is a visual representation of the method by which image data in a landscape orientation is processed for photo-cropping.
  • the image 51 Oa contains three points of interest 520a.
  • the system processes this image for cropping, it begins with the cropping area 530a at the left side of the image and walks it over to the right side pixel by pixel.
  • the cropping area is in position 530a, two points of interest are captured; however, when it is in position 540a, only one point of interest is captured.
  • the system crops this image it will crop it at position 530a.
  • FIG. 5b is a visual representation of a method by which image data in a portrait orientation is processed for photo-cropping.
  • the image 51 Ob contains five points of interest 520b, 53Ob, 540b, 550b, and 560b.
  • the cropping area 590b moves from top to bottom in search of the area that contains the most points of interest.
  • position 570b contains three complete points of interest and therefore has the highest concentration.
  • the next highest concentration is found in position 580b, which only has two complete points of interest.
  • FIG. 531 is a visual representation of a method by which image data in a portrait orientation is processed for photo-cropping.
  • the image 51 Ob contains five points of interest 520b, 53Ob, 540b, 550b, and 560b.
  • the cropping area 590b moves from top to bottom in search of the area that contains the most points of interest.
  • position 570b contains three complete points of interest and therefore has the highest concentration.
  • the next highest concentration is found in position 580b
  • 5c is a visual representation of a method by which image data is processed for photo-cropping.
  • the image 510c in FIG. 5c is processed much in the same manner as discussed in reference to FIGs. 5a and 5b, above.
  • the point of interest 530c is only partially contained within the cropping area.
  • position 550c both the points of interest are completely contained in the cropping area.
  • Position 550c would thus have a higher score than position 560c, and the system would therefore crop this image at position 550c.
  • the points of interest are used as hot-spots for navigation.
  • Users can associate hyperlinks or text with a hot-spot through a graphical user interface.
  • a viewer watches the image on a client machine
  • positioning the mouse cursor over a hot-spot can display a text popup, and clicking o the hotspot can perform a navigation action, such as opening a new web- page.
  • the points of interest are used as reference points for automatic animation of the image.
  • the user may have the system assign hot-spots and/or perform animation on the cropped image, or the user may use the system solely to detect points of interest and then use an uncropped image for hot-spots and/or animation.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Processing Or Creating Images (AREA)

Abstract

L'invention concerne un système d'édition d'image pour l'utilisation avec un système d'édition vidéo Web. Le système d'édition d'image comprend : un système de détection configuré pour recevoir des données d'image ; détecter des régions de l'image représentant des visages ; stocker les données représentant les coordonnées des visages comme des points intéressants ; et recevoir une entrée d'utilisateur ajoutant des points intéressants additionnels, modifiant les points intéressants ou supprimant les points intéressants présélectionnés ; et un sous-système de recadrage configuré pour déterminer la partie de l'image contenant le nombre maximum de points intéressants qui rentreront dans la zone de recadrage ; et recadrer l'image.
PCT/US2008/052367 2007-01-29 2008-01-29 Système et procédé d'édition d'image WO2008094951A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US89820107P 2007-01-29 2007-01-29
US60/898,201 2007-01-29

Publications (1)

Publication Number Publication Date
WO2008094951A1 true WO2008094951A1 (fr) 2008-08-07

Family

ID=39674483

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2008/052367 WO2008094951A1 (fr) 2007-01-29 2008-01-29 Système et procédé d'édition d'image

Country Status (1)

Country Link
WO (1) WO2008094951A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013049374A3 (fr) * 2011-09-27 2013-05-23 Picsured, Inc. Numérisation de photographie par l'utilisation de photographie vidéo et de technologie de vision informatique

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040228528A1 (en) * 2003-02-12 2004-11-18 Shihong Lao Image editing apparatus, image editing method and program
US20050025387A1 (en) * 2003-07-31 2005-02-03 Eastman Kodak Company Method and computer program product for producing an image of a desired aspect ratio
US20050278636A1 (en) * 2004-06-09 2005-12-15 Canon Kabushiki Kaisha Image processing apparatus, image processing method, program for implementing the method, and storage medium storing the program
US20060238827A1 (en) * 2005-04-20 2006-10-26 Fuji Photo Film Co., Ltd. Image processing apparatus, image processing system, and image processing program storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040228528A1 (en) * 2003-02-12 2004-11-18 Shihong Lao Image editing apparatus, image editing method and program
US20050025387A1 (en) * 2003-07-31 2005-02-03 Eastman Kodak Company Method and computer program product for producing an image of a desired aspect ratio
US20050278636A1 (en) * 2004-06-09 2005-12-15 Canon Kabushiki Kaisha Image processing apparatus, image processing method, program for implementing the method, and storage medium storing the program
US20060238827A1 (en) * 2005-04-20 2006-10-26 Fuji Photo Film Co., Ltd. Image processing apparatus, image processing system, and image processing program storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013049374A3 (fr) * 2011-09-27 2013-05-23 Picsured, Inc. Numérisation de photographie par l'utilisation de photographie vidéo et de technologie de vision informatique
US20140348394A1 (en) * 2011-09-27 2014-11-27 Picsured, Inc. Photograph digitization through the use of video photography and computer vision technology

Similar Documents

Publication Publication Date Title
US8218830B2 (en) Image editing system and method
US11706521B2 (en) User interfaces for capturing and managing visual media
US11223771B2 (en) User interfaces for capturing and managing visual media
US20230319394A1 (en) User interfaces for capturing and managing visual media
US11941762B2 (en) System and method for augmented reality scenes
US8749587B2 (en) System and method for content based automatic zooming for document viewing on small displays
EP1630704B1 (fr) Procédé, système, programme et support de stockage pour la gestion de fichiers d'image
US7006091B2 (en) Method and system for optimizing the display of a subject of interest in a digital image
US20230137901A1 (en) Techniques to Modify Content and View Content on Mobile Devices
AU2022200966B2 (en) User interfaces for capturing and managing visual media
US8601393B2 (en) System and method for supporting document navigation on mobile devices using segmentation and keyphrase summarization
US20140248950A1 (en) System and method of interaction for mobile devices
US20150007024A1 (en) Method and apparatus for generating image file
US8532435B1 (en) System and method for automatically adapting images
US20050116966A1 (en) Web imaging serving technology
US20090100333A1 (en) Visualizing circular graphic objects
GB2314492A (en) Computer based apparatus and method for searching and retrieving moving image information
CN111612873A (zh) Gif图片生成方法、装置及电子设备
EP2747404A1 (fr) Terminal et système de traitement d'image, support de stockage lisible par ordinateur stockant un programme de commande de terminal de traitement d'image
JP2007133878A (ja) 多数の画像のブラウズ方法、装置、システム、及びプログラム
EP1755051A1 (fr) Procédé et Appareil pour accéder données, utilisant un espace de représentation symbolique
US10304232B2 (en) Image animation in a presentation document
JP2003303333A (ja) 画像表示制御装置
US20120306736A1 (en) System and method to control surveillance cameras via a footprint
JP2019101559A (ja) 情報処理装置、情報処理方法、プログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 08714118

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112 (1) EPC, EPO FORM 1205A DATED 03-12-2009

122 Ep: pct application non-entry in european phase

Ref document number: 08714118

Country of ref document: EP

Kind code of ref document: A1