US20130106990A1 - Planar panorama imagery generation - Google Patents

Planar panorama imagery generation Download PDF

Info

Publication number
US20130106990A1
US20130106990A1 US13/286,756 US201113286756A US2013106990A1 US 20130106990 A1 US20130106990 A1 US 20130106990A1 US 201113286756 A US201113286756 A US 201113286756A US 2013106990 A1 US2013106990 A1 US 2013106990A1
Authority
US
United States
Prior art keywords
object
desired
imagery
desired object
planar panorama
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US13/286,756
Inventor
Samuel Thomas Williams
Jonathan Paul Rosenberg
Johannes Kopf
Don Dongwoo Kim
Sandeep Kanumuri
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Microsoft Technology Licensing LLC
Original Assignee
Microsoft Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Microsoft Corp filed Critical Microsoft Corp
Priority to US13/286,756 priority Critical patent/US20130106990A1/en
Assigned to MICROSOFT CORPORATION reassignment MICROSOFT CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: ROSENBERG, JONATHAN PAUL, KOPF, JOHANNES, KANUMURI, SANDEEP, KIM, DON DONGWOO, WILLIAMS, SAMUEL THOMAS
Publication of US20130106990A1 publication Critical patent/US20130106990A1/en
Assigned to MICROSOFT TECHNOLOGY LICENSING, LLC reassignment MICROSOFT TECHNOLOGY LICENSING, LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: MICROSOFT CORPORATION
Application status is Abandoned legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/222Studio circuitry; Studio devices; Studio equipment ; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, TV cameras, video cameras, camcorders, webcams, camera modules for embedding in other devices, e.g. mobile phones, computers or vehicles
    • H04N5/225Television cameras ; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, camcorders, webcams, camera modules specially adapted for being embedded in other devices, e.g. mobile phones, computers or vehicles
    • H04N5/232Devices for controlling television cameras, e.g. remote control ; Control of cameras comprising an electronic image sensor
    • H04N5/23238Control of image capture or reproduction to achieve a very large field of view, e.g. panorama
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/36Image preprocessing, i.e. processing the image information without deciding about the identity of the image
    • G06K9/46Extraction of features or characteristics of the image
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/62Methods or arrangements for recognition using electronic means
    • G06K9/6201Matching; Proximity measures
    • G06K9/6202Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/62Methods or arrangements for recognition using electronic means
    • G06K9/6267Classification techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/002D [Two Dimensional] image generation
    • G06T11/60Editing figures and text; Combining figures or text
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformation in the plane of the image
    • G06T3/40Scaling the whole image or part thereof
    • G06T3/4038Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/36Image preprocessing, i.e. processing the image information without deciding about the identity of the image
    • G06K9/46Extraction of features or characteristics of the image
    • G06K2009/4666Extraction of features or characteristics of the image regional/local feature not essentially salient, e.g. local binary pattern

Abstract

One or more techniques and/or systems are disclosed for generating improved planar panorama imagery, which may be used to view street-level, human-scale locations along a desired path, for example. One or more desired objects can be detected in source imagery that may be used to produce the resulting planar panorama imagery. Respective object regions comprising a desired object can be identified in one or more frames of the source imagery. One or more portions of the source imagery that do not comprise the object region for the desired object can be joined with at least some of the one or more frames that do comprise the object region for the desired object. The resulting planar panorama imagery can thus comprise the desired object. Also, the planar panorama imagery can be generated so as to lack one or more non-desired objects and/or adverse effects that may result therefrom.

Description

    BACKGROUND
  • Digital photography can allow for a sequence of images to be stitched or glued together to provide for a relatively seamless transition from one image to the next. Further, images, such as human-scale lateral images, can be collected while traveling along a route, such as a street, and/or generated from other local imagery sources, for example. Stitching such images together can provide a user experience of travelling along a particular locale, for example.
  • SUMMARY
  • This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key factors or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
  • A side or lateral-view of panoramic imagery may comprise a series of images (e.g., photo frames, video frames, etc.) stitched/glued together to form a somewhat seamless view of the imagery. This type of planar panorama imagery typically displays many of the stitched together images at the same time. Currently, images stitched together in this manner can be utilized in conjunction with digital mapping services, where, for example, a user may view planar panorama-type images of a street associated with a map they may be viewing.
  • When moving laterally along planar panorama imagery a user might see different perspectives of objects in the panorama and/or objects in the images may be altered due to different viewing angles from which the different images were acquired and/or resulting from imprecisions in image stitching processes, for example. A business sign may, for example, be comprised within/across two adjacent images of a planar panorama image sequence, such that a first portion of the sign is in a first image and a second portion of the sign is in a second image. When the adjacent images are stitched together a resulting view of the sign may be distorted due to respective (different) viewing angles of the first and second images and/or the stitching process not aligning the images appropriately, for example. Further, the business sign may be fully or partially obscured by one or more other objects, such as a telephone pole, person, etc., where the sign may be more desirable to display than the obscuring object(s) in the resulting planar panorama of images.
  • Accordingly, one or more techniques and/or systems are disclosed that can provide for identifying one or more desired objects (e.g., interesting to a user) in one or more portions of planar panorama source imagery (e.g., sequential images or frames collected along a path). Further, non-desired objects (e.g., obscuring, distracting, and/or distorted objects) may be identified in at least some portions of the planar panorama source imagery. Regions in the source imagery that comprise desired and/or non-desired objects can be identified, and merely those frames (e.g., or portions thereof) of the source imagery that comprise desired views (e.g., comprising desired objects and not non-desired objects) may be joined together to generate planar panorama imagery.
  • In one embodiment of generating planar panorama imagery, an object region, comprising a desired object, can be identified in one or more frames of planar panorama source imagery. Additionally, one or more portions of the planar panorama source imagery, which do not comprise the desired object, can be joined with the one or more frames comprising the object region to generate the planar panorama imagery comprising the desired object.
  • To the accomplishment of the foregoing and related ends, the following description and annexed drawings set forth certain illustrative aspects and implementations. These are indicative of but a few of the various ways in which one or more aspects may be employed. Other aspects, advantages, and novel features of the disclosure will become apparent from the following detailed description when considered in conjunction with the annexed drawings.
  • DESCRIPTION OF THE DRAWINGS
  • FIG. 1 is a flow diagram illustrating an exemplary method for generating planar panorama imagery.
  • FIG. 2 is a flow diagram illustrating an example embodiment where one or more portions of one or more techniques described herein may be implemented.
  • FIG. 3 is a flow diagram illustrating an example embodiment where one or more portions of one or more techniques described herein may be implemented.
  • FIGS. 4A and 4B illustrate example embodiments where one or more portions of one or more techniques described herein may be implemented.
  • FIG. 5 is a component diagram illustrating an exemplary system for generating planar panorama imagery.
  • FIG. 6 is a component diagram illustrating an example embodiment where one or more systems described herein may be implemented.
  • FIG. 7 is an illustration of an exemplary computer-readable medium comprising processor-executable instructions configured to embody one or more of the provisions set forth herein.
  • FIG. 8 illustrates an exemplary computing environment wherein one or more of the provisions set forth herein may be implemented.
  • DETAILED DESCRIPTION
  • The claimed subject matter is now described with reference to the drawings, wherein like reference numerals are generally used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the claimed subject matter. It may be evident, however, that the claimed subject matter may be practiced without these specific details. In other instances, structures and devices are shown in block diagram form in order to facilitate describing the claimed subject matter.
  • As provided herein, a method may be devised that provides for mitigating partial or full loss of interesting objects in planar panorama imagery. Further, objects of interest that may often be distorted due to incorrect meshing or joining of images in the planar panorama imagery may be addressed and/or appropriately corrected. Additionally, objects of distracting interest in the planar panorama imagery, such as people, moving cars, trees or poles, etc. that obscure interesting objects, can be accounted for by removal and/or by selecting a viewing angle with a desired result. In this way, for example, resulting planar panorama imagery may provide an improved user experience, where desired objects can be viewed and the presence of non-desired objects may be mitigated.
  • FIG. 1 is a flow diagram illustrating an exemplary method 100 for generating planar panorama imagery. The exemplary method begins at 102. At 104, a desired object can be identified in planar panorama source imagery (e.g., in one or more images or frames of the planar panorama source imagery). In one embodiment, a desired object may comprise an object (e.g., sign, building, landmark, portion of scenery, lamp-post, person, vehicle, etc.) in an image or frame that can be targeted for further processing. As an example, the desired object may comprise a business sign (e.g., hotel, shop, restaurant, etc.) captured in an image (e.g., comprised in the planar panorama source imagery) on a front of a building within which the business is located. In this embodiment, for example, the business sign may comprise a positive object that is intended to be included in the resulting planar panorama imagery (e.g., resulting from further processing), where the business sign may provide valuable information to a user viewing the planar panorama imagery (e.g., in an online mapping application).
  • At 106, an object region comprising the desired object is identified in one or more frames of the planar panorama source imagery. For example, the planar panorama source imagery can comprise a series (e.g., in relative sequence) of images or frames, which may have been collected along a path (e.g., road, street highway, etc.). In this example, respective frames may capture short intervals of a side-view of the path (e.g., one side of the street, laterally). Further, the object region can comprise, or rather be comprised within, a portion of the frame (e.g., a defined group of pixels, sub-pixels, etc.) that comprises the desired object (e.g., at least a portion of, or the entire object).
  • At 108, one or more portions of the planar panorama source imagery that do not comprise the desired object are combined with the one or more frames that comprise the identified object region (e.g., where the portions of the planar panorama source imagery may merely comprise (remaining) regions of frames because some regions of the frames have been excised and/or are otherwise not usable (e.g., because they contain undesirable/non-desired objects, such as a telephone pole, for example, and/or because they contain a desired object that has already been captured from another frame and thus “doubling” may occur if the object in that region was re-presented)). In this way, the resulting planar panorama imagery comprises the desired object. For example, one or more frames that comprise the object region, indicating the desired object, can be selected to be meshed with (e.g., stitched to, merged with, combined in some manner, etc.) at least some of the portions of the planar panorama source imagery that do not have the desired object. By selecting one or more frames that comprise a desired view of the desired object, the resulting planar panorama imagery, for example, can comprise the desired view of the desired object instead of a view that may provide an obscured view and/or distorted view of the desired object, for example.
  • FIG. 2 is a flow diagram illustrating an example embodiment 200 where one or more portions of one or more techniques described herein may be implemented. Planar panorama source imagery 250 (e.g., images acquired while driving down a street, and/or generated from other imagery acquired while driving down a street (e.g., or other location) from the same and/or different sources) may be collected and sequenced accordingly (e.g., in series illustrating a lateral view of one or more locations comprised along the street). At 202, the planar panorama source imagery 250 can be input to one or more object detectors, for example, where the one or more object detectors can respectively detect merely a particular object in image data.
  • In one embodiment, the planar panorama source imagery 250 can be input to a first object detector used for detecting merely a first object, and the planar panorama source imagery 250 can be input to a second object detector that may be used to merely detect a second object (e.g., and input to a third object detector to merely detect a third object, and so-on). As one example, a set of object detectors may comprise one or more of: a building detector, person detector (e.g., faces and/or people), landscape detector (e.g., trees, bushes, etc.), vehicle detector (e.g., stopped and/or moving), sign detector (e.g., text detector for business and/or street signs), utility object detector (e.g., utility pole, stop light, power lines, etc.), road section detector (e.g., medians), business logo detector, and/or building entryway detector (e.g., business entries, doors, etc.), etc. In this example, the planar panorama source imagery 250 can be input to the plurality of object detectors, where one or more of the object detectors may be activated for detecting an object in the planar panorama source imagery 250.
  • At 204 in the example embodiment 200, one of more desired objects can be identified in the planar panorama source imagery 250. In one embodiment, a desired object can be identified in the planar panorama source imagery using a desired object detector, for example, that may be designed to detect a desired object from among one or more detected objects (e.g., and that may comprise one or more object detectors). At 206, an object region that comprises the identified desired object can be identified in the planar panorama source imagery 250. In one embodiment, a first and a second desired object may be identified in the planar panorama source imagery 250 (e.g., using first and second desired object detectors, respectively); and, in one embodiment, a first object region comprising the first desired object, and a second object region comprising the second desired object, can be identified in one or more frames (e.g., images) of the planar panorama source imagery 250 (e.g., and a third, and fourth, etc.).
  • As an example, a desired object may be detected in image data (e.g., comprised in the planar panorama source imagery 250) based on the object's shape, color, edge arrangement, and/or other characteristics, for example. Further, in one example, using image data information provided by the object detection, a region in one of the frames of the image data that comprises the detected object can be identified. In one example, the object region identified in the frame may comprise a group of pixels and/or sub-pixels (e.g., color units described by the image data), which encompass the desired object. Additionally, in one example, a plurality of desired objects may be detected in the planar panorama source imagery 250, and, for the respective desired objects, a corresponding object region can be identified in at least one frame from the planar panorama source imagery 250.
  • As an illustrative example, FIG. 4A illustrates an example embodiment 400 where one or more portions of one or more techniques described herein may be implemented. In this example 400, a set of frames or images 412 (e.g., photographs, video frames, etc.) may be arranged from one to sixteen, such that they can represent a consecutive sequence of images 412, for example, indicating a lateral view along one side of a path (e.g., road). In this example 400, the images 412 are stacked, where frame number nine 410 is situated on top of the stack (e.g., exposed in full view), with the frames one through eight 408 stacked underneath frame nine 410 to the left, and frames ten through sixteen stacked under frame number nine 410 to the right.
  • In this example 400, a desired object comprises a building 402. The building 402 is captured (e.g., at least partially) in frames six 404, seven 406, eight 408, and nine 410. In one embodiment, the desired object 402 can be identified in the set of images 412, where the images 412 comprise at least a portion of the planar panorama source imagery. In this embodiment, for example, an object region that comprises at least a portion of the desired object 402 may be identified in the respective frames six 404, seven 406, eight 408, and nine 410.
  • Returning to FIG. 2, at 208, it may be determined whether two or more object regions, respectively comprising a desired object, overlap in a frame. As an illustrative example, FIG. 4B illustrates an example embodiment 450 where one or more portions of one or more techniques described herein may be implemented. In frame number nine 410 two objects 402, 414 may have been identified, for example, and an object region may have been determined for the respective objects 402, 414. As illustrated in the frame number nine 410, the objects 402, 414 (e.g., and therefore the object regions) overlap, where the utility pole 414 may lie in front of the building 402. In the frame 408 of FIG. 4B, the building 402 and utility pole 414 do not overlap each other and, therefore, the respective object regions for the objects 402, 414 may not overlap in the frame 408.
  • Returning to FIG. 2, if at least two identified object regions do not overlap in a frame of the planar panorama source imagery 250 (NO at 208), at least one frame can be selected that comprises a desired object, at 210. In one embodiment, at least one frame may be selected that comprises more than one desired object, where the object regions for the respective desired objects do not overlap (e.g., frame eight 408 of FIG. 4B). In another embodiment, a first frame may be selected, where the first frame comprises a first object region comprising a first desired object; and a second frame may be selected, where the second frame comprises a second object region comprising a second desired object. In this way, for example, if at least a portion of the planar panorama source imagery comprises more than one desired object, merely those frames that comprise a non-overlapping view of the respective objects can be selected.
  • If at least two identified object regions do overlap in a frame (e.g., frame nine 410 of FIG. 4B) of the planar panorama source imagery 250 (YES at 208), at least one frame can be selected from the planar panorama source imagery 250 that comprises a prioritized desired object. In one embodiment, a first desired object can be prioritized over a second desired object, such that the first desired object may be selected for inclusion in resulting planar panorama imagery over the second desired object (e.g., and/or an object may be excluded from resulting planar panorama imagery). In this embodiment, a first object region, comprising the first desired object, can be prioritized over a second object region comprising the second desired object (e.g., and/or a desired object may be prioritized over a non-desired object, and/or a first non-desired object may be prioritized over a second non-desired object, etc.), based on the assigned prioritizations. As one example, a building and/or building sign may be assigned a higher prioritization than a utility pole, due to a likelihood of a user desiring to view the building and/or building sign in the resulting planar panorama imagery rather than a utility pole, for example.
  • As an illustrative example, in FIG. 4B, the frame number nine 410 may comprise overlapping object regions for the object 402, a building, and object 414, a utility pole. In this example, the frame number seven 406 may merely comprise the object region for the building object 402, based on a viewing angle 406A from which the image was captured. That is, for example, a viewing angle 410A for the frame number nine 410 may have placed the utility pole object 414 in front of the building object 402, based on a foreground distance between the utility pole and building from the viewing point 410A. In one example, the viewing angle 406A may provide for a view of the building, unobscured by the utility pole. Therefore, in this embodiment, for example, the frame number seven 406 may be selected, as it merely comprises the desired object 402, which is prioritized over the utility pole, and frame number seven 406 comprises the building 402 without the utility pole 414 overlapping.
  • In one embodiment, a frame that is selected from the one or more frames, such as to join with the one or more portions of the planar panorama source imagery, can comprise an object region that comprises an entirety of the desired object. As an illustrative example, in FIG. 4B, the frame number six 404 comprises merely a portion of the building object 402, which may be a result of a viewing angle 404A from which the image was captured. However, the frame seven 406 (e.g., and eight 408, and nine 410) comprises and entirety of the building object 402. In this embodiment, the frame seven 406 may be selected to join with the one or more portions of the planar panorama source imagery over the frame six 404.
  • In this way, for example, the entire desired object can be included (e.g., not just a portion of the object) in any resulting planar panorama imagery. As one example, a desired object may comprise a business sign (e.g., indicating a name and/or logo for a business). It may be desirable to include the names of businesses in street level, human view imagery, for example, so that a user can identify a desired business on a map (e.g., which may otherwise not be visible if merely a portion of a business sign was included). Further, in one embodiment, by selecting merely one frame that comprises the desired object in its entirety, for example, when frames are joined with the planar panorama source imagery that do not comprise the desired object, merely one instance of the desired object may be included in resulting imagery (e.g., and not two instances of the building sign and/or additional portions of the building).
  • Returning to FIG. 2, at 214, the one or more selected frames, comprising the desired object(s), can be joined with one or more portions of the planar panorama source imagery that do not comprise the desired object(s). In one embodiment, the joining can result in planar panorama imagery 252 that comprises one or more desired objects. As one example, a frame that comprises a business sign and/or logo can be joined with a first portion of the planar panorama source imagery at a first side of the frame, and joined with a second portion of the planar panorama source imagery at a second side of the frame.
  • As an illustrative example, in FIGS. 4A and 4B, the frame seven 406 may be selected for joining with the planar panorama source imagery, as it comprises the desired object 402. In one example, neighboring frames (e.g., six 404, eight 408 and nine 410) may not be selected for joining to the source imagery, as they comprise an instance of the desired object 402 (e.g., or a portion thereof), which can result in an undesirable “doubling” affect of the object in the planar panorama imagery. In this example, a left portion of the resulting planar panorama imagery may comprise the frames 412 from five to one of the planar panorama source imagery, and a right portion of the resulting planar panorama imagery may comprise the frames 412 from ten to sixteen of the planar panorama source imagery. Further, in this example, because there is some overlap between the frames 412, details of the planar panorama source imagery may not be lost by omitting some frames from the resulting planar panorama imagery.
  • It may be appreciated that, in one embodiment, frames need not be dropped (e.g., not used) merely because they contain a desired object which has already been captured from another frame and/or they contain a non-desired object. Rather, remaining portions (e.g., regions) of the frame can still be used. For example, the remainder of a frame is still usable where an object region of that frame comprises an object that has already been obtained from a different frame (e.g., merely the object region of that frame would not be used, such as to avoid “doubling”, for example). Similarly, a negative region of a frame comprising a non-desired object may be excised from that frame leaving the remainder of the frame usable. For example, regions of frames six 404 and/or seven 406 that do not comprise the house may still be usable where the house has already been captured from another frame, such as frame eight 408, for example. Similarly, regions of frame eight 408 that do not comprise the telephone pole (non-desired object) may usable for joining with other frames and/or regions of frames, for example. It may be appreciated that such “region level” granularity is applicable to one or more of the embodiments (e.g., method(s), system(s), etc.) provided herein.
  • Further, as an example, the planar panorama imagery (e.g., 252 of FIG. 2) resulting from joining the one or more frames with the one or more portions of the planar panorama source imagery (e.g., 250 of FIG. 2), may comprise a plurality of desired objects, which have respectively been detected by a corresponding desired object detector, and for which a corresponding object region was identified. Additionally, as one example, based on a prioritization of respective desired objects, one or more frames may be selected that comprise a view of some, but not all of the desired objects, and/or a frame may comprise more than one desired object.
  • FIG. 3 is a flow diagram illustrating an example embodiment 300 where one or more portions of one or more techniques described herein may be implemented. At 302, planar panorama source imagery 350 (e.g., 250 from FIG. 2) can be input to one or more negative object detectors. In one embodiment, a negative object detector may be used to identify a non-desired object in the planar panorama source imagery 350. As one example, a non-desired object can comprise an object that may reduce a user experience when the user views resulting planar panorama imagery 352, such as an object that obscures a view of a desired object (e.g., a tree in front of a sign indicating a name of a business), people present on a sidewalk, cars, utility structures, etc.
  • At 304 in the example embodiment 300, one or more non-desired objects can be identified in the planar panorama source imagery 350, for example, using the one or more negative object detectors. As an example, users typically view a planar panorama in conjunction with a mapping service that indicates a particular location, so that the user may become familiar with the environment at the location, or to browse various locations for desired businesses and/or landmarks. In one example, non-desired objects may be identified, which can be at least partially removed from the resulting planar panorama imagery viewed by the user, thereby improving the user experience. At 306, a negative object region can be identified in the one or more negative object frames, of the planar panorama source imagery 350, for the respective one or more non-desired objects, where the negative object region can comprise at least a portion of the non-desired object.
  • At 308, it may be determined whether at least one frame is available in the planar panorama source imagery 350 that does not comprise the negative object region. As an illustrative example, in FIG. 4B, the non-desired object may comprise a utility pole 414 that partially obscures a view of a desired object 402, the building, in frame number 410. In the example 450, frame number seven 406 comprises a view of the desired object 402, but does not comprise the non-desired object 414.
  • Returning to FIG. 3, if at least one frame is available that does not comprise the negative object region (YES at 308), the at least one frame without the non-desired object can be selected (e.g., select a non-negative frame) for joining with the planar panorama source imagery 350, at 310. If at least one frame is not available that does not comprise the negative object region (NO at 308), at least one negative frame that comprises the negative object region may be selected, for example, where the negative object region is arranged in a way that allows it to be removed with little to no effect one or more desired objects in the negative frame. For example, a negative frame may comprise the negative object region in such a way that it may be removed (e.g., cut out, covered, stitched over with another frame) from the frame without removing at least a portion of a desired object. As an illustrative example, in FIG. 4B, frame number eight 408 comprises a view of the desired object 402 and the non-desired object 414. In this example, the negative object region comprising the non-desired object 414 may be removed without removing a portion of the desired object 402.
  • Returning to FIG. 3, at 314, the one or more portions of the planar panorama source imagery can be joined with the one or more frames, that may not comprise the non-desired object(s), resulting in the planar panorama imagery 352 not comprising the non-desired object, and comprising one or more desired objects. In one embodiment, a non-negative frame that does not comprise the non-desired object, can be joined with the one or more portions of the planar panorama source imagery 350, resulting in the planar panorama imagery 352 not comprising the non-desired object (e.g., via path 310). In one embodiment, the negative object region can be cut from at least one of the negative frames selected, and the cut negative frame can be joined with the one or more portions of the planar panorama source imagery 350, resulting in the planar panorama imagery 352 not comprising the non-desired object (e.g., via path 312). It may be appreciated that aspects illustrated with respect to FIG. 2 and aspects illustrated with respect to FIG. 3 may be combined such that planar panorama source imagery comprising both desired objects and non-desired objects may be used to produce planar panorama imagery comprising a desired object but not comprising a non-desired object, for example.
  • A system may be devised that can mitigate partial or complete loss, and/or distortion of interesting objects in a planar panorama of images, such as a human scale view of a street side panorama. Objects of interest that may be distorted due to misalignment of neighboring images may be appropriately meshed. Further, those objects that may obscure an interesting object or are of a distracting interest in the planar panorama, such as people, moving cars, trees or poles, etc. may be removed and/or meshed in a way that provides a desired view. Resulting planar panorama imagery may provide an improved view of desired objects, where the influence of non-desired objects may be mitigated.
  • FIG. 5 is a component diagram illustrating an exemplary system 500 for generating planar panorama imagery. In the exemplary system 500, a computer-based processor 502, configured to process data for the system 500, is operably coupled with a desired object detection component 504. The desired object detection component 504 is configured to identify a desired object in one or more frames 552 of planar panorama source imagery 550. A desired object can comprise an image of an object, comprised in image data, for example, which may be of interest to a user of resulting planar panorama imagery 554 (e.g., used in association with a mapping application to view street-level, human-scale imagery of a desired location).
  • In the exemplary system 500, an image joining component 506 is operably coupled with the desired object detection component 504. The image joining component 506 is configured to join one or more portions (e.g., regions) of the planar panorama source imagery 550, which do not comprise the desired object, with the one or more frames 552 comprising the desired object, resulting in the planar panorama imagery 554. For example, a frame 552 of the planar panorama source imagery 550, comprising the desired object, may be joined with a first appropriate portion of the planar panorama source imagery 550, not comprising the desired object, at a first side of the frame; and may be joined with a second appropriate portion of the planar panorama source imagery 550, not comprising the desired object, at a second side of the frame, by the image joining component 506. In this way, for example, merely a desired view of the desired object (e.g., not distorted, and/or an entirety of the object) is comprised in the resulting planar panorama imagery 552, and/or merely a single view (e.g., not multiple instances and/or partial artifacts) is comprised in the resulting planar panorama imagery 552.
  • FIG. 6 is a component diagram illustrating an example embodiment 600 where one or more systems described herein may be implemented. In this example 600, an extension of FIG. 5 is provided and thus description of elements, components, etc. described with respect to FIG. 5 may not be repeated for simplicity. In the example embodiment 600, a frame selection component 612 can be configured to select at least a first frame 654 from the one or more frames 652 for the image joining component 506. The first frame 654 can comprise an object region 658 comprising a desired object.
  • In one embodiment, the desired object detection component 504 can be configured to identify one or more object regions 658 in the one or more frames 652 of the planar panorama source imagery 650. In this embodiment, the one or more object regions 658 may respectively comprise at least one desired object. As one example, the desired object detection component 504 may identify a plurality of frames 652 that comprise the desired object, and identify an object region 658 in the respective plurality of frames 652. In this example, the frame selection component 612 may identify a frame 654, from the plurality of frames 652, which comprises a desired view of the desired object. In this example, this selected frame 654, comprising the object region 658 with the desired view of the desired object, may be provided to the image joining component 506.
  • In the example embodiment 600, the desired object detection component 504 can comprise a plurality of object detectors 610, where the respective object detectors 610 can be configured to detect merely one type of object in the planar panorama source imagery 650. As an example, the plurality of detectors 610 may comprise a building detector, a person detector, landscape detector (e.g., trees, bushes, etc.), vehicle detector (e.g., stopped and/or moving), sign detector (e.g., text detector for business and/or street signs), utility object detector (e.g., utility pole, stop light, power lines, etc.), road section detector (e.g., medians), business logo detector, and/or building entryway detector (e.g., business entries, doors, etc.), etc. In one embodiment, respective detectors may be activated and/or deactivated to provide a desired detection result.
  • In one embodiment, the plurality of object detectors 610 may comprise one or more positive object detectors that can be configured to detect a desired object, where the desired object may be intended to be included in the resulting planar panorama imagery 656. For example, a desired object can comprise things that a user of the resulting planar panorama imagery 656 may find useful, such as building signs, buildings, street signs, landmarks, street features, etc. In this example, detectors 610 that detect these desired objects may comprise positive object detectors.
  • In one embodiment, the plurality of object detectors 610 may comprise one or more negative object detectors that can be configured to detect a non-desired object, where the non-desired object may comprise an object that is not intended to be included in the resulting planar panorama imagery 656. For example, a non-desired object may comprise an item in an image that can reduce the user experience, such as objects that obscure a view of a desired object (e.g., trees, utility poles, etc.), objects that distract from a view of a location (e.g., moving cars, people, etc.), and/or distorted objects, etc. In this example, a detector 610 that detects a non-desired object may comprise negative object detector.
  • In the example embodiment 600, an object prioritization component 614 can be configured to prioritize selection of a first desired object over selection of a second desired object for inclusion in the resulting planar panorama imagery 656. The prioritization can be based at least upon a first desired prioritization associated with the first desired object and a second desired prioritization associated with the second desired object. As an example, certain objects may have a higher priority for a user of the resulting planar panorama imagery 656, such as business signs/logos that may identify a particular business, over other objects, such as landscapes features. In one embodiment, object prioritization may be assigned based on user input, or may be assigned using a default setting (e.g., programmatically). In this way, for example, the object prioritization component 614 can provide for selection of a frame that may comprise a desired view of a higher priority object over selection of a frame that comprises a view of a lower priority object.
  • In one embodiment, the image joining component 506 can be configured to generate the resulting planar panorama imagery 656, where the resulting planar panorama imagery comprises a first desired object if the first desired object is prioritized over a second object and a first object region, comprising the first desired object, overlaps a second object region, comprising the second desired object, in the planar panorama source imagery 650. That is, for example, if the first and second desired objects overlap in one or more frames 652 of the planar panorama source imagery 650, the object prioritization component 614 can provide for selecting a frame that merely comprises the first object, and/or a frame that allows for cutting out the second object without affecting a view of the first object. Further, the object prioritization component 614 can provide for selecting a frame that comprises a view of both the first and second object, where they do not overlap, if available (e.g., 408 of FIG. 4B).
  • In one embodiment, the frame selection component 612 may be configured to select one or more of the one or more frames 652 for the image joining component 506, by selecting at least a first frame 654 that comprises at least a desired view of the desired object (e.g., unobscured, undistorted, an entirety of the object, etc.). In one embodiment, the frame selection component 612 can select at least a second frame 654, from the one or more frames, where the second frame does not comprise a non-desired object, for example, effectively removing the non-desired object from the resulting planar panorama imagery 656.
  • In one embodiment, the frame selection component 612 can select at least a third frame 654, from the one or more frames 652, where the third frame can comprise a desired view of the non-desired object (e.g., a view that does not obscure a desired object). Further, in this embodiment, the desired view of the non-desired object may allow for the non-desired object to be cut from the third frame, resulting in desired planar panorama imagery 656, for example, without the non-desired object.
  • Still another embodiment involves a computer-readable medium comprising processor-executable instructions configured to implement one or more of the techniques presented herein. An exemplary computer-readable medium that may be devised in these ways is illustrated in FIG. 7, wherein the implementation 700 comprises a computer-readable medium 708 (e.g., a CD-R, DVD-R, or a platter of a hard disk drive), on which is encoded computer-readable data 706. This computer-readable data 706 in turn comprises a set of computer instructions 704 configured to operate according to one or more of the principles set forth herein. In one such embodiment 702, the processor-executable instructions 704 may be configured to perform a method, such as at least some of the exemplary method 100 of FIG. 1, for example. In another such embodiment, the processor-executable instructions 704 may be configured to implement a system, such as at least some of the exemplary system 500 of FIG. 5, for example. Many such computer-readable media may be devised by those of ordinary skill in the art that are configured to operate in accordance with the techniques presented herein.
  • Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.
  • As used in this application, the terms “component,” “module,” “system”, “interface”, and the like are generally intended to refer to a computer-related entity, either hardware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a controller and the controller can be a component. One or more components may reside within a process and/or thread of execution and a component may be localized on one computer and/or distributed between two or more computers.
  • Furthermore, the claimed subject matter may be implemented as a method, apparatus, or article of manufacture using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof to control a computer to implement the disclosed subject matter. The term “article of manufacture” as used herein is intended to encompass a computer program accessible from any computer-readable device, carrier, or media. Of course, those skilled in the art will recognize many modifications may be made to this configuration without departing from the scope or spirit of the claimed subject matter.
  • FIG. 8 and the following discussion provide a brief, general description of a suitable computing environment to implement embodiments of one or more of the provisions set forth herein. The operating environment of FIG. 8 is only one example of a suitable operating environment and is not intended to suggest any limitation as to the scope of use or functionality of the operating environment. Example computing devices include, but are not limited to, personal computers, server computers, hand-held or laptop devices, mobile devices (such as mobile phones, Personal Digital Assistants (PDAs), media players, and the like), multiprocessor systems, consumer electronics, mini computers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
  • Although not required, embodiments are described in the general context of “computer readable instructions” being executed by one or more computing devices. Computer readable instructions may be distributed via computer readable media (discussed below). Computer readable instructions may be implemented as program modules, such as functions, objects, Application Programming Interfaces (APIs), data structures, and the like, that perform particular tasks or implement particular abstract data types. Typically, the functionality of the computer readable instructions may be combined or distributed as desired in various environments.
  • FIG. 8 illustrates an example of a system 800 comprising a computing device 812 configured to implement one or more embodiments provided herein. In one configuration, computing device 812 includes at least one processing unit 816 and memory 818. Depending on the exact configuration and type of computing device, memory 818 may be volatile (such as RAM, for example), non-volatile (such as ROM, flash memory, etc., for example) or some combination of the two. This configuration is illustrated in FIG. 8 by dashed line 814.
  • In other embodiments, device 812 may include additional features and/or functionality. For example, device 812 may also include additional storage (e.g., removable and/or non-removable) including, but not limited to, magnetic storage, optical storage, and the like. Such additional storage is illustrated in FIG. 8 by storage 820. In one embodiment, computer readable instructions to implement one or more embodiments provided herein may be in storage 820. Storage 820 may also store other computer readable instructions to implement an operating system, an application program, and the like. Computer readable instructions may be loaded in memory 818 for execution by processing unit 816, for example.
  • The term “computer readable media” as used herein includes computer storage media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions or other data. Memory 818 and storage 820 are examples of computer storage media. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVDs) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by device 812. Any such computer storage media may be part of device 812.
  • Device 812 may also include communication connection(s) 826 that allows device 812 to communicate with other devices. Communication connection(s) 826 may include, but is not limited to, a modem, a Network Interface Card (NIC), an integrated network interface, a radio frequency transmitter/receiver, an infrared port, a USB connection, or other interfaces for connecting computing device 812 to other computing devices. Communication connection(s) 826 may include a wired connection or a wireless connection. Communication connection(s) 826 may transmit and/or receive communication media.
  • The term “computer readable media” may include communication media. Communication media typically embodies computer readable instructions or other data in a “modulated data signal” such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” may include a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.
  • Device 812 may include input device(s) 824 such as keyboard, mouse, pen, voice input device, touch input device, infrared cameras, video input devices, and/or any other input device. Output device(s) 822 such as one or more displays, speakers, printers, and/or any other output device may also be included in device 812. Input device(s) 824 and output device(s) 822 may be connected to device 812 via a wired connection, wireless connection, or any combination thereof. In one embodiment, an input device or an output device from another computing device may be used as input device(s) 824 or output device(s) 822 for computing device 812.
  • Components of computing device 812 may be connected by various interconnects, such as a bus. Such interconnects may include a Peripheral Component Interconnect (PCI), such as PCI Express, a Universal Serial Bus (USB), firewire (IEEE 1394), an optical bus structure, and the like. In another embodiment, components of computing device 812 may be interconnected by a network. For example, memory 818 may be comprised of multiple physical memory units located in different physical locations interconnected by a network.
  • Those skilled in the art will realize that storage devices utilized to store computer readable instructions may be distributed across a network. For example, a computing device 830 accessible via network 828 may store computer readable instructions to implement one or more embodiments provided herein. Computing device 812 may access computing device 830 and download a part or all of the computer readable instructions for execution. Alternatively, computing device 812 may download pieces of the computer readable instructions, as needed, or some instructions may be executed at computing device 812 and some at computing device 830.
  • Various operations of embodiments are provided herein. In one embodiment, one or more of the operations described may constitute computer readable instructions stored on one or more computer readable media, which if executed by a computing device, will cause the computing device to perform the operations described. The order in which some or all of the operations are described should not be construed as to imply that these operations are necessarily order dependent. Alternative ordering will be appreciated by one skilled in the art having the benefit of this description. Further, it will be understood that not all operations are necessarily present in each embodiment provided herein.
  • Moreover, the word “exemplary” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as advantageous over other aspects or designs. Rather, use of the word exemplary is intended to present concepts in a concrete fashion. As used in this application, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or”. That is, unless specified otherwise, or clear from context, “X employs A or B” is intended to mean any of the natural inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, then “X employs A or B” is satisfied under any of the foregoing instances. Further, at least one of A and B and/or the like generally means A or B or both A and B. In addition, the articles “a” and “an” as used in this application and the appended claims may generally be construed to mean “one or more” unless specified otherwise or clear from context to be directed to a singular form.
  • Also, although the disclosure has been shown and described with respect to one or more implementations, equivalent alterations and modifications will occur to others skilled in the art based upon a reading and understanding of this specification and the annexed drawings. The disclosure includes all such modifications and alterations and is limited only by the scope of the following claims. In particular regard to the various functions performed by the above described components (e.g., elements, resources, etc.), the terms used to describe such components are intended to correspond, unless otherwise indicated, to any component which performs the specified function of the described component (e.g., that is functionally equivalent), even though not structurally equivalent to the disclosed structure which performs the function in the herein illustrated exemplary implementations of the disclosure. In addition, while a particular feature of the disclosure may have been disclosed with respect to only one of several implementations, such feature may be combined with one or more other features of the other implementations as may be desired and advantageous for any given or particular application. Furthermore, to the extent that the terms “includes”, “having”, “has”, “with”, or variants thereof are used in either the detailed description or the claims, such terms are intended to be inclusive in a manner similar to the term “comprising.”

Claims (20)

What is claimed is:
1. A computer-based method for generating planar panorama imagery, comprising:
identifying an object region in one or more frames of planar panorama source imagery, the object region comprising a desired object; and
joining one or more portions of the planar panorama source imagery, not comprising the desired object, with the one or more frames comprising the object region, to generate the planar panorama imagery comprising the desired object, at least a portion of the method performed by a processing unit.
2. The method of claim 1, comprising identifying the desired object in the planar panorama source imagery using an object detector.
3. The method of claim 1, comprising selecting merely a first frame from the one or more frames to join with the one or more portions of the planar panorama source imagery.
4. The method of claim 1, comprising selecting a first frame from the one or more frames to join with the one or more portions of the planar panorama source imagery, the first frame comprising an entirety of the desired object.
5. The method of claim 1, comprising identifying the desired object in the planar panorama source imagery, comprising:
identifying a first desired object in the planar panorama source imagery; and
identifying a second desired object in the planar panorama source imagery.
6. The method of claim 5, identifying the object region comprising:
identifying a first object region comprising the first desired object in one or more first frames of the planar panorama source imagery; and
identifying a second object region comprising the second desired object in one or more second frames of the planar panorama source imagery.
7. The method of claim 6, comprising prioritizing the first object region over the second object region based on a first priority associated with the first object region and a second priority associated with the second object region.
8. The method of claim 6, the joining resulting in:
the planar panorama imagery comprising merely the first desired object if the first desired object is prioritized over the second desired object and the first object region and the second object region overlap in the planar panorama imagery; else
the planar panorama imagery comprising the first desired object and the second desired object.
9. The method of claim 1, comprising identifying a negative object region in one or more negative frames of the planar panorama source imagery, the negative object region comprising a non-desired object.
10. The method of claim 9, comprising joining the one or more portions of the planar panorama source imagery with the one or more frames, resulting in the planar panorama imagery not comprising the non-desired object.
11. The method of claim 9, comprising one or more of:
selecting a first frame that does not comprise the non-desired object, from the one or more frames to join with the one or more portions of the planar panorama source imagery; and
cutting the negative object region from at least a second frame of the one or more negative frames, and joining the one or more portions of the planar panorama source imagery with at least the second frame.
12. A system for generating planar panorama imagery, comprising:
a computer-based processor configured to process data for the system;
a desired object detection component, operably coupled with the processor, configured to identify a desired object in one or more frames of planar panorama source imagery; and
an image joining component, operably coupled with the desired object detection component, configured to join one or more portions of the planar panorama source imagery, not comprising the desired object, with the one or more frames comprising the desired object, resulting in the planar panorama imagery.
13. The system of claim 12, the desired object detection component configured to identify an object region in the one or more frames of the planar panorama source imagery, the object region comprising the desired object.
14. The system of claim 13, comprising a frame selection component configured to select at least a first frame from the one or more frames for the image joining component, the first frame comprising the object region comprising the desired object.
15. The system of claim 12, the desired object detection component comprising a plurality of object detectors, the respective object detectors configured to identify merely one type of object in the planar panorama source imagery.
16. The system of claim 15, the plurality of object detectors comprising one or more of:
one or more positive object detectors configured to detect the desired object, the desired object intended to be included in the resulting planar panorama imagery; and
one or more negative object detectors configured to detect a non-desired object, the non-desired object intended not to be included in the resulting planar panorama imagery.
17. The system of claim 12, comprising an object prioritization component configured to prioritize selection of a first desired object over selection of a second desired object for inclusion in the resulting planar panorama imagery, based at least upon a first desired prioritization associated with the first desired object and a second desired prioritization associated with the second desired object.
18. The system of claim 17, the image joining component configured to generate the resulting planar panorama imagery, the resulting planar panorama imagery comprising:
the first desired object if the first desired object is prioritized over the second desired object and a first object region, comprising the first desired object, overlaps a second object region, comprising the second desired object, in the planar panorama source imagery; else
the first desired object and the second desired object.
19. The system of claim 12, comprising a frame selection component configured to select one or more of the one or more frames for the image joining component, comprising one or more of:
selecting at least a first frame, from the one or more frames, the first frame comprising at least a desired view of the desired object;
selecting at least a second frame, from the one or more frames, the second frame not comprising a non-desired object; and
selecting at least a third frame, from the one or more frames, the third frame comprising a desired view of the non-desired object, the desired view of the non-desired object resulting in the desired planar panorama imagery when the non-desired object is cut from the third frame.
20. A computer readable medium comprising computer executable instructions that when executed via a processor perform a method for generating planar panorama imagery, comprising:
identifying an object region in one or more frames of planar panorama source imagery, the object region comprising a desired object;
identifying a negative object region in one or more negative frames of the planar panorama source imagery, the negative object region comprising a non-desired object; and
joining one or more portions of the planar panorama source imagery with at least some of the one or more frames and at least some of the one or more negative frames in a manner that generates the planar panorama imagery comprising the desired object and not comprising the non-desired object.
US13/286,756 2011-11-01 2011-11-01 Planar panorama imagery generation Abandoned US20130106990A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US13/286,756 US20130106990A1 (en) 2011-11-01 2011-11-01 Planar panorama imagery generation

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US13/286,756 US20130106990A1 (en) 2011-11-01 2011-11-01 Planar panorama imagery generation
US14/714,768 US10038842B2 (en) 2011-11-01 2015-05-18 Planar panorama imagery generation

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US14/714,768 Continuation US10038842B2 (en) 2011-11-01 2015-05-18 Planar panorama imagery generation

Publications (1)

Publication Number Publication Date
US20130106990A1 true US20130106990A1 (en) 2013-05-02

Family

ID=48172005

Family Applications (2)

Application Number Title Priority Date Filing Date
US13/286,756 Abandoned US20130106990A1 (en) 2011-11-01 2011-11-01 Planar panorama imagery generation
US14/714,768 Active 2032-12-03 US10038842B2 (en) 2011-11-01 2015-05-18 Planar panorama imagery generation

Family Applications After (1)

Application Number Title Priority Date Filing Date
US14/714,768 Active 2032-12-03 US10038842B2 (en) 2011-11-01 2015-05-18 Planar panorama imagery generation

Country Status (1)

Country Link
US (2) US20130106990A1 (en)

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8610741B2 (en) 2009-06-02 2013-12-17 Microsoft Corporation Rendering aligned perspective images
US9106872B2 (en) * 2008-10-27 2015-08-11 Sony Corporation Image processing apparatus, image processing method, and program
US20150302633A1 (en) * 2014-04-22 2015-10-22 Google Inc. Selecting time-distributed panoramic images for display
US9324184B2 (en) 2011-12-14 2016-04-26 Microsoft Technology Licensing, Llc Image three-dimensional (3D) modeling
CN105578035A (en) * 2015-12-10 2016-05-11 联想(北京)有限公司 Image processing method and electronic device
US9406153B2 (en) 2011-12-14 2016-08-02 Microsoft Technology Licensing, Llc Point of interest (POI) data positioning in image
USD780211S1 (en) 2014-04-22 2017-02-28 Google Inc. Display screen with graphical user interface or portion thereof
USD780210S1 (en) 2014-04-22 2017-02-28 Google Inc. Display screen with graphical user interface or portion thereof
USD780797S1 (en) 2014-04-22 2017-03-07 Google Inc. Display screen with graphical user interface or portion thereof
US20170155836A1 (en) * 2015-11-30 2017-06-01 Inventec (Pudong) Technology Corporation System and method for shoot integration
US9934222B2 (en) 2014-04-22 2018-04-03 Google Llc Providing a thumbnail image that follows a main image
US10008021B2 (en) 2011-12-14 2018-06-26 Microsoft Technology Licensing, Llc Parallax compensation
US10038842B2 (en) 2011-11-01 2018-07-31 Microsoft Technology Licensing, Llc Planar panorama imagery generation
US10051180B1 (en) * 2016-03-04 2018-08-14 Scott Zhihao Chen Method and system for removing an obstructing object in a panoramic image
USD868093S1 (en) 2018-08-20 2019-11-26 Google Llc Display screen with graphical user interface or portion thereof

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015085589A1 (en) 2013-12-13 2015-06-18 华为终端有限公司 Method and terminal for acquiring panoramic image
US10460512B2 (en) * 2017-11-07 2019-10-29 Microsoft Technology Licensing, Llc 3D skeletonization using truncated epipolar lines

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070076920A1 (en) * 2005-10-04 2007-04-05 Microsoft Corporation Street side maps and paths
US20090021576A1 (en) * 2007-07-18 2009-01-22 Samsung Electronics Co., Ltd. Panoramic image production

Family Cites Families (105)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5138460A (en) 1987-08-20 1992-08-11 Canon Kabushiki Kaisha Apparatus for forming composite images
JP2883265B2 (en) 1993-09-24 1999-04-19 キヤノン株式会社 Image processing apparatus
US5473364A (en) 1994-06-03 1995-12-05 David Sarnoff Research Center, Inc. Video technique for indicating moving objects from a movable platform
US6205253B1 (en) 1996-08-19 2001-03-20 Harris Corporation Method and apparatus for transmitting and utilizing analog encoded information
US6075905A (en) 1996-07-17 2000-06-13 Sarnoff Corporation Method and apparatus for mosaic image construction
US6597818B2 (en) 1997-05-09 2003-07-22 Sarnoff Corporation Method and apparatus for performing geo-spatial registration of imagery
US6064399A (en) 1998-04-03 2000-05-16 Mgi Software Corporation Method and system for panel alignment in panoramas
EP1048167B1 (en) 1998-09-17 2009-01-07 Yissum Research Development Company Of The Hebrew University Of Jerusalem System and method for generating and displaying panoramic images and movies
US6415227B1 (en) 1999-04-21 2002-07-02 American Gnc Corporation Enhanced global positioning system and map navigation process
DE19920709A1 (en) 1999-05-05 2000-11-16 Siemens Ag A method for obtaining a three-dimensional map display navigation system and
AU7939300A (en) 1999-08-20 2001-03-19 Emaki, Inc. System and method for rectified mosaicing of images recorded by a moving camera
JP2002258740A (en) 2001-03-02 2002-09-11 Mixed Reality Systems Laboratory Inc Method and device for recording picture and method and device for reproducing picture
US7194112B2 (en) 2001-03-12 2007-03-20 Eastman Kodak Company Three dimensional spatial panorama formation with a range imaging system
US6831643B2 (en) 2001-04-16 2004-12-14 Lucent Technologies Inc. Method and system for reconstructing 3D interactive walkthroughs of real-world environments
US6872181B2 (en) 2001-04-25 2005-03-29 Siemens Medical Solutions Usa, Inc. Compound image display system and method
US6580393B2 (en) 2001-06-25 2003-06-17 Harris Corporation System and method for determining the location of a transmitter using passive reflectors or refractors as proxy receivers and using database querying
JP4235393B2 (en) 2002-03-11 2009-03-11 富士フイルム株式会社 Index image display control device
US8427538B2 (en) 2004-04-30 2013-04-23 Oncam Grandeye Multiple view and multiple object processing in wide-angle video camera
CN100414566C (en) 2003-06-19 2008-08-27 邓兴峰;张正辉;游建军 Panoramic reconstruction method of three dimensional image from two dimensional image
US20070103544A1 (en) 2004-08-26 2007-05-10 Naofumi Nakazawa Panorama image creation device and panorama image imaging device
US7747067B2 (en) 2003-10-08 2010-06-29 Purdue Research Foundation System and method for three dimensional modeling
CN1641702A (en) 2004-01-13 2005-07-20 邓兴峰 Method for designing stereo image from planar image
US7437005B2 (en) 2004-02-17 2008-10-14 Microsoft Corporation Rapid visual sorting of digital files and data
DE602004014901D1 (en) 2004-04-29 2008-08-21 Mitsubishi Electric Corp Adaptive quantization of a depth map
US7298548B2 (en) 2004-08-16 2007-11-20 International Electronic Machines Corp. Multi-directional viewing and imaging
AU2004231206A1 (en) 2004-11-19 2006-06-08 Canon Kabushiki Kaisha Displaying a plurality of images in a stack arrangement
US7756358B2 (en) 2004-11-30 2010-07-13 Hewlett-Packard Development Company, L.P. System and method of aligning images
US20060271286A1 (en) 2005-05-27 2006-11-30 Outland Research, Llc Image-enhanced vehicle navigation systems and methods
US7760269B2 (en) 2005-08-22 2010-07-20 Hewlett-Packard Development Company, L.P. Method and apparatus for sizing an image on a display
US7499586B2 (en) 2005-10-04 2009-03-03 Microsoft Corporation Photographing big things
US8085990B2 (en) 2006-07-28 2011-12-27 Microsoft Corporation Hybrid maps with embedded street-side images
US7580076B2 (en) 2005-10-31 2009-08-25 Hewlett-Packard Development Company, L.P. Devices and methods for calculating pixel values representative of a scene
JP4740723B2 (en) 2005-11-28 2011-08-03 富士通株式会社 Image analysis program, recording medium storing the program, image analysis apparatus, and image analysis method
KR100653200B1 (en) 2006-01-09 2006-11-27 삼성전자주식회사 Method and apparatus for providing panoramic view with geometry correction
US7823068B2 (en) 2006-02-28 2010-10-26 Mark Anthony Ogle Cowtan Internet-based, dual-paned virtual tour presentation system with orientational capabilities and versatile tabbed menu-driven area for multi-media content delivery
US20070204014A1 (en) 2006-02-28 2007-08-30 John Wesley Greer Mobile Webcasting of Multimedia and Geographic Position for a Real-Time Web Log
US8269822B2 (en) 2007-04-03 2012-09-18 Sony Computer Entertainment America, LLC Display viewing system and methods for optimizing display view based on active tracking
WO2007124664A1 (en) 2006-04-29 2007-11-08 Shanghai Jietu Software Co., Ltd. Apparatus and method for collecting panorama graph with location information and method for building, annotating and switching panorama electric map service
CN100489851C (en) 2006-04-29 2009-05-20 上海杰图软件技术有限公司 Method for establishing panorama electronic map service
US8477636B2 (en) 2006-07-28 2013-07-02 Telefonaktiebolaget L M Ericsson (Publ) Method and system for providing updates on access network capability in an IP multimedia system network
US20080043020A1 (en) 2006-08-18 2008-02-21 Microsoft Corporation User interface for viewing street side imagery
US8453060B2 (en) 2006-08-25 2013-05-28 Microsoft Corporation Panoramic ring user interface
KR100790887B1 (en) 2006-09-22 2008-01-02 삼성전자주식회사 Apparatus and method for processing image
US8571787B2 (en) 2006-12-06 2013-10-29 Sony Corporation Dynamic routing
US8368720B2 (en) 2006-12-13 2013-02-05 Adobe Systems Incorporated Method and apparatus for layer-based panorama adjustment and editing
US8593506B2 (en) 2007-03-15 2013-11-26 Yissum Research Development Company Of The Hebrew University Of Jerusalem Method and system for forming a panoramic image of a scene having minimal aspect distortion
US20080268876A1 (en) 2007-04-24 2008-10-30 Natasha Gelfand Method, Device, Mobile Terminal, and Computer Program Product for a Point of Interest Based Scheme for Improving Mobile Visual Searching Functionalities
US20080309668A1 (en) 2007-05-17 2008-12-18 Igor Borovikov Image processing method and apparatus
US7990394B2 (en) 2007-05-25 2011-08-02 Google Inc. Viewing and navigating within panoramic images, and applications thereof
US7843451B2 (en) 2007-05-25 2010-11-30 Google Inc. Efficient rendering of panoramic images, and applications thereof
RU2009148504A (en) 2007-06-08 2011-07-20 Теле Атлас Б.В. (NL) Method and device for creating a panorama with multiple surveillance points
US8605133B2 (en) 2007-06-27 2013-12-10 University Of Florida Research Foundation, Inc. Display-based interactive simulation with dynamic panorama
CN100589148C (en) 2007-07-06 2010-02-10 浙江大学 Method for implementing automobile driving analog machine facing to disciplinarian
KR101396346B1 (en) 2007-09-21 2014-05-20 삼성전자주식회사 Method and apparatus for creating a 3D image using 2D photograph images
US8174562B2 (en) 2007-11-09 2012-05-08 Honeywell International Inc. Stereo camera having 360 degree field of view
US8490025B2 (en) 2008-02-01 2013-07-16 Gabriel Jakobson Displaying content associated with electronic mapping systems
US8103134B2 (en) 2008-02-20 2012-01-24 Samsung Electronics Co., Ltd. Method and a handheld device for capturing motion
WO2009120303A1 (en) 2008-03-24 2009-10-01 Google Inc. Panoramic images within driving directions
US8724909B2 (en) 2008-06-03 2014-05-13 Kooaba Ag Method and system for generating a pictorial reference database using geographical information
EP2309225A4 (en) 2008-07-01 2014-10-15 Topcon Corp Position measurement method, position measurement device, and program
US8600193B2 (en) 2008-07-16 2013-12-03 Varian Medical Systems, Inc. Image stitching and related method therefor
US9307165B2 (en) 2008-08-08 2016-04-05 Qualcomm Technologies, Inc. In-camera panorama image stitching assistance
CN101666656B (en) 2008-09-05 2012-05-30 宏达国际电子股份有限公司 Method and device for operating the visual field of electronic map
CN101673395B (en) 2008-09-10 2012-09-05 华为终端有限公司 Image mosaic method and image mosaic device
US20100110069A1 (en) 2008-10-31 2010-05-06 Sharp Laboratories Of America, Inc. System for rendering virtual see-through scenes
WO2010052550A2 (en) 2008-11-05 2010-05-14 Easywalk Capital S.A. System and method for creating and broadcasting interactive panoramic walk-through applications
US8493408B2 (en) 2008-11-19 2013-07-23 Apple Inc. Techniques for manipulating panoramas
US20100194851A1 (en) 2009-02-03 2010-08-05 Aricent Inc. Panorama image stitching
US9098926B2 (en) 2009-02-06 2015-08-04 The Hong Kong University Of Science And Technology Generating three-dimensional façade models from images
US9454847B2 (en) 2009-02-24 2016-09-27 Google Inc. System and method of indicating transition between street level images
US20100220893A1 (en) 2009-03-02 2010-09-02 Gwo Giun Lee Method and System of Mono-View Depth Estimation
US8862987B2 (en) 2009-03-31 2014-10-14 Intel Corporation Capture and display of digital images based on related metadata
CN101852620A (en) 2009-04-03 2010-10-06 上海任登信息科技有限公司 Method for displaying points of interest at identical geographic position in electronic map
US20100259610A1 (en) 2009-04-08 2010-10-14 Celsia, Llc Two-Dimensional Display Synced with Real World Object Movement
US8237787B2 (en) 2009-05-02 2012-08-07 Steven J. Hollinger Ball with camera and trajectory control for reconnaissance or recreation
US8610741B2 (en) 2009-06-02 2013-12-17 Microsoft Corporation Rendering aligned perspective images
US8203600B2 (en) 2009-06-04 2012-06-19 Transpacific Image, Llc Projection-type stereoscopic display
CN101923709B (en) 2009-06-16 2013-06-26 日电(中国)有限公司 Image splicing method and equipment
US20110010650A1 (en) 2009-07-09 2011-01-13 Mapquest, Inc. Systems and methods for decluttering electronic map displays
JP5418127B2 (en) * 2009-10-09 2014-02-19 ソニー株式会社 Image processing apparatus and method, and program
US20110141229A1 (en) 2009-12-11 2011-06-16 Fotonation Ireland Limited Panorama imaging using super-resolution
US9766089B2 (en) 2009-12-14 2017-09-19 Nokia Technologies Oy Method and apparatus for correlating and navigating between a live image and a prerecorded panoramic image
JP2011138011A (en) 2009-12-28 2011-07-14 Sony Corp Information processing apparatus, program, information processing method, and map data
CN101841727B (en) 2009-12-30 2011-12-07 中华映管股份有限公司 Image processing method
US8290248B2 (en) 2009-12-31 2012-10-16 Mitsubishi Electric Research Laboratories, Inc. Determining disparity search range in stereo videos
US20110265023A1 (en) 2010-04-21 2011-10-27 Microsoft Corporation Interactive Representation of Clustered Entities
US20110279446A1 (en) 2010-05-16 2011-11-17 Nokia Corporation Method and apparatus for rendering a perspective view of objects and content related thereto for location-based services on mobile device
US20110279445A1 (en) 2010-05-16 2011-11-17 Nokia Corporation Method and apparatus for presenting location-based content
US8640020B2 (en) 2010-06-02 2014-01-28 Microsoft Corporation Adjustable and progressive mobile device street view
JP5541974B2 (en) 2010-06-14 2014-07-09 任天堂株式会社 Image display program, apparatus, system and method
US8315791B2 (en) 2010-06-18 2012-11-20 Nokia Coporation Method and apparatus for providing smart zooming of a geographic representation
US8913083B1 (en) 2010-07-13 2014-12-16 Google Inc. System and method of generating a view for a point of interest
US8913056B2 (en) 2010-08-04 2014-12-16 Apple Inc. Three dimensional user interface effects on a display by using properties of motion
JP5643617B2 (en) 2010-11-18 2014-12-17 任天堂株式会社 Image processing program, image processing apparatus, image processing method, and image processing system
US20120133639A1 (en) 2010-11-30 2012-05-31 Microsoft Corporation Strip panorama
US9762794B2 (en) 2011-05-17 2017-09-12 Apple Inc. Positional sensor-assisted perspective correction for panoramic photography
US8600194B2 (en) 2011-05-17 2013-12-03 Apple Inc. Positional sensor-assisted image registration for panoramic photography
JP5845776B2 (en) 2011-09-27 2016-01-20 セイコーエプソン株式会社 Control device, control method of control device for controlling multifunction device, and program
US20130106990A1 (en) 2011-11-01 2013-05-02 Microsoft Corporation Planar panorama imagery generation
US8995788B2 (en) 2011-12-14 2015-03-31 Microsoft Technology Licensing, Llc Source imagery selection for planar panorama comprising curve
US10008021B2 (en) 2011-12-14 2018-06-26 Microsoft Technology Licensing, Llc Parallax compensation
US9324184B2 (en) 2011-12-14 2016-04-26 Microsoft Technology Licensing, Llc Image three-dimensional (3D) modeling
US9406153B2 (en) 2011-12-14 2016-08-02 Microsoft Technology Licensing, Llc Point of interest (POI) data positioning in image
US9552129B2 (en) 2012-03-23 2017-01-24 Microsoft Technology Licensing, Llc Interactive visual representation of points of interest data
US20150116360A1 (en) 2012-07-17 2015-04-30 Google Inc. Generating A Viewpoint On a Digital Map For a Point of Interest

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070076920A1 (en) * 2005-10-04 2007-04-05 Microsoft Corporation Street side maps and paths
US20090021576A1 (en) * 2007-07-18 2009-01-22 Samsung Electronics Co., Ltd. Panoramic image production

Cited By (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9106872B2 (en) * 2008-10-27 2015-08-11 Sony Corporation Image processing apparatus, image processing method, and program
US8610741B2 (en) 2009-06-02 2013-12-17 Microsoft Corporation Rendering aligned perspective images
US10038842B2 (en) 2011-11-01 2018-07-31 Microsoft Technology Licensing, Llc Planar panorama imagery generation
US10008021B2 (en) 2011-12-14 2018-06-26 Microsoft Technology Licensing, Llc Parallax compensation
US9324184B2 (en) 2011-12-14 2016-04-26 Microsoft Technology Licensing, Llc Image three-dimensional (3D) modeling
US9406153B2 (en) 2011-12-14 2016-08-02 Microsoft Technology Licensing, Llc Point of interest (POI) data positioning in image
USD780795S1 (en) 2014-04-22 2017-03-07 Google Inc. Display screen with graphical user interface or portion thereof
USD780210S1 (en) 2014-04-22 2017-02-28 Google Inc. Display screen with graphical user interface or portion thereof
USD780797S1 (en) 2014-04-22 2017-03-07 Google Inc. Display screen with graphical user interface or portion thereof
USD780796S1 (en) 2014-04-22 2017-03-07 Google Inc. Display screen with graphical user interface or portion thereof
USD780211S1 (en) 2014-04-22 2017-02-28 Google Inc. Display screen with graphical user interface or portion thereof
USD780794S1 (en) 2014-04-22 2017-03-07 Google Inc. Display screen with graphical user interface or portion thereof
USD781337S1 (en) 2014-04-22 2017-03-14 Google Inc. Display screen with graphical user interface or portion thereof
USD830407S1 (en) 2014-04-22 2018-10-09 Google Llc Display screen with graphical user interface or portion thereof
USD835147S1 (en) 2014-04-22 2018-12-04 Google Llc Display screen with graphical user interface or portion thereof
USD791811S1 (en) 2014-04-22 2017-07-11 Google Inc. Display screen with graphical user interface or portion thereof
USD792460S1 (en) 2014-04-22 2017-07-18 Google Inc. Display screen with graphical user interface or portion thereof
US9934222B2 (en) 2014-04-22 2018-04-03 Google Llc Providing a thumbnail image that follows a main image
US9972121B2 (en) * 2014-04-22 2018-05-15 Google Llc Selecting time-distributed panoramic images for display
US20150302633A1 (en) * 2014-04-22 2015-10-22 Google Inc. Selecting time-distributed panoramic images for display
USD830399S1 (en) 2014-04-22 2018-10-09 Google Llc Display screen with graphical user interface or portion thereof
USD791813S1 (en) 2014-04-22 2017-07-11 Google Inc. Display screen with graphical user interface or portion thereof
USD829737S1 (en) 2014-04-22 2018-10-02 Google Llc Display screen with graphical user interface or portion thereof
US20170155836A1 (en) * 2015-11-30 2017-06-01 Inventec (Pudong) Technology Corporation System and method for shoot integration
CN105578035A (en) * 2015-12-10 2016-05-11 联想(北京)有限公司 Image processing method and electronic device
US10051180B1 (en) * 2016-03-04 2018-08-14 Scott Zhihao Chen Method and system for removing an obstructing object in a panoramic image
USD868092S1 (en) 2018-06-11 2019-11-26 Google Llc Display screen with graphical user interface or portion thereof
USD868093S1 (en) 2018-08-20 2019-11-26 Google Llc Display screen with graphical user interface or portion thereof

Also Published As

Publication number Publication date
US10038842B2 (en) 2018-07-31
US20150249786A1 (en) 2015-09-03

Similar Documents

Publication Publication Date Title
US8275414B1 (en) User augmented reality for camera-enabled mobile devices
US8489115B2 (en) Sensor-based mobile search, related methods and systems
CA2792336C (en) Intuitive computing methods and systems
Maddern et al. 1 year, 1000 km: The Oxford RobotCar dataset
US8447136B2 (en) Viewing media in the context of street-level images
US9197736B2 (en) Intuitive computing methods and systems
US8339399B2 (en) Declustering point-of-interest icons
KR20160119221A (en) Photo composition and position guidance in an imaging device
US9256806B2 (en) Methods and systems for determining image processing operations relevant to particular imagery
US20130113827A1 (en) Hands-free augmented reality for wireless communication devices
US8977293B2 (en) Intuitive computing methods and systems
EP2232746B1 (en) Image record trend identification for user profiles
US20120290591A1 (en) Method and apparatus for enabling virtual tags
EP2680222A1 (en) Method, apparatus and computer program product for processing media content
US20100251101A1 (en) Capture and Display of Digital Images Based on Related Metadata
KR102010221B1 (en) Smartphone-based methods and systems
US20090324103A1 (en) Method, apparatus and computer program product for providing image modification
US20140270549A1 (en) Using extracted image text
US7953295B2 (en) Enhancing text in images
US8532927B2 (en) Generating photogenic routes from starting to destination locations
US20140164927A1 (en) Talk Tags
US8031940B2 (en) Recognizing text in images using ranging data
US8391615B2 (en) Image recognition algorithm, method of identifying a target image using same, and method of selecting data for transmission to a portable electronic device
US8970665B2 (en) Orientation-based generation of panoramic fields
US20110235858A1 (en) Grouping Digital Media Items Based on Shared Features

Legal Events

Date Code Title Description
AS Assignment

Owner name: MICROSOFT CORPORATION, WASHINGTON

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:WILLIAMS, SAMUEL THOMAS;ROSENBERG, JONATHAN PAUL;KOPF, JOHANNES;AND OTHERS;SIGNING DATES FROM 20111027 TO 20111031;REEL/FRAME:027165/0776

AS Assignment

Owner name: MICROSOFT TECHNOLOGY LICENSING, LLC, WASHINGTON

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MICROSOFT CORPORATION;REEL/FRAME:034544/0001

Effective date: 20141014

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION