US20110254915A1 - Three-Dimensional Overlays Within Navigable Panoramic Images, and Applications Thereof - Google Patents

Three-Dimensional Overlays Within Navigable Panoramic Images, and Applications Thereof Download PDF

Info

Publication number
US20110254915A1
US20110254915A1 US13/171,143 US201113171143A US2011254915A1 US 20110254915 A1 US20110254915 A1 US 20110254915A1 US 201113171143 A US201113171143 A US 201113171143A US 2011254915 A1 US2011254915 A1 US 2011254915A1
Authority
US
United States
Prior art keywords
panoramic image
viewport
overlay
street
panorama
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US13/171,143
Inventor
Luc Vincent
Daniel Filip
Stephen Chau
Stéphane Lafon
Zhonghao Yang
Andrew Timothy Szybalski
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Google LLC
Original Assignee
Google LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US11/754,267 priority Critical patent/US7990394B2/en
Application filed by Google LLC filed Critical Google LLC
Priority to US13/171,143 priority patent/US20110254915A1/en
Assigned to GOOGLE INC. reassignment GOOGLE INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: VINCENT, LUC, FILIP, DANIEL, LAFON, STEPHANE, YANG, ZHONGHAO, CHAU, STEPHEN, SZYBALSKI, ANDREW TIMOTHY
Publication of US20110254915A1 publication Critical patent/US20110254915A1/en
Assigned to GOOGLE LLC reassignment GOOGLE LLC CHANGE OF NAME (SEE DOCUMENT FOR DETAILS). Assignors: GOOGLE INC.
Application status is Abandoned legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00Manipulating 3D models or images for computer graphics
    • G06T19/003Navigation within 3D models or images
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04815Interaction with three-dimensional environments, e.g. control of viewpoint to navigate in the environment
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04817Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/05Geographic models
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00Manipulating 3D models or images for computer graphics
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00Manipulating 3D models or images for computer graphics
    • G06T19/20Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2219/00Indexing scheme for manipulating 3D models or images for computer graphics
    • G06T2219/028Multiple view windows (top-side-front-sagittal-orthogonal)
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2219/00Indexing scheme for manipulating 3D models or images for computer graphics
    • G06T2219/20Indexing scheme for editing of 3D models
    • G06T2219/2004Aligning objects, relative positioning of parts

Abstract

A panorama viewer is disclosed which facilitates navigation from within the panorama of a larger, structured system such as a map. The panorama viewer presents a viewport on a portion of a panoramic image, the viewport including a three-dimensional overlay rendered with the panoramic image. As the orientation of the viewport within the panoramic image changes, the three-dimensional overlay's orientation in three-dimensional space also changes as it is rendered with the panoramic image in a manner that matches the change in orientation of the viewport.

Description

  • This application is a continuation of U.S. patent application Ser. No. 11/754,267, filed May 25, 2007, which is incorporated by reference herein in its entirety.
  • FIELD OF THE INVENTION
  • The present invention generally relates to displaying imagery, and more particularly to viewing and navigating within panoramic images, and applications thereof.
  • BACKGROUND
  • Computerized mapping systems traditionally provide a top-down presentation of the mapping data. Enhancing the mapping system with street-level imagery presents various interface challenges: such as dealing with navigation within the street-level view, including turning at intersections, and correlating the user's location and orientation on a map with the user's location and orientation within the street-level view, both in absolute terms (e.g., latitude, longitude, and heading) and relative to landmarks like city streets and intersections. Recently, A9 BlockView (no longer online) and Windows Live Local Technology Preview powered by Microsoft Virtual Earth (http://preview.local.live.com) have attempted to provide a usable interface to street-level views of a city. A9 BlockView addressed the orientation problem by flattening imagery into two strips, so that the user does not have the freedom to look around 360 degrees. Windows Live Local presents a “car” view of street-level imagery which is rotated in 90 degree increments by manipulating a car avatar on the map view.
  • BRIEF SUMMARY OF THE INVENTION
  • The present invention relates to viewing and navigating within panoramic images, and applications thereof. In an embodiment, a panorama viewer is disclosed which facilitates navigation from within the panorama of a larger, structured system such as a map. The panorama viewer presents a viewport on a portion of a panoramic image, the viewport including a three-dimensional overlay rendered with the panoramic image. As the orientation of the viewport within the panoramic image changes, the three-dimensional overlay's orientation in three-dimensional space also changes as it is rendered with the panoramic image in a manner that matches the change in orientation of the viewport. For example, where the panoramic image is of street-level imagery, the three-dimensional overlay can comprise a three-dimensional representation of annotations to a map, including three-dimensional representation of the lines that represent the streets on the map.
  • Further embodiments, features, and advantages of the present invention, as well as the structure and operation of the various embodiments of the present invention, are described in detail below with reference to the accompanying drawings.
  • BRIEF DESCRIPTION OF THE DRAWINGS/FIGURES
  • Embodiments of the invention are described with reference to the accompanying drawings. In the drawings, like reference numbers may indicate identical or functionally similar elements.
  • FIG. 1 is a diagram of an exemplary distributed system suitable for practicing an embodiment.
  • FIG. 2 is a diagram illustrating an example of how a mapping service can be integrated with a panorama viewer, in accordance with an embodiment.
  • FIG. 3 depicts an example of a browser display, in accordance with an embodiment.
  • FIG. 4 is a flowchart illustrating exemplary processing performed by a panorama viewer, in accordance with an embodiment.
  • FIG. 5 depicts exemplary Extensible Markup Language (XML) configuration information.
  • FIG. 6 illustrates an example of a panoramic image.
  • FIGS. 7A, 7B, and 7C illustrate user interaction with the panorama viewer viewport.
  • FIG. 8 is a flowchart of processing performed by a renderer, in accordance with an embodiment.
  • FIGS. 9A, 9B, and 9C illustrate a relationship between a surface, a precomputed region, and a viewport.
  • FIGS. 10A and 10B illustrate a simple example of generating transformation parameters.
  • FIG. 11 depicts a panorama which has been warped, in accordance with an embodiment of the invention.
  • FIG. 12 depicts an exemplary transformation based on yaw and pitch for forming the panorama of FIG. 11.
  • FIG. 13 depicts an exemplary panorama image displayed in accordance with an embodiment of the invention.
  • FIG. 14 is a diagram illustrating how to generate coordinates for a user annotation, in accordance with an embodiment.
  • FIG. 15 is a flowchart of processing performed in the generation of user annotation coordinates, in accordance with an embodiment.
  • The present invention is described with reference to the accompanying drawings. The drawing in which an element first appears is typically indicated by the leftmost digit or digits in the corresponding reference number.
  • DETAILED DESCRIPTION
  • The present invention relates to viewing and navigating within panoramic images, and applications thereof. In the detailed description of the invention herein, references to “one embodiment”, “an embodiment”, “an example embodiment”, etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to effect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.
  • FIG. 1 is a distributed system suitable for practice of an embodiment of the invention. A client 110 communicates with one or more servers 150, for example, across a network such as the Internet or a local area network. Client 110 can be a general-purpose computer with a processor, local memory, a display, and one or more input devices such as a keyboard or a mouse. Alternatively, client 110 can be a specialized computing device such as, for example, a mobile handset. Server(s) 150, similarly, can be implemented using any general-purpose computer capable of serving data to client 110.
  • Client 110 executes a panorama viewer 120, the operation of which is further described herein.
  • As illustrated by FIG. 1, panorama viewer 120 requests configuration information 130 from server(s) 150. As discussed in further detail herein, the configuration information includes meta-information about a panorama to be loaded, including information on links within the panorama to other panoramas. In an embodiment, the configuration information is presented in a form such as the Extensible Markup Language (XML). Panorama viewer 120 retrieves visual assets 140 for the panorama, for example, in the form of panoramic images or in the form of panoramic image tiles. In another embodiment, the visual assets include the configuration information in the relevant file format. Panorama viewer 120 presents a visual representation on the client display of the panorama and additional user interface elements, as generated from configuration information 130 and visual assets 140, as further described herein. As a user interacts with an input device to manipulate the visual representation of the panorama, panorama viewer 120 updates the visual representation and proceeds to download additional configuration information and visual assets as needed.
  • In an embodiment, panorama viewer 120 can be a standalone application, or it can be executed within a browser 115, such as Mozilla Firefox or Microsoft Internet Explorer. Panorama viewer 120, for example, can be executed as a script within browser 115, as a plug-in within browser 115, or as a program which executes within a browser plug-in, such as the Adobe (Macromedia) Flash plug-in. In an embodiment, panorama viewer 120 is integrated with a mapping service, such as the one described in U.S. Pat. No. 7,158,878, “DIGITAL MAPPING SYSTEM”, which is incorporated by reference in its entirety herein.
  • FIG. 2 illustrates an example of how a mapping service 210 can be integrated with panorama viewer 120. Mapping service 210 displays a visual representation of a map, e.g., as a viewport into a grid of map tiles. Mapping system 210 is implemented using a combination of markup and scripting elements, e.g., using HTML and Javascript. As the viewport is moved, mapping service 210 requests additional map tiles 220 from server(s) 150, assuming the requested map tiles have not already been cached in local cache memory. Notably, the server(s) which serve map tiles 220 can be the same or different server(s) from the server(s) which serve panorama tiles 140 or the other data involved herein.
  • In an embodiment, mapping service 210 can request that browser 115 proceed to download a program 250 for panorama viewer 120 from server(s) 150 and to instantiate any plug-in necessary to run program 250. Program 250 may be a Flash file or some other form of executable content. Panorama viewer 120 executes and operates as described above. Alternatively, configuration information 130 and even panorama tiles 140 can be retrieved by mapping service 210 and passed to panorama viewer 120. Panorama viewer 120 and mapping service 210 communicate so as to coordinate the operation of the user interface elements, to allow the user to interact with either panorama viewer 120 or mapping service 210, and to have the Change in location or orientation reflected in both.
  • FIG. 3 is an example browser display 300 that presents both a mapping service such as mapping service 210 and an integrated panorama viewer such as panorama viewer 120. The mapping service provides a button 310 entitled “Street View” that, when selected, preferably changes the appearance of the map in areas where panorama data is available. For example, in FIG. 3, streets with available panorama data are highlighted. This highlighting can be, for example, a colored and/or shaded outline or overlay, or a change in color and/or shading. This can be implemented by using a transparency image with the map tile or by directly including the effect in the map tile served to the mapping service.
  • The mapping service allows a user to activate the panorama viewer by further selecting a point on the map. When a point is selected by the user, a character or avatar icon 320 is displayed at the point on the map. In an embodiment, the avatar icon includes an indicator of what direction the avatar icon is facing, which in FIG. 3 is represented as an arrow underneath the avatar icon.
  • In an embodiment, as the panorama viewer is instantiated by the mapping service, the panorama viewer is presented in the form of viewport 330 embedded in an informational balloon window associated with avatar icon 320. The orientation of the visual representation of the panorama within viewport 330 matches the orientation of avatar icon 320. As the user manipulates the visual representation of the panorama within viewport 330, the panorama viewer informs the mapping service of any changes in orientation or location so that the mapping service can update the orientation and location of avatar icon 320. Likewise, as the user manipulates the orientation or location of avatar icon 320 within the mapping service, the mapping service informs the panorama viewer so that the panorama viewer can update its visual representation.
  • In an embodiment, the viewport 330 of the panorama viewer presents a panoramic image of the selected area. The user can click and drag around on the image to look around 360 degrees. In the example viewport 330 depicted in FIG. 3, a variety of user interface elements are added to the underlying panorama. These elements include navigation inputs such as, for example, zoom and panning controls (e.g., navigation buttons) on the left side of the viewport and annotations in the form of lines/bars, arrows, and text that are provided directly in the panorama itself. The annotations are rendered in a three dimensional manner that roughly matches the three dimensional scene depicted in the panorama.
  • In FIG. 3, for example, the lines/bars in viewport 330 correspond to the streets depicted in the corresponding map and can even be rendered in the same color as the streets depicted in the map. The arrows are selectable by a user (by clicking or by dragging along the street line), one going in each direction that there is another panorama available. These allow the user to navigate up and down the street (i.e., to change the vantage point from which the street is viewed). As the user looks around 360 degrees, the lines and arrows smoothly track the underlying imagery so that the lines remain on top of the underlying streets, and so that the arrows are always visible on the screen. This allows the user to navigate along the street while looking straight ahead, or while looking to the side of the storefront.
  • When the user clicks on an arrow to navigate within the viewport, a zooming cross-fade effect and other visual cues give the user a sense of movement. When the user arrives at an intersection of two streets, there is one green line and two arrows for each street. All of these are visible at the same time, and all are labeled, so that the user knows the current location and can proceed in any direction. This technique can readily scale to accommodate complex intersections with more than four directions. When the user reaches a “dead end” where the road continues but no further imagery is available, there is only one arrow on the street indicating the direction in which the user can navigate. In the other direction, a symbol and message embedded in the image can be presented to inform the user that imagery is not available in this direction.
  • The user interface is not restricted to navigating along a line to walk down a street and can be readily extended to allow users to stray from the line elements when useful: for example, to cross over to the opposite side of the street to get a closer look at something. Moreover, there are environments within a city where a user might be expected to desire to snap off of a street and navigate freely within an adjacent area, for example, a park, plaza, shopping area, or other pedestrian-friendly public place. The interface can be readily enhanced with “free movement zones” to provide this functionality. It should also be noted that although the user interface is presented in the context of navigation between discrete street-level panoramas, it could equally well be used to allow a user to navigate through a more continuous set of panoramic data, such that navigating along a street would be as smooth as video.
  • The operation and implementation of the user interface elements are described in further detail below.
  • FIG. 4 is an exemplary flowchart of processing performed by a panorama viewer such as, for example, panorama viewer 120, in accordance with an embodiment of the invention.
  • At step 402, the panorama viewer receives an identifier for the initial panorama to be presented and various viewport parameters, such as the size of the viewport and the orientation to be viewed within the panorama. This information can be passed to the panorama viewer from a mapping service, e.g., by using Flashvars or ExternalInterface between the Flash plug-in and the Javascript in the mapping service.
  • At step 404, the panorama viewer uses the panorama identifier to request configuration information from the server (e.g., an XML file). FIG. 5 depicts exemplary XML configuration information 500 and is discussed in further detail below. The XML is parsed, and the information is loaded into various data structures for use by the panorama viewer. In an embodiment, the XML includes information for the panorama viewer such as data properties and projection properties of the current panorama, and information on annotations/links within the panorama, including links to other panoramas.
  • At step 406, the panorama viewer requests the visual assets for the panorama and stores the received visual assets, for example, in local memory/storage. In an embodiment, the panorama viewer can maintain a cache of visual assets and limit bandwidth usage to retrieval of visual assets which are not in the cache. FIG. 6 illustrates an example of an initial panoramic image 600. The complete panoramic image 600 can be retrieved by the panorama viewer, or panoramic image 600 can be divided into multiple panorama image tiles and the tiles requested only as needed by the panorama viewer.
  • At step 408, the panorama viewer processes the configuration information and the visual assets to prepare for rendering the visual representation of the panorama in the viewport at step 410. With regard to the visual assets, the panorama viewer can assemble the panorama image tiles into the portion of the complete panoramic image which overlaps with the viewport. The panorama viewer can present the panoramic image as a flat surface or as a texture-mapped three dimensional surface such as, for example, a cylinder or a sphere, as further discussed herein. With regard to the annotations overlay presented in the viewport, the panorama viewer uses the configuration information to compute the shapes and locations for these various elements such as, for example, the lines/bars and the arrows presented in the viewport.
  • In an embodiment, the polygons/shapes are modeled in a three-dimensional space that corresponds to the space depicted in the panorama. These polygons/shapes can be modeled, for example, using a pinhole camera model (e.g., the focal length can be generated by multiplying the height of the viewport by a constant relative depth of the center of rotation). The polygons/shapes of the annotations overlay change their orientation in the three-dimensional space in a manner that matches the change in orientation of the viewport. In one embodiment, the polygons/shapes are rotated by an angle equal to the difference between the current orientation of the user's point-of-view in the panorama and the direction of the annotation, as specified in the configuration information. The polygons/shapes can be further transformed around different spatial axes in order to take into account non-flat panoramas, as further described herein.
  • At step 410, the visual representation of the panorama in the viewport is rendered.
  • At step 412, the panorama viewer receives and manages input, for example, by capturing input events such as mouse and keyboard events. The panorama viewer, for example, detects whether the user has panned the viewport (e.g., by dragging the mouse or by selecting a pan control button), has selected to zoom (e.g., by clicking on the panorama or by moving the zoom slider control on the left of the viewport with the mouse) or has selected a link to another panorama (e.g., by clicking on an arrow with the mouse).
  • At step 420, a determination is made regarding whether a user has panned the viewport. If the user has panned the viewport, control transfers to step 422. If the user has not panned the viewport, control transfers to step 430.
  • At step 422, the panorama viewer determines whether the viewport will overlap with any panorama image tiles which will need to be retrieved from either the server or a cache.
  • At step 424, the panorama viewer executes the necessary computations for allowing the viewport correctly to be rendered in a different orientation, as further described in detail herein.
  • At step 426, the panorama viewer notifies the mapping service of the new orientation selected by the user so that the mapping service can update its avatar icon's facing indicator. The panorama viewer re-computes the shapes and locations for the viewport elements and renders the viewport. To illustrate this point, consider FIG. 7A which depicts the panorama viewer viewport from FIG. 3. FIG. 7B shows the result after a user has selected to pan the panorama to the left. Note that the lines/bars that correspond to the roads depicted in the panorama change their orientation as the panorama viewer changes the orientation of the panorama.
  • At step 430, a determination is made regarding whether a user has zoomed the viewport. If the user has zoomed the viewport, control transfers to step 432. If the user has not zoomed the viewport, control transfers to step 440.
  • At step 432, the panorama viewer determines, for example, whether to request new higher resolution panorama image tiles from a server (or from cache), or whether to utilize existing tiles at a different close-up resolution, for example, where no such higher resolution tiles exist.
  • At step 434, the viewport parameters are changed to reflect the different zoom level. A transition can be provided between the zoom levels so as to give the appearance of actively zooming into the next zoom level of the panorama. FIG. 7C shows the result after the user has selected to zoom in on a feature in FIG. 7A.
  • At step 440, a determination is made regarding whether a user has selected a link to another panorama. If the user has selected a link to another panorama, control transfers to step 442. If the user has not selected a link to another panorama, control transfers to step 412.
  • At step 442, the panorama viewer proceeds to begin the process of transitioning between the original panorama and the new panorama. The panorama viewer can, for example, zoom the original panorama and perform a cross-fade to the new panorama to give the user a sense of movement. Alternatively, the panorama viewer can play an actual video transition between the two panoramas.
  • At step 444, the panorama viewer notifies the mapping service of the new location selected by the user so that the mapping service can update its avatar icon's location and can scroll the map accordingly.
  • In embodiments, the panorama viewer can be implemented using any advantageous programming language or style of programming. For example, the panorama viewer can be implemented using object-oriented programming with separate classes designated to handle the XML configuration information, the annotations, the texture generation, the tile management, and the mesh generation.
  • FIG. 5 sets forth exemplary XML configuration information 500 (e.g., metadata). As illustrated by the example shown in FIG. 5, the schema for the configuration information is organized into “data_properties”, “projection_properties”, and “annotation_properties”.
  • The subgroup Data_Properties contains attributes such as “pano_id” (e.g., a unique identifier for the panorama), “image_width” and “image_height” (e.g., dimensions of the panoramic image before being split into tiles), “tile_width” and “tile_height” (e.g., dimensions of the tiles), “lat” and “lng” (e.g., coordinates of the current panorama), and “num_zoom_levels” (e.g., the number of zoom levels that the user will be able to view in the panorama viewer). This subgroup also contains elements such as “text” (e.g., that can be used to represent the street name of the current panorama), “copyright” (e.g., copyright information), and “street_range” (e.g., the range of numbers in the given street).
  • The subgroup Projection_properties contains attributes such as “Pano_yaw_deg” (e.g., orientation of the vehicle which captured the images which generated the panoramic image), “tilt_yaw_deg” and “tilt_pitch_deg” (e.g., the yaw and pitch of the line of highest slope which, as further described herein, is useful for dealing with panoramas with sloped features), and “vertical_scale” (e.g., fraction of the image along the y-axis that is visible at the lower zoom level).
  • The subgroup Annotation_properties contains attributes such as “horizon_height_fraction” (e.g., vertical position (height) of the horizon, expressed as a fraction of the visible strip, which can be adjusted to maximize the fit between the annotations and the imagery of the tiles) and “annotation_height_fraction” (e.g., vertical position (height) of the plan containing the annotations, expressed as a fraction of the visible strip). This subgroup also includes the “pano_link” subgroup which describes properties of link symbols that allow a user to navigate to a neighboring panorama or to another related document. The “link” subgroup includes “link_text” (e.g., description of the landing panorama) as an element and includes the following as attributes: “yaw_deg” (e.g., direction that the link is pointing to), “pano_id” (e.g., identifier to linked panorama), and “road_argb” (e.g., an attribute of the road, such as the color of the road on the map) (not shown). The subgroup can also include a “floating_text” group or element which identifies arbitrary features in the panorama and could also provide for an arbitrary link, for example to a local data repository or a website (not shown).
  • It should be noted that the above schema for the configuration information is merely illustrative and can be arranged in any of a number of advantageous ways, including using techniques that do not rely on XML.
  • FIG. 8 is a flowchart of processing performed by a renderer in accordance with an embodiment of the invention.
  • At step 802, the renderer precomputes a pre-image of a viewport by a backward transform. This defines a portion of a surface, which is referred to herein as the “precomputed region”. FIG. 9A illustrates this in the context of a cylindrical surface 900 with a rectangular viewport 920, Which defines a precomputed region 910 by the backward transformation. It should be noted that the viewport does not have to be rectangular and that the technique works for discretized cylinders (based on a mesh) or for continuous voxel-to-pixel mappings. For example, a mesh can be defined on the viewport with a corresponding mesh on the cylinder. These meshes do not have to be uniform, and are images of one another as defined by the forward or backward mappings. The mesh on the cylinder will typically only cover a portion of the cylinder. In the case of a continuous transformation, the pre-image of the viewport would define a continuous region of the cylinder.
  • At step 804, the renderer precomputes the transform, which maps each pixel from the precomputed region to a pixel in the viewport. In a sense, it is assumed that the cylinder is standing still in space. Instead of attaching a texture image to a changing cylinder, it is assumed that the texture will “slide” on the cylinder.
  • At step 806, the renderer translates an image/texture in response to a user input.
  • At step 808, the renderer determines that portion of the image/texture that intersects the precomputed region of the surface. This defines the set of pixels that need to be rendered in the viewport. If the user has changed the point of view recently, then this needs to be updated. More precisely, any panning to the left or right of the viewport can be readily achieved by translating the texture in the corresponding direction, at step 806, thereby generating a different intersection with the precomputed region. Likewise, any panning up or down is achieved by translating the texture along these directions. Any arbitrary direction of panning is achieved by simply translating the texture in the corresponding direction. Each time, a new intersection with the precomputed region is generated. This is illustrated in FIGS. 9B and 9C, where 950 represents the precomputed region and 960 represents the image/texture.
  • At step 810, the precomputed transform is applied to the portion of image/texture that intersects with the precomputed region.
  • Finally, at step 812, the transformed imagery is rendered into the viewport.
  • In an embodiment, the renderer utilizes properties of rectilinear projections to speed up the rendering of the panoramic images. If a surface like a cylinder is viewed as infinite, then it is a group endowed G with the natural operations (e.g., translation along axis, and rotation around axis). Likewise, the texture, if viewed as infinite, is also a group H endowed with translations in the plane. It turns out that there is a canonical homomorphism between G and H. In other words, a rotation of the cylinder around its axis is equivalent to as translation of the texture, for example, in the x-direction. A translation of the cylinder along its axis is equivalent to a translation of the texture, for example, in the y-direction. This allows one to pre-compute all projection parameters in advance and to simulate a change of viewpoint as a translation of the texture. FIG. 10A and 10B illustrate an example of how to compute projection parameters from the screen space to the texture space. As illustrated by FIGS. 10A and 10B:
      • (1) If a point M on screen 1020 has coordinates (x, y), then in space it has coordinates (x, y, R), where R is the radius of the cylinder 1010.
      • (2) In this case,
  • tan θ = x R ,
  • and the point P has, in the texture space, the following coordinates:
  • P = ( arctan ( x R ) Ry x 2 + R 2 ) .
  • A dynamic texture based on the current zoom level can be generated and positioned in the image space. This texture changes when the user changes the point-of-view (e.g., by zooming or panning). This texture can be obtained by concatenating tiles from a tile pyramid at the appropriate level and scale. If some tiles are missing, one can fall back on a tile at a parent level in the tile pyramid. The texture is modeled as being mapped over a cylinder. A projection is performed over the screen space. This nonlinear projection can be approximated as a piecewise affine transformation. More precisely, the cylinder and screen spaces can be discretized using a triangular mesh. Each triangle can be rendered by linearly (or rather, affinely) mapping a piece of the texture over it. This is well-defined as an affine transform in the two dimensional plane since it is uniquely determined by its action on three points (hence, the use of triangles). The mesh can be made uniform in the screen space (and not in the texture space). The screen mesh is always the same regardless of the zoom level. Different texture meshes can be used depending on the zoom level. For each triangle, a texture mesh corresponds to a unique triangle in the screen mesh and a unique (affine) transformation matrix. Such a matrix can be pre-computed as the product of a screen matrix and (the inverse of) a texture matrix.
  • When a user pans, all the renderer needs to do is adjust and/or refresh the texture. This is fast, because it consists of memory copies. Copying large chunks of pixels is usually highly optimized in several programming languages.
  • In an embodiment, zooming in consists of dividing both horizontal and vertical fields of vision by two and using the next zoom level to generate the texture. When the user zooms in/put, the panorama viewer can pre-cache a few bitmap images to make the animation smooth. As far as the projection itself, one can use the various sets of transformation matrices at integral zoom levels. At non-integral zoom levels, one can linearly interpolate the transformation matrices (still fast) between the previous zoom level and the next one.
  • In a case where pixels are assumed to be square, they correspond to homogeneous solid angles. The elementary field of vision of a given pixel is the same in the horizontal and the vertical directions. This enables a trade-off to be made. For example, one can choose exactly to preserve straight lines, but this will result in higher distortion on the side. Alternatively, one can decide to have straight lines become a bit curvy, and thereby reduce the amount of distortion. In an embodiment, the height of the original image is scaled to that of the viewport. Because the pixels are square, the ratio of the width over height of the viewport determines the horizontal field of vision.
  • In the case of a sphere rather than a cylinder, the assumptions above are no longer true. Accordingly, the above technique alone cannot simulate a true pan up/down, because all that is performed is a move along the cylinder's axis with the vision vector perpendicular to this axis. Nevertheless, the motion of a true pan up/down can be simulated by pre-computing a series of transforms and linearly interpolating between the transforms.
  • In an embodiment, the panorama viewer is configured to handle non-flat panoramas. Not all panoramas depict flat and horizontal features, e.g., consider many of the streets of San Francisco. Cameras mounted on a vehicle, for example, used to capture panoramas are parallel to the ground. Thus, traveling on a steep incline can result in misoriented pictures. Accordingly, in such situations, it can be advantageous to warp the panorama so as to ensure that vertical buildings in the real world remain vertical in the texture space. FIG. 11 depicts an example of how a panorama 1100 can be so warped. As illustrated by FIG. 12, the example roughly follows a periodic function which can be used to guide the placement of the viewport as well as the generation of the annotations in a manner that takes the slope of the panorama into account.
  • As noted, herein, the configuration information can include projection properties such as the yaw and the pitch of the highest slope in the panorama. As illustrated by FIG. 12, the panorama viewer can use the yaw and pitch of the direction of the steepest slope to constrain the viewport to the sinusoidal strip of the warped panorama. The rendering of the annotation elements in the viewport also can be modified to take into account the yaw and pitch information of the slope of the panorama. The spatial orientation of the lines/bars and arrows can be transformed based on the yaw and pitch information or, alternatively, can be estimated based on the relative yaw of the annotation and the yaw of the steepest slope. FIG. 13 illustrates the result of such processing of configuration information on the slope of a panorama. The panorama correctly places the vertical buildings in the panoramic image on the steeply-sloped street. Moreover, the line/bar (e.g., street line metadata) depicting the road is tilted at an angle which roughly matches the slope of the street.
  • In an embodiment, the panorama viewer also is able to facilitate user annotations to a panorama image. User annotations to panoramas represent a challenge with respect to how to reference an annotation in three-dimensional space.
  • FIGS. 14 and 15 illustrate an embodiment which addresses user annotations in three-dimensional space. The processing illustrated in FIG. 15 can occur at the panorama viewer (or the mapping service), at the server, or a combination of the two.
  • Referring to FIG. 15, at step 1502, a user inputs an annotation with respect to one panorama. The panorama viewer can receive the user input in any of a number of different ways, including by receiving a click event on the spot on the panorama which the user desires to annotate. The two-dimensional location of the annotation on the panorama is recorded in some advantageous coordinate system, e.g., by location on the panorama image or by yaw and pitch coordinates.
  • At step 1504, the user navigates to another nearby panorama in the panorama viewer, locates the same feature to be annotated, and again inputs an annotation with respect to the second panorama. The panorama viewer or the mapping service can presumably also offer the ability to add additional metadata associated with the annotation, such as a title, link, graphics, etc.
  • At step 1506, the annotations coordinates on the two panoramas are used to generate three-dimensional coordinates for the annotation. Given the known position of the cameras which took the images for the panorama 1410, 1420 and the user-input annotation coordinates relative to the two-dimensional images, as illustrated in FIG. 14, it is possible to compute the intersection of the two, depicted as 1450. The result is a three-dimensional coordinate for the annotation.
  • At step 1508, the three-dimensional coordinate for the annotation is assigned to the annotation and stored in the database of annotations. The annotation can then be included with any panorama within some advantageous range of the computed coordinates, including panoramas which were not originally annotated by the user.
  • Alternatively, where the relative pitch information is not particularly important to an annotation, it is possible to receive the user annotations as a one-dimensional yaw direction on both panoramas, which facilitates the assignment of a two-dimensional geocode to the annotation (with or without default pitch information).
  • While various embodiments of the present invention have been described above, it should be understood that they have been presented by way of example, and not limitation. It will be apparent to persons skilled in the relevant art(s) that various changes can be made therein without departing from the scope of the invention. Furthermore, it should be appreciated that the detailed description of the present invention provided herein, and not the summary and abstract sections, is intended to be used to interpret the claims. The summary and abstract sections may set forth one or more but not all exemplary embodiments of the present invention as contemplated by the inventors.
  • The foregoing description of the specific embodiments will so fully reveal the general nature of the invention that others can, by applying knowledge within the skill of the art, readily modify and/or adapt for various applications such specific embodiments, without undue experimentation, without departing from the general concept of the present invention. Therefore, such adaptations and modifications are intended to be within the meaning and range of equivalents of the disclosed embodiments, based on the teaching and guidance presented herein. It is to be understood that the phraseology or terminology herein is for the purpose of description and not of limitation, such that the terminology or phraseology of the present specification is to be interpreted by the skilled artisan in light of the teachings and guidance.

Claims (20)

1. A method for viewing panoramic images, comprising:
(a) receiving at least a portion of a first panoramic image geocoded to a first location;
(b) receiving configuration information specifying an overlay for the first panoramic image;
(c) rendering, in a three-dimensional space, the at least a portion of the first panoramic image for display;
(d) rendering the overlay in the three-dimensional space of the first panoramic image for display with the first panoramic image in a viewport;
(e) in response to receipt of a first input to change orientation of the viewport, changing the orientation of the overlay in three-dimensional space as it is rendered with the first panoramic image so as to match a change in orientation of the first panoramic image within the viewport; and
(f) in response to receipt of a second input specifying a position on the first panoramic image, presenting a second panoramic image geocoded to a second location different from the first location, the second location determined according to the position.
2. The method of claim 1, wherein the rendering (d) comprises modeling the overlay using a pinhole camera model, wherein a focal length of the pinhole camera model corresponds to a height of the viewport and a depth of the three-dimensional space.
3. The method of claim 1, wherein the rendering (c) comprises texture mapping the at least a portion of the first panoramic image to a surface having canonical homomorphism.
4. The method of claim 1, wherein the overlay comprises an annotation describing a feature of the first panoramic image, the annotation positioned to correspond to a position of the feature in the first panoramic image.
5. The method of claim 4, wherein the annotation is a street line positioned to correspond to the direction of a street in the first panoramic image, and wherein the overlay further comprises a street name positioned to identify the street in the first panoramic image.
6. The method of claim 1, wherein the overlay includes a navigation icon associated with the second panoramic image and wherein the second input in (f) is a selection of the navigation icon.
7. The method of claim 1, wherein the receiving (a) comprises receiving street-level imagery.
8. The method of claim 1, further comprising:
(g) displaying a user orientation character at a first position on a map, the first position corresponding to the first location; and
(h) in response to the second input, displaying the user orientation character at a second position on the map, the second position corresponding to the second location.
9. The method of claim 8, further comprising:
(j) presenting a first line on the map representing a first street where panoramic image data is unavailable; and
(k) presenting a second line on the map differently from the presenting in (j), the second line representing a second street where panoramic image data is available,
whereby the difference in the presenting (j) and the presenting (k) indicates to a user where panoramic image data is available.
10. A system for viewing panoramic images, comprising:
a panorama viewer that
(i) receives at least a portion of a first panoramic image and configuration information, the first panoramic image geocoded to a first location, and the configuration information specifying an overlay for the first panoramic image,
(ii) renders, in a three-dimensional space, the at least a portion of the first panoramic image and the overlay of the first panoramic image for display in a viewport,
(iii) in response to receipt of a first input to change orientation of the viewport, changes the orientation of the overlay in three-dimensional space as it is rendered with the first panoramic image so as to match a change in orientation of the first panoramic image within the viewport, and
(iv) in response to receipt of a second input specifying a position on the first panoramic image, presents a second panoramic image geocoded to a second location different from the first location, the second location determined according to the position.
11. The system of claim 10, wherein the panorama viewer models the overlay using a pinhole camera model to render the overlay in the three-dimensional space, wherein a focal length of the pinhole camera model corresponds to a height of the viewport and a depth of the three-dimensional space.
12. The system of claim 10, wherein the panorama viewer models texture maps the at least a portion of the first panoramic image to a surface having canonical homomorphism to render the first panoramic image.
13. The system of claim 10, wherein the overlay comprises an annotation describing a feature of the first panoramic image, the annotation positioned in the first panoramic image to correspond to a position of the feature.
14. The system of claim 13, wherein the annotation is a street line positioned to correspond to the direction of a street in the first panoramic image, and wherein the overlay further comprises a street name positioned to identify the street in the first panoramic image.
15. The system of claim 10, wherein the overlay includes a navigation icon associated with the second panoramic image and wherein the second input is a selection of the navigation icon.
16. The system of claim 10, wherein the first and second panoramic images comprise street-level imagery.
17. The system of claim 10, further comprising:
a mapping service that displays a user orientation character at a first position on a map, the first position corresponding to the first location and, in response to the second input, displays the user orientation character at a second position on the map, the second position corresponding to the second location.
18. The system of claim 17, wherein the mapping service presents a first line on the map representing a first street where panoramic image data is unavailable and presents a second line on the map differently from the presentation of the first line, the second line representing a second street where panoramic image data is available,
whereby the difference in the presentation of the first line and the presentation of the second line indicates to a user where panoramic image data is available.
19. A system for serving panoramic images and metadata, comprising:
a server that sends data to a client, the data including at least a portion of a panoramic image geocoded to a location, a configuration information specifying an overlay for the panoramic image, and a file,
wherein the file instructs the client to render, in a three-dimensional space, the at least a portion of the panoramic image for display and to render the overlay in the three-dimensional space of the panoramic image for display with the panoramic image in a viewport,
wherein the file instructs the client, in response to receipt of a first input to change orientation of the viewport, to change the orientation of the overlay in three-dimensional space as it is rendered with the panoramic image so as to match a change in orientation of the panoramic image within the viewport, and
wherein the file instructs the client, in response to receipt of a second input specifying a position on the panoramic image, to present a second panoramic image geocoded to a second location different from the first location, the second location determined according to the position.
20. The system of claim 19, wherein the file is a Flash file.
US13/171,143 2007-05-25 2011-06-28 Three-Dimensional Overlays Within Navigable Panoramic Images, and Applications Thereof Abandoned US20110254915A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US11/754,267 US7990394B2 (en) 2007-05-25 2007-05-25 Viewing and navigating within panoramic images, and applications thereof
US13/171,143 US20110254915A1 (en) 2007-05-25 2011-06-28 Three-Dimensional Overlays Within Navigable Panoramic Images, and Applications Thereof

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US13/171,143 US20110254915A1 (en) 2007-05-25 2011-06-28 Three-Dimensional Overlays Within Navigable Panoramic Images, and Applications Thereof
US14/182,374 US8982154B2 (en) 2007-05-25 2014-02-18 Three-dimensional overlays within navigable panoramic images, and applications thereof
US14/620,421 US20150161820A1 (en) 2007-05-25 2015-02-12 Three-dimensional overlays within navigable panoramic images, and applications thereof

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US11/754,267 Continuation US7990394B2 (en) 2007-05-25 2007-05-25 Viewing and navigating within panoramic images, and applications thereof

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US14/182,374 Continuation US8982154B2 (en) 2007-05-25 2014-02-18 Three-dimensional overlays within navigable panoramic images, and applications thereof

Publications (1)

Publication Number Publication Date
US20110254915A1 true US20110254915A1 (en) 2011-10-20

Family

ID=40071984

Family Applications (4)

Application Number Title Priority Date Filing Date
US11/754,267 Active 2029-06-03 US7990394B2 (en) 2007-05-25 2007-05-25 Viewing and navigating within panoramic images, and applications thereof
US13/171,143 Abandoned US20110254915A1 (en) 2007-05-25 2011-06-28 Three-Dimensional Overlays Within Navigable Panoramic Images, and Applications Thereof
US14/182,374 Active US8982154B2 (en) 2007-05-25 2014-02-18 Three-dimensional overlays within navigable panoramic images, and applications thereof
US14/620,421 Abandoned US20150161820A1 (en) 2007-05-25 2015-02-12 Three-dimensional overlays within navigable panoramic images, and applications thereof

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US11/754,267 Active 2029-06-03 US7990394B2 (en) 2007-05-25 2007-05-25 Viewing and navigating within panoramic images, and applications thereof

Family Applications After (2)

Application Number Title Priority Date Filing Date
US14/182,374 Active US8982154B2 (en) 2007-05-25 2014-02-18 Three-dimensional overlays within navigable panoramic images, and applications thereof
US14/620,421 Abandoned US20150161820A1 (en) 2007-05-25 2015-02-12 Three-dimensional overlays within navigable panoramic images, and applications thereof

Country Status (1)

Country Link
US (4) US7990394B2 (en)

Cited By (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100161207A1 (en) * 2008-12-24 2010-06-24 In-Young Do Mobile terminal and method for providing location-based service thereof
US20110141141A1 (en) * 2009-12-14 2011-06-16 Nokia Corporation Method and apparatus for correlating and navigating between a live image and a prerecorded panoramic image
US20120218266A1 (en) * 2011-02-24 2012-08-30 Nintendo Co., Ltd. Storage medium having stored therein display control program, display control apparatus, display control system, and display control method
US8306749B1 (en) 2011-11-22 2012-11-06 Google Inc. Systems and methods for route summarization
US20130083055A1 (en) * 2011-09-30 2013-04-04 Apple Inc. 3D Position Tracking for Panoramic Imagery Navigation
US8570329B1 (en) * 2012-10-31 2013-10-29 Google Inc. Subtle camera motions to indicate imagery type in a mapping system
US20130332810A1 (en) * 2012-06-07 2013-12-12 Microsoft Corporation Managing objects in panorama display to navigate spreadsheet
WO2014008320A1 (en) * 2012-07-03 2014-01-09 Tourwrist, Inc. Systems and methods for capture and display of flex-focus panoramas
US20140161323A1 (en) * 2010-09-21 2014-06-12 Mobileye Technologies Limited Dense structure from motion
US20140169699A1 (en) * 2012-09-21 2014-06-19 Tamaggo Inc. Panoramic image viewer
US8767040B2 (en) 2012-01-11 2014-07-01 Google Inc. Method and system for displaying panoramic imagery
US8907968B2 (en) 2011-03-31 2014-12-09 Panasonic Corporation Image rendering device, image rendering method, and image rendering program for rendering stereoscopic panoramic images
US9036000B1 (en) * 2011-09-27 2015-05-19 Google Inc. Street-level imagery acquisition and selection
US9055277B2 (en) 2011-03-31 2015-06-09 Panasonic Intellectual Property Management Co., Ltd. Image rendering device, image rendering method, and image rendering program for rendering stereoscopic images
US9354791B2 (en) 2013-06-20 2016-05-31 Here Global B.V. Apparatus, methods and computer programs for displaying images
US9361283B2 (en) 2011-11-30 2016-06-07 Google Inc. Method and system for projecting text onto surfaces in geographic imagery
US9632659B2 (en) 2008-02-27 2017-04-25 Google Inc. Using image content to facilitate navigation in panoramic image data
WO2017138801A1 (en) * 2016-02-12 2017-08-17 삼성전자 주식회사 Method and apparatus for processing 360-degree image
US9754413B1 (en) 2015-03-26 2017-09-05 Google Inc. Method and system for navigating in panoramic images using voxel maps
WO2017204491A1 (en) * 2016-05-26 2017-11-30 엘지전자 주식회사 Method for transmitting 360-degree video, method for receiving 360-degree video, apparatus for transmitting 360-degree video, and apparatus for receiving 360-degree video
EP2650844A3 (en) * 2012-04-09 2017-12-20 Nintendo Co., Ltd. Information processing apparatus, information processing program, and information processing method and system
WO2017217763A1 (en) * 2016-06-14 2017-12-21 Samsung Electronics Co., Ltd. Image processing apparatus and method
US9986140B2 (en) * 2013-11-21 2018-05-29 International Business Machines Corporation Utilizing metadata for automated photographic setup
US10078788B2 (en) 2010-09-21 2018-09-18 Mobileye Vision Technologies Ltd. Barrier and guardrail detection using a single camera
US10185463B2 (en) 2015-02-13 2019-01-22 Nokia Technologies Oy Method and apparatus for providing model-centered rotation in a three-dimensional user interface
EP3528212A4 (en) * 2016-10-12 2019-09-18 Samsung Electronics Co Ltd Method and apparatus for processing virtual reality image
WO2019235904A1 (en) * 2018-06-08 2019-12-12 엘지전자 주식회사 Method for processing overlay in 360 video system and device therefor

Families Citing this family (130)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA2559726C (en) 2004-03-24 2015-10-20 A9.Com, Inc. System and method for displaying images in an online directory
US8751156B2 (en) 2004-06-30 2014-06-10 HERE North America LLC Method of operating a navigation system using images
US7460953B2 (en) * 2004-06-30 2008-12-02 Navteq North America, Llc Method of operating a navigation system using images
USRE45264E1 (en) 2004-08-31 2014-12-02 Visual Real Estate, Inc. Methods and apparatus for generating three-dimensional image data models
US8195386B2 (en) * 2004-09-28 2012-06-05 National University Corporation Kumamoto University Movable-body navigation information display method and movable-body navigation information display unit
US7451041B2 (en) 2005-05-06 2008-11-11 Facet Technology Corporation Network-based navigation system having virtual drive-thru advertisements integrated with actual imagery from along a physical route
US9063633B2 (en) * 2006-03-30 2015-06-23 Arjuna Indraeswaran Rajasingham Virtual navigation system for virtual and real spaces
US8453060B2 (en) * 2006-08-25 2013-05-28 Microsoft Corporation Panoramic ring user interface
US7929800B2 (en) 2007-02-06 2011-04-19 Meadow William D Methods and apparatus for generating a continuum of image data
JPWO2008099483A1 (en) * 2007-02-15 2010-05-27 パイオニア株式会社 Display control apparatus, display control method, display control program, and recording medium
US8009178B2 (en) * 2007-06-29 2011-08-30 Microsoft Corporation Augmenting images for panoramic display
US20100122208A1 (en) * 2007-08-07 2010-05-13 Adam Herr Panoramic Mapping Display
US8345049B2 (en) * 2007-08-16 2013-01-01 International Business Machine Corporation Method and apparatus for predicting avatar movement in a virtual universe
US7990387B2 (en) 2007-08-16 2011-08-02 International Business Machines Corporation Method and apparatus for spawning projected avatars in a virtual universe
US9003304B2 (en) * 2007-08-16 2015-04-07 International Business Machines Corporation Method and apparatus for moving an avatar in a virtual universe
US9123159B2 (en) * 2007-11-30 2015-09-01 Microsoft Technology Licensing, Llc Interactive geo-positioning of imagery
US8127235B2 (en) 2007-11-30 2012-02-28 International Business Machines Corporation Automatic increasing of capacity of a virtual space in a virtual world
JP2009140119A (en) * 2007-12-05 2009-06-25 Sharp Corp Graphic display device and graphic display method
JP5239328B2 (en) * 2007-12-21 2013-07-17 ソニー株式会社 Information processing apparatus and touch motion recognition method
US20090167786A1 (en) * 2007-12-24 2009-07-02 Ronald Stanions Methods and apparatus for associating image data
JP4994256B2 (en) * 2008-01-28 2012-08-08 株式会社ジオ技術研究所 Data structure of route guidance database
JP5337170B2 (en) 2008-02-08 2013-11-06 グーグル インコーポレイテッド Panorama camera with multiple image sensors using timed shutters
US8207964B1 (en) 2008-02-22 2012-06-26 Meadow William D Methods and apparatus for generating three-dimensional image data models
US8671349B2 (en) * 2008-05-15 2014-03-11 International Business Machines Corporation Virtual universe teleportation suggestion service
US9019302B2 (en) * 2008-05-29 2015-04-28 Tomtom International B.V. Generating a display image
US20090315766A1 (en) * 2008-06-19 2009-12-24 Microsoft Corporation Source switching for devices supporting dynamic direction information
US8700301B2 (en) 2008-06-19 2014-04-15 Microsoft Corporation Mobile computing devices, architecture and user interfaces based on dynamic direction information
US20100009662A1 (en) * 2008-06-20 2010-01-14 Microsoft Corporation Delaying interaction with points of interest discovered based on directional device information
US20090315775A1 (en) * 2008-06-20 2009-12-24 Microsoft Corporation Mobile computing services based on devices with dynamic direction information
US8493408B2 (en) * 2008-11-19 2013-07-23 Apple Inc. Techniques for manipulating panoramas
US20110113315A1 (en) * 2008-12-31 2011-05-12 Microsoft Corporation Computer-assisted rich interactive narrative (rin) generation
US9683853B2 (en) * 2009-01-23 2017-06-20 Fuji Xerox Co., Ltd. Image matching in support of mobile navigation
JP5347716B2 (en) * 2009-05-27 2013-11-20 ソニー株式会社 Image processing apparatus, information processing method, and program
US8610741B2 (en) * 2009-06-02 2013-12-17 Microsoft Corporation Rendering aligned perspective images
US9298345B2 (en) 2009-06-23 2016-03-29 Microsoft Technology Licensing, Llc Block view for geographic navigation
JP5792424B2 (en) * 2009-07-03 2015-10-14 ソニー株式会社 Map information display device, map information display method, and program
US8872767B2 (en) 2009-07-07 2014-10-28 Microsoft Corporation System and method for converting gestures into digital graffiti
KR101005594B1 (en) 2009-07-15 2011-01-11 엔에이치엔(주) Method, system and computer-readable recording medium for providing aerial panoramic image
JP5464955B2 (en) * 2009-09-29 2014-04-09 株式会社ソニー・コンピュータエンタテインメント Panorama image display device
KR20110052124A (en) * 2009-11-12 2011-05-18 삼성전자주식회사 Method for generating and referencing panorama image and mobile terminal using the same
JPWO2011092746A1 (en) * 2010-01-29 2013-05-23 三菱電機株式会社 Map information processing device
US8773424B2 (en) * 2010-02-04 2014-07-08 Microsoft Corporation User interfaces for interacting with top-down maps of reconstructed 3-D scences
US8624902B2 (en) 2010-02-04 2014-01-07 Microsoft Corporation Transitioning between top-down maps and local navigation of reconstructed 3-D scenes
US8660316B2 (en) * 2010-03-04 2014-02-25 Navteq B.V. Navigating on images
US20110242271A1 (en) * 2010-03-31 2011-10-06 Google Inc. Synthesizing Panoramic Three-Dimensional Images
US20110273451A1 (en) * 2010-05-10 2011-11-10 Salemann Leo J Computer simulation of visual images using 2d spherical images extracted from 3d data
US9582166B2 (en) * 2010-05-16 2017-02-28 Nokia Technologies Oy Method and apparatus for rendering user interface for location-based service having main view portion and preview portion
US8295589B2 (en) 2010-05-20 2012-10-23 Microsoft Corporation Spatially registering user photographs
KR20110139375A (en) * 2010-06-23 2011-12-29 삼성전자주식회사 Method for displaying image including a position information and apparatus of enabling the method
US9864501B2 (en) * 2010-07-30 2018-01-09 Apaar Tuli Displaying information
US8797315B1 (en) * 2010-09-29 2014-08-05 Google Inc. Segmented editor for tours of a geographic information system, and applications thereof
US8780174B1 (en) 2010-10-12 2014-07-15 The Boeing Company Three-dimensional vision system for displaying images taken from a moving vehicle
US9342998B2 (en) * 2010-11-16 2016-05-17 Microsoft Technology Licensing, Llc Techniques to annotate street view images with contextual information
DE202011110906U1 (en) 2010-11-24 2017-02-27 Google Inc. Path planning for street level navigation in a three dimensional environment and applications thereof
WO2012071435A1 (en) 2010-11-24 2012-05-31 Google Inc. Rendering and navigating photographic panoramas with depth information in a geographic information system
EP2643822B1 (en) 2010-11-24 2017-03-22 Google, Inc. Guided navigation through geo-located panoramas
CN102129717A (en) * 2011-03-09 2011-07-20 上海诚明融鑫科技有限公司 Method and system for browsing panorama by hot spot in three-dimensional panoramic space
CN102736882B (en) * 2011-04-07 2015-08-05 株式会社东芝 Mobile terminal device and display control method, display system
EP3579178A1 (en) * 2011-04-12 2019-12-11 Google LLC Integrating maps and street views
US20120268485A1 (en) 2011-04-22 2012-10-25 Panasonic Corporation Visualization of Query Results in Relation to a Map
US9692965B2 (en) * 2011-04-25 2017-06-27 Mitsuo Hayashi Omnidirectional image editing program and omnidirectional image editing apparatus
US9274595B2 (en) 2011-08-26 2016-03-01 Reincloud Corporation Coherent presentation of multiple reality and interaction models
US20130106990A1 (en) 2011-11-01 2013-05-02 Microsoft Corporation Planar panorama imagery generation
CN103135970A (en) * 2011-11-28 2013-06-05 鸿富锦精密工业(深圳)有限公司 Product partial reporting system and product partial reporting method
US9870429B2 (en) 2011-11-30 2018-01-16 Nokia Technologies Oy Method and apparatus for web-based augmented reality application viewer
US10008021B2 (en) 2011-12-14 2018-06-26 Microsoft Technology Licensing, Llc Parallax compensation
US9965140B2 (en) * 2011-12-26 2018-05-08 TrackThings LLC Method and apparatus of a marking objects in images displayed on a portable unit
US9396577B2 (en) 2012-02-16 2016-07-19 Google Inc. Using embedded camera parameters to determine a position for a three-dimensional model
US9349195B2 (en) * 2012-03-19 2016-05-24 Google Inc. Apparatus and method for spatially referencing images
US20130257742A1 (en) * 2012-03-28 2013-10-03 Google Inc. Method and System for Controlling Imagery Panning Based on Displayed Content
CN102708530A (en) * 2012-04-17 2012-10-03 江南大学 Municipal online virtual display method based on panoramic platform
CN103455480B (en) * 2012-05-28 2017-03-08 深圳市腾讯计算机系统有限公司 A kind of position search method based on electronic chart and device
JP2015156523A (en) * 2012-06-06 2015-08-27 ソニー株式会社 Image processing device, image processing method, and program
US9043722B1 (en) 2012-06-19 2015-05-26 Surfwax, Inc. User interfaces for displaying relationships between cells in a grid
US9619138B2 (en) * 2012-06-19 2017-04-11 Nokia Corporation Method and apparatus for conveying location based images based on a field-of-view
US9786097B2 (en) 2012-06-22 2017-10-10 Matterport, Inc. Multi-modal method for interacting with 3D models
US10139985B2 (en) * 2012-06-22 2018-11-27 Matterport, Inc. Defining, displaying and interacting with tags in a three-dimensional model
US9646522B2 (en) * 2012-06-29 2017-05-09 Intel Corporation Enhanced information delivery using a transparent display
US20150154784A1 (en) * 2012-07-20 2015-06-04 Google Inc. Use of Photo Animation Transitions to Mask Latency
US9175975B2 (en) 2012-07-30 2015-11-03 RaayonNova LLC Systems and methods for navigation
KR20140055132A (en) * 2012-10-30 2014-05-09 삼성전자주식회사 Method and apparatus for processing webpage in terminal using cloud server
US9245372B2 (en) * 2012-12-04 2016-01-26 Nintendo Co., Ltd. Map systems and methods for displaying panoramic images
JP5745497B2 (en) * 2012-12-04 2015-07-08 任天堂株式会社 Display system, display control apparatus, information processing program, and display method
JP6030935B2 (en) * 2012-12-04 2016-11-24 任天堂株式会社 Information processing program, display control apparatus, display system, and display method
JP2014132325A (en) * 2012-12-04 2014-07-17 Nintendo Co Ltd Information processing system, information processor, program and display method
JP6075066B2 (en) 2012-12-28 2017-02-08 株式会社リコー Image management system, image management method, and program
JP6022386B2 (en) * 2013-03-21 2016-11-09 株式会社ジオ技術研究所 3D map display device, 3D map display method, and computer program
CN104180814A (en) * 2013-05-22 2014-12-03 北京百度网讯科技有限公司 Navigation method in live-action function on mobile terminal, and electronic map client
US9310987B2 (en) 2013-08-19 2016-04-12 Google Inc. Projections to fix pose of panoramic photos
US9430990B2 (en) 2013-08-30 2016-08-30 International Business Machines Corporation Presenting a data in a graphical overlay
US20150062173A1 (en) * 2013-08-30 2015-03-05 International Business Machines Corporation Method to Visualize Semantic Data in Contextual Window
US10168871B2 (en) * 2013-09-16 2019-01-01 Rovi Guides, Inc. Methods and systems for presenting direction-specific media assets
CN104636358B (en) * 2013-11-08 2018-02-27 联想(北京)有限公司 A kind of map stacking method and electronic equipment
US20150130843A1 (en) * 2013-11-14 2015-05-14 Microsoft Corporation Lens view for map
US10311633B2 (en) 2014-01-17 2019-06-04 Nokia Technologies Oy Method and apparatus for visualization of geo-located media contents in 3D rendering applications
US20150206337A1 (en) * 2014-01-17 2015-07-23 Nokia Corporation Method and apparatus for visualization of geo-located media contents in 3d rendering applications
US10163261B2 (en) 2014-03-19 2018-12-25 Matterport, Inc. Selecting two-dimensional imagery data for display within a three-dimensional model
US9972121B2 (en) * 2014-04-22 2018-05-15 Google Llc Selecting time-distributed panoramic images for display
USD780777S1 (en) 2014-04-22 2017-03-07 Google Inc. Display screen with graphical user interface or portion thereof
USD781317S1 (en) 2014-04-22 2017-03-14 Google Inc. Display screen with graphical user interface or portion thereof
USD781318S1 (en) 2014-04-22 2017-03-14 Google Inc. Display screen with graphical user interface or portion thereof
US9934222B2 (en) 2014-04-22 2018-04-03 Google Llc Providing a thumbnail image that follows a main image
US9508195B2 (en) * 2014-09-03 2016-11-29 Microsoft Technology Licensing, Llc Management of content in a 3D holographic environment
US20180373483A1 (en) * 2014-12-08 2018-12-27 Hirohisa Inamoto Image management system, image management method, and program
US9984494B2 (en) 2015-01-26 2018-05-29 Uber Technologies, Inc. Map-like summary visualization of street-level distance data and panorama data
US10027887B1 (en) 2015-01-30 2018-07-17 Google Llc Dynamic 3D panoramas
US20160255271A1 (en) * 2015-02-27 2016-09-01 International Business Machines Corporation Interactive surveillance overlay
US9471954B2 (en) * 2015-03-16 2016-10-18 International Business Machines Corporation Video sequence assembly
US9934608B2 (en) * 2015-05-29 2018-04-03 Hover Inc. Graphical overlay guide for interface
US10410412B2 (en) * 2015-05-29 2019-09-10 Hover Inc. Real-time processing of captured building imagery
US10038838B2 (en) * 2015-05-29 2018-07-31 Hover Inc. Directed image capture
US10410413B2 (en) 2015-05-29 2019-09-10 Hover Inc. Image capture for a multi-dimensional building model
US10127722B2 (en) 2015-06-30 2018-11-13 Matterport, Inc. Mobile capture visualization incorporating three-dimensional and two-dimensional imagery
US9551579B1 (en) * 2015-08-07 2017-01-24 Google Inc. Automatic connection of images using visual features
US9934823B1 (en) * 2015-08-27 2018-04-03 Amazon Technologies, Inc. Direction indicators for panoramic images
US20170060404A1 (en) * 2015-08-28 2017-03-02 Facebook, Inc. Systems and methods for providing interactivity for panoramic media content
US10521100B2 (en) * 2015-08-28 2019-12-31 Facebook, Inc. Systems and methods for providing interactivity for panoramic media content
US10521099B2 (en) * 2015-08-28 2019-12-31 Facebook, Inc. Systems and methods for providing interactivity for panoramic media content
CN105306887A (en) * 2015-09-21 2016-02-03 北京奇虎科技有限公司 Method and device for sharing panoramic data
US10217283B2 (en) 2015-12-17 2019-02-26 Google Llc Navigation through multidimensional images spaces
IN2015CH06769A (en) * 2015-12-18 2016-01-15 Wipro Ltd
US10147224B2 (en) * 2016-02-16 2018-12-04 Samsung Electronics Co., Ltd. Method and apparatus for generating omni media texture mapping metadata
US20170255372A1 (en) * 2016-03-07 2017-09-07 Facebook, Inc. Systems and methods for presenting content
US10366144B2 (en) 2016-04-01 2019-07-30 Ebay Inc. Analyzing and linking a set of images by identifying objects in each image to determine a primary image and a secondary image
EP3387622A1 (en) 2016-05-03 2018-10-17 Google LLC Method and system for obtaining pair-wise epipolar constraints and solving for panorama pose on a mobile device
CN106658009A (en) * 2016-12-29 2017-05-10 上海国茂数字技术有限公司 Improved double-ring sampling method and device for panoramic video
CN108882018A (en) * 2017-05-09 2018-11-23 合信息技术(北京)有限公司 Video playing, data offering method, client and server in virtual scene
US10417276B2 (en) * 2017-05-15 2019-09-17 Adobe, Inc. Thumbnail generation from panoramic images
KR20190074490A (en) * 2017-12-20 2019-06-28 삼성전자주식회사 Image processing apparatus and method for image processing thereof
EP3588249A1 (en) * 2018-06-26 2020-01-01 Koninklijke Philips N.V. Apparatus and method for generating images of a scene

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020063725A1 (en) * 2000-11-30 2002-05-30 Virtuallylisted Llc Method and apparatus for capturing and presenting panoramic images for websites
US6486877B1 (en) * 1998-06-17 2002-11-26 Olympus Optical Company, Ltd. Method and apparatus for creating virtual environment, and record medium having recorded thereon computer readable program for creating virtual environment
US20040107043A1 (en) * 2002-11-29 2004-06-03 De Silva Andrew S. Navigation method and system
US20040257436A1 (en) * 1997-04-21 2004-12-23 Sony Corporation Controller for photographing apparatus and photographing system
US20060268153A1 (en) * 2005-05-11 2006-11-30 Xenogen Corporation Surface contruction using combined photographic and structured light information

Family Cites Families (61)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2900632B2 (en) 1991-04-19 1999-06-02 株式会社日立製作所 Display method of digital map processing apparatus and a digital map
US5396583A (en) 1992-10-13 1995-03-07 Apple Computer, Inc. Cylindrical to planar image mapping using scanline coherence
US5559707A (en) * 1994-06-24 1996-09-24 Delorme Publishing Company Computer aided routing system
US6195122B1 (en) 1995-01-31 2001-02-27 Robert Vincent Spatial referenced photography
US6097393A (en) 1996-09-03 2000-08-01 The Takshele Corporation Computer-executed, three-dimensional graphical resource management process and system
US6285317B1 (en) * 1998-05-01 2001-09-04 Lucent Technologies Inc. Navigation system with three-dimensional display
US6687753B2 (en) * 1998-06-25 2004-02-03 International Business Machines Corporation Method and system for providing three-dimensional graphics over computer networks
US6477268B1 (en) * 1998-11-17 2002-11-05 Industrial Technology Research Institute Producing transitions between vistas
US20030210228A1 (en) * 2000-02-25 2003-11-13 Ebersole John Franklin Augmented reality situational awareness system and method
US6346938B1 (en) * 1999-04-27 2002-02-12 Harris Corporation Computer-resident mechanism for manipulating, navigating through and mensurating displayed image of three-dimensional geometric model
US7574381B1 (en) * 1999-08-06 2009-08-11 Catherine Lin-Hendel System and method for constructing and displaying active virtual reality cyber malls, show rooms, galleries, stores, museums, and objects within
US6563529B1 (en) * 1999-10-08 2003-05-13 Jerry Jongerius Interactive system for displaying detailed view and direction in panoramic images
US6515664B1 (en) 1999-11-12 2003-02-04 Pixaround.Com Pte Ltd Fast single-pass cylindrical to planar projection
US6771304B1 (en) 1999-12-31 2004-08-03 Stmicroelectronics, Inc. Perspective correction device for panoramic digital camera
US6717608B1 (en) 1999-12-31 2004-04-06 Stmicroelectronics, Inc. Motion estimation for panoramic digital camera
US6885392B1 (en) 1999-12-31 2005-04-26 Stmicroelectronics, Inc. Perspective correction for preview area of panoramic digital camera
US20020010734A1 (en) * 2000-02-03 2002-01-24 Ebersole John Franklin Internetworked augmented reality system and method
US6606091B2 (en) 2000-02-07 2003-08-12 Siemens Corporate Research, Inc. System for interactive 3D object extraction from slice-based medical images
US6895126B2 (en) 2000-10-06 2005-05-17 Enrico Di Bernardo System and method for creating, storing, and utilizing composite images of a geographic location
US7006707B2 (en) * 2001-05-03 2006-02-28 Adobe Systems Incorporated Projecting images onto a surface
JP2003006680A (en) 2001-06-20 2003-01-10 Zenrin Co Ltd Method for generating three-dimensional electronic map data
CA2459748A1 (en) 2001-09-07 2003-03-20 The General Hospital Corporation Medical procedure training system
US7096428B2 (en) * 2001-09-28 2006-08-22 Fuji Xerox Co., Ltd. Systems and methods for providing a spatially indexed panoramic video
US7274380B2 (en) * 2001-10-04 2007-09-25 Siemens Corporate Research, Inc. Augmented reality system
JP2003141562A (en) 2001-10-29 2003-05-16 Sony Corp Image processing apparatus and method for nonplanar image, storage medium, and computer program
JP4610156B2 (en) 2001-11-13 2011-01-12 アルスロン株式会社 A system for grasping the space of roads, rivers, and railways using panoramic images
JP2003172899A (en) 2001-12-05 2003-06-20 Fujitsu Ltd Display device
US7411594B2 (en) * 2002-01-15 2008-08-12 Canon Kabushiki Kaisha Information processing apparatus and method
CN1162806C (en) 2002-03-07 2004-08-18 上海交通大学 Shooting, formation, transmission and display method of road overall view image tape
JP4487775B2 (en) 2002-10-15 2010-06-23 セイコーエプソン株式会社 Panorama composition processing of multiple image data
US7424133B2 (en) 2002-11-08 2008-09-09 Pictometry International Corporation Method and apparatus for capturing, geolocating and measuring oblique images
KR100464342B1 (en) 2003-04-14 2005-01-03 삼성전자주식회사 To-can type optical module
US7526718B2 (en) * 2003-04-30 2009-04-28 Hewlett-Packard Development Company, L.P. Apparatus and method for recording “path-enhanced” multimedia
US6968973B2 (en) * 2003-05-31 2005-11-29 Microsoft Corporation System and process for viewing and navigating through an interactive video tour
KR100703444B1 (en) * 2003-06-03 2007-04-03 삼성전자주식회사 Device and method for downloading and displaying a images of global position information in navigation system
JP4321128B2 (en) 2003-06-12 2009-08-26 株式会社デンソー Image server, image collection device, and image display terminal
JP2005149409A (en) * 2003-11-19 2005-06-09 Canon Inc Image reproduction method and apparatus
CA2455359C (en) 2004-01-16 2013-01-08 Geotango International Corp. System, computer program and method for 3d object measurement, modeling and mapping from single imagery
JP4437677B2 (en) 2004-03-01 2010-03-24 三菱電機株式会社 Landscape display device
EP1741064A4 (en) 2004-03-23 2010-10-06 Google Inc A digital mapping system
DE102004020861B4 (en) 2004-04-28 2009-10-01 Siemens Ag Method for the reconstruction of projection data sets with dose-reduced section-wise spiral scanning in computed tomography
US7746376B2 (en) * 2004-06-16 2010-06-29 Felipe Mendoza Method and apparatus for accessing multi-dimensional mapping and information
US20060002590A1 (en) 2004-06-30 2006-01-05 Borak Jason M Method of collecting information for a geographic database for use with a navigation system
US7460953B2 (en) * 2004-06-30 2008-12-02 Navteq North America, Llc Method of operating a navigation system using images
JP4130428B2 (en) 2004-09-02 2008-08-06 ザイオソフト株式会社 Image processing method and image processing program
US8195386B2 (en) * 2004-09-28 2012-06-05 National University Corporation Kumamoto University Movable-body navigation information display method and movable-body navigation information display unit
JP2006105640A (en) 2004-10-01 2006-04-20 Hitachi Ltd Navigation system
US7529552B2 (en) * 2004-10-05 2009-05-05 Isee Media Inc. Interactive imaging for cellular phones
EP1820159A1 (en) * 2004-11-12 2007-08-22 MOK3, Inc. Method for inter-scene transitions
FR2879791B1 (en) 2004-12-16 2007-03-16 Cnes Epic Method for processing images using automatic georeferencing of images from a couple of images taken in the same focal plan
US7466244B2 (en) * 2005-04-21 2008-12-16 Microsoft Corporation Virtual earth rooftop overlay and bounding
US20070035563A1 (en) * 2005-08-12 2007-02-15 The Board Of Trustees Of Michigan State University Augmented reality spatial interaction and navigational system
WO2007032553A1 (en) 2005-09-14 2007-03-22 Sony Corporation Audio-visual system
JP4853149B2 (en) 2005-09-14 2012-01-11 ソニー株式会社 Image processing apparatus, image display apparatus, image processing method, program, and recording medium
US8160400B2 (en) * 2005-11-17 2012-04-17 Microsoft Corporation Navigating images using image based geometric alignment and object based controls
US7643673B2 (en) 2006-06-12 2010-01-05 Google Inc. Markup language for interactive geographic information system
MX2009005351A (en) 2006-11-20 2009-06-08 Satori Pharmaceuticals Inc Compounds useful for treating neurodegenerative disorders.
US20080143709A1 (en) * 2006-12-14 2008-06-19 Earthmine, Inc. System and method for accessing three dimensional information from a panoramic image
US20080189031A1 (en) * 2007-02-06 2008-08-07 Meadow William D Methods and apparatus for presenting a continuum of image data
US20080309668A1 (en) * 2007-05-17 2008-12-18 Igor Borovikov Image processing method and apparatus
JP5039922B2 (en) * 2008-03-21 2012-10-03 インターナショナル・ビジネス・マシーンズ・コーポレーションInternational Business Maschines Corporation Image drawing system, image drawing server, image drawing method, and computer program

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040257436A1 (en) * 1997-04-21 2004-12-23 Sony Corporation Controller for photographing apparatus and photographing system
US6486877B1 (en) * 1998-06-17 2002-11-26 Olympus Optical Company, Ltd. Method and apparatus for creating virtual environment, and record medium having recorded thereon computer readable program for creating virtual environment
US20020063725A1 (en) * 2000-11-30 2002-05-30 Virtuallylisted Llc Method and apparatus for capturing and presenting panoramic images for websites
US20040107043A1 (en) * 2002-11-29 2004-06-03 De Silva Andrew S. Navigation method and system
US20060268153A1 (en) * 2005-05-11 2006-11-30 Xenogen Corporation Surface contruction using combined photographic and structured light information

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Thomas H. Kolbe, Bonn, "Augmented Videos and Panoramas for Pedestrian Navigation", Proceedings of the 2nd Symposium on Location Based Services & TeleCartography 2004 *

Cited By (41)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10163263B2 (en) 2008-02-27 2018-12-25 Google Llc Using image content to facilitate navigation in panoramic image data
US9632659B2 (en) 2008-02-27 2017-04-25 Google Inc. Using image content to facilitate navigation in panoramic image data
US20100161207A1 (en) * 2008-12-24 2010-06-24 In-Young Do Mobile terminal and method for providing location-based service thereof
US9372094B2 (en) * 2009-12-14 2016-06-21 Nokia Technologies Oy Method and apparatus for correlating and navigating between a live image and a prerecorded panoramic image
US20110141141A1 (en) * 2009-12-14 2011-06-16 Nokia Corporation Method and apparatus for correlating and navigating between a live image and a prerecorded panoramic image
US9766089B2 (en) * 2009-12-14 2017-09-19 Nokia Technologies Oy Method and apparatus for correlating and navigating between a live image and a prerecorded panoramic image
US20120033032A1 (en) * 2009-12-14 2012-02-09 Nokia Corporation Method and apparatus for correlating and navigating between a live image and a prerecorded panoramic image
US20140161323A1 (en) * 2010-09-21 2014-06-12 Mobileye Technologies Limited Dense structure from motion
US10445595B2 (en) 2010-09-21 2019-10-15 Mobileye Vision Technologies Ltd. Barrier and guardrail detection using a single camera
US10115027B2 (en) 2010-09-21 2018-10-30 Mibileye Vision Technologies Ltd. Barrier and guardrail detection using a single camera
US9959595B2 (en) * 2010-09-21 2018-05-01 Mobileye Vision Technologies Ltd. Dense structure from motion
US10078788B2 (en) 2010-09-21 2018-09-18 Mobileye Vision Technologies Ltd. Barrier and guardrail detection using a single camera
US9491430B2 (en) * 2011-02-24 2016-11-08 Nintendo Co., Ltd. Storage medium having stored therein display control program, display control apparatus, display control system, and display control method
US20120218266A1 (en) * 2011-02-24 2012-08-30 Nintendo Co., Ltd. Storage medium having stored therein display control program, display control apparatus, display control system, and display control method
US9055277B2 (en) 2011-03-31 2015-06-09 Panasonic Intellectual Property Management Co., Ltd. Image rendering device, image rendering method, and image rendering program for rendering stereoscopic images
US8907968B2 (en) 2011-03-31 2014-12-09 Panasonic Corporation Image rendering device, image rendering method, and image rendering program for rendering stereoscopic panoramic images
US9036000B1 (en) * 2011-09-27 2015-05-19 Google Inc. Street-level imagery acquisition and selection
US9489716B1 (en) * 2011-09-27 2016-11-08 Google Inc. Street-level imagery acquisition and selection
US20130083055A1 (en) * 2011-09-30 2013-04-04 Apple Inc. 3D Position Tracking for Panoramic Imagery Navigation
US9121724B2 (en) * 2011-09-30 2015-09-01 Apple Inc. 3D position tracking for panoramic imagery navigation
US8306749B1 (en) 2011-11-22 2012-11-06 Google Inc. Systems and methods for route summarization
US8583374B1 (en) 2011-11-22 2013-11-12 Google Inc. Systems and methods for route summarization
US9361283B2 (en) 2011-11-30 2016-06-07 Google Inc. Method and system for projecting text onto surfaces in geographic imagery
US8767040B2 (en) 2012-01-11 2014-07-01 Google Inc. Method and system for displaying panoramic imagery
US9007440B1 (en) 2012-01-11 2015-04-14 Google Inc. Method and system for displaying panoramic imagery
EP2650844A3 (en) * 2012-04-09 2017-12-20 Nintendo Co., Ltd. Information processing apparatus, information processing program, and information processing method and system
US9164972B2 (en) * 2012-06-07 2015-10-20 Microsoft Technology Licensing, Llc Managing objects in panorama display to navigate spreadsheet
US20130332810A1 (en) * 2012-06-07 2013-12-12 Microsoft Corporation Managing objects in panorama display to navigate spreadsheet
WO2014008320A1 (en) * 2012-07-03 2014-01-09 Tourwrist, Inc. Systems and methods for capture and display of flex-focus panoramas
US20140169699A1 (en) * 2012-09-21 2014-06-19 Tamaggo Inc. Panoramic image viewer
US8570329B1 (en) * 2012-10-31 2013-10-29 Google Inc. Subtle camera motions to indicate imagery type in a mapping system
US9354791B2 (en) 2013-06-20 2016-05-31 Here Global B.V. Apparatus, methods and computer programs for displaying images
US9986140B2 (en) * 2013-11-21 2018-05-29 International Business Machines Corporation Utilizing metadata for automated photographic setup
US10185463B2 (en) 2015-02-13 2019-01-22 Nokia Technologies Oy Method and apparatus for providing model-centered rotation in a three-dimensional user interface
US9754413B1 (en) 2015-03-26 2017-09-05 Google Inc. Method and system for navigating in panoramic images using voxel maps
US10186083B1 (en) 2015-03-26 2019-01-22 Google Llc Method and system for navigating in panoramic images using voxel maps
WO2017138801A1 (en) * 2016-02-12 2017-08-17 삼성전자 주식회사 Method and apparatus for processing 360-degree image
WO2017204491A1 (en) * 2016-05-26 2017-11-30 엘지전자 주식회사 Method for transmitting 360-degree video, method for receiving 360-degree video, apparatus for transmitting 360-degree video, and apparatus for receiving 360-degree video
WO2017217763A1 (en) * 2016-06-14 2017-12-21 Samsung Electronics Co., Ltd. Image processing apparatus and method
EP3528212A4 (en) * 2016-10-12 2019-09-18 Samsung Electronics Co Ltd Method and apparatus for processing virtual reality image
WO2019235904A1 (en) * 2018-06-08 2019-12-12 엘지전자 주식회사 Method for processing overlay in 360 video system and device therefor

Also Published As

Publication number Publication date
US8982154B2 (en) 2015-03-17
US20150161820A1 (en) 2015-06-11
US20140160119A1 (en) 2014-06-12
US20080291217A1 (en) 2008-11-27
US7990394B2 (en) 2011-08-02

Similar Documents

Publication Publication Date Title
KR101013633B1 (en) System and method for advanced 3d visualization for mobile navigation units
US8565514B2 (en) Markup language for interactive geographic information system
US7933395B1 (en) Virtual tour of user-defined paths in a geographic information system
CA2753419C (en) System and method of indicating transition between street level images
US9361943B2 (en) System and method for tagging objects in a panoramic video and associating functions and indexing panoramic images with same
US8990239B2 (en) Systems and methods for photograph mapping
US10163263B2 (en) Using image content to facilitate navigation in panoramic image data
Sarkar et al. Stretching the rubber sheet: a metaphor for viewing large layouts on small screens
CA2812008C (en) Methods and apparatus for navigating an image
US8872847B2 (en) Architectures and methods for creating and representing time-dependent imagery
JP5899232B2 (en) Navigation with guidance through geographically located panoramas
US6563529B1 (en) Interactive system for displaying detailed view and direction in panoramic images
Zheng Digital route panoramas
JP2014511534A (en) System and method for navigating a three-dimensional environment using a multi-input interface
JP2004523036A (en) Image-based digital representation of landscape
US10032306B2 (en) Method for inter-scene transitions
JP2004213663A (en) Navigation system
US8893026B2 (en) System and method for creating and broadcasting interactive panoramic walk-through applications
US9369679B2 (en) System and process for projecting location-referenced panoramic images into a 3-D environment model and rendering panoramic images from arbitrary viewpoints within the 3-D environment model
US9396578B2 (en) Method and system for displaying visual content in a virtual three-dimensional space
US7990397B2 (en) Image-mapped point cloud with ability to accurately represent point coordinates
US9092899B1 (en) Automatic method for photo texturing geolocated 3D models from geolocated imagery
JP2004534960A (en) Elastic display space
Cohen et al. Harold: A world made of drawings
US20100325589A1 (en) Block view for geographic navigation

Legal Events

Date Code Title Description
AS Assignment

Owner name: GOOGLE INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:VINCENT, LUC;CHAU, STEPHEN;FILIP, DANIEL;AND OTHERS;SIGNING DATES FROM 20070809 TO 20070906;REEL/FRAME:026526/0124

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION

AS Assignment

Owner name: GOOGLE LLC, CALIFORNIA

Free format text: CHANGE OF NAME;ASSIGNOR:GOOGLE INC.;REEL/FRAME:044142/0357

Effective date: 20170929