WO2018022977A1 - Systems and methods for providing individual and/or synchronized virtual tours through a realm for a group of users - Google Patents

Systems and methods for providing individual and/or synchronized virtual tours through a realm for a group of users Download PDF

Info

Publication number
WO2018022977A1
WO2018022977A1 PCT/US2017/044342 US2017044342W WO2018022977A1 WO 2018022977 A1 WO2018022977 A1 WO 2018022977A1 US 2017044342 W US2017044342 W US 2017044342W WO 2018022977 A1 WO2018022977 A1 WO 2018022977A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
realm
computing device
server
location
Prior art date
Application number
PCT/US2017/044342
Other languages
French (fr)
Inventor
James Schoonmaker
Matthew Dunn
Keno MULLINGS
Stephen SEIBERT
Original Assignee
Everyscape, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US15/224,077 external-priority patent/US11153355B2/en
Priority claimed from US15/224,068 external-priority patent/US20180035048A1/en
Application filed by Everyscape, Inc. filed Critical Everyscape, Inc.
Priority to CA3071398A priority Critical patent/CA3071398A1/en
Priority to EP17749318.6A priority patent/EP3491589A1/en
Publication of WO2018022977A1 publication Critical patent/WO2018022977A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management

Definitions

  • the present invention relates to providing individual and/or synchronized virtual tours through locations within a realm for a group of users within a networked environment, and more particularly, to allowing users to either select their own locations or rely on other users to select locations for experiencing the realm.
  • the house hunter may view the images on his or her own.
  • other parties such as real estate agents, cannot share the house hunter's experience of the realm and cannot control what the house hunter views. In this manner, each individual may have a unique, personal experience of the realm that is not readily communicated with others.
  • a method coordinates experiences of a realm for a group of users, including at least an administrative user and at least one participating user.
  • the method includes transmitting, from the server to an administrative user, a session identifier associated with the realm.
  • the method also includes receiving, by the server, a user acceptance to join a session using the session identifier so that the at least one participating user can experience the realm.
  • the method also includes transmitting to a computing device of the participating user, by the server, executable code including data describing a series of locations in the realm.
  • the series of locations comprises at least a first location and a second location, and the first location is associated with a first panoramic image and the second location is associated with a second panoramic image.
  • the executable code is configured to receive instructions from at least one computing device of at least one user within the group to display the realm at at least one selected location, and display, on the computing device, data corresponding to the realm at at least one of the locations selected by at least one of the users within the group.
  • the executable code is further configured to receive instructions from the server regarding locations selected by a user within the group.
  • the method may include receiving, by the server from the administrative user, a request to synchronize an experience of the realm for the participating user with an experience of the realm for the administrative user.
  • the request may include a location in the realm selected by the administrative user.
  • the method may include transmitting, by the server, the location in the realm selected by the administrative user to a computing device of the participating user.
  • the location causes the executable code running on the computing device of the participating user to display data corresponding to the location selected by the administrative user.
  • the method may include transmitting, by the server to a computing device of the participating user, locations in the realm selected by the users of the group. Additionally, the method may include receiving, by the server from the administrative user, a request to synchronize an experience of the realm for the participating user with an experience of the realm for the administrative user. In response to the request from the administrative user, the method may include transmitting, by the server to the computing device of the participating user, an instruction to display data corresponding to the locations in the realm selected by the administrative user.
  • the computing device of the participating user may create at least one virtual image representing a view of the realm between a location selected by the participating user and the location selected by the administrative user. Also, the computing device may display data corresponding to the virtual image and the location selected by the administrative user to simulate motion in the realm from the location selected by the participating user to the location selected by the administrative user.
  • the method may include receiving, by the server from the computing device of the administrative user, an updated location selected by the administrative user.
  • the method may also include transmitting, by the server to the computing device of the participating user, the updated selected location so that the computing device of the participating user displays data corresponding to the updated selected location in the realm.
  • the instruction to synchronize the experiences of the participating user and the administrative user causes the computing device of the participating user to disregard instructions from the participating user.
  • the method may include receiving, by the server from a first user in the group, a request to synchronize an experience of the realm for a second user in the group with an experience of the realm for the first user.
  • the selected location in the realm includes a view direction.
  • the method may include receiving, by the server from an administrative user, a request to view a list of users who have joined the experience of the realm. Also, the method may include transmitting, by the server to a computing device of the administrative user, the list of users for display.
  • the method includes receiving, by the server from the administrative user, a request to view a history of selected locations in the realm for a user in the group of users.
  • the method may include transmitting, by the server to a computing device of the administrative user, the history of selected locations in the realm for the user.
  • the method may include receiving, by the server from the administrative user, a request to follow the view of the participating user.
  • the method may also include sending, by the server to the computing device of the participating user, an instruction to provide, on a continual basis, updates on the location in the realm selected by the participating user.
  • the method may further include forwarding, by the server to the computing device of the administrative user, the updated location selected by the participating user.
  • the updated location causes the executable code running on the computing device of the administrative user to display data corresponding to the realm at the updated selected location of the participating user.
  • the method also includes transmitting, by the server to a computing device of the administrative user, locations in the realm selected by the users of the group.
  • the executable code on the computing device of the administrative user may be configured to receive a request to follow the view of the participating user and display data corresponding to the locations in the realm received from the server that are selected by the participating user.
  • the method may include receiving, by the server from an administrative user, an instruction to display a second realm to the group of users. Additionally, the method may include transmitting, by the server to each computing device for the group of users, executable code including data describing a series of locations in the second realm.
  • Fig. 1 depicts a block diagram representing a tour platform including at least one server that provides individual and synchronized experiences of a realm for a group of users;
  • FIG. 2A is an exemplary flow diagram depicting the initiation of a session in which a group of users experience the same realm
  • Fig. 2B is an exemplary flow diagram depicting the initiation of the session of Fig. 2A, in which users bypass the server to update one another regarding their locations in the realm;
  • FIG. 3 depicts exemplary relationships between images in a set of images for a realm
  • Fig. 4A is an exemplary flow diagram depicting one user commanding and controlling the experience of the realm for another user;
  • Fig. 4B is an exemplary flow diagram depicting one user bypassing the server to directly command and control the experience of the realm for another user;
  • Fig. 5 is an exemplary flow diagram depicting the manner in which the image data for one user is updated, based on the location of another user;
  • FIG. 6 A is an exemplary flow diagram depicting a user's selection of a new realm for a session, and the propagation of that realm to the other users in the group;
  • Fig. 6B is an exemplary flow diagram depicting the initiation of the session of Fig. 6A, in which users bypass the server to update one another regarding their locations in the realm;
  • Fig. 7 is an exemplary flow diagram for creating a session history in real time.
  • Fig. 8 depicts a platform for an exemplary apparatus that leverages features of panoramic image capturing programs on mobile computing devices to facilitate the capture of panoramic images;
  • Fig. 9 depicts an isometric view of a receptacle for the mobile computing device when mounted on a rotating member of a platform;
  • Fig. 10 depicts exemplary elements of the platform of Fig. 8;
  • Figs. 11-13 depict side, front, and top views of the receptacle and rotating member, illustrating the configuration of the receptacle to offset the mobile computing device from the axis of rotation of the member;
  • Figs. 14-17 depict top, side, bottom, and isometric views of an exemplary set of gears that may be used in the platform of the apparatus of Fig. 8.
  • a "realm" means a three-dimensional space. Parts of a realm need not be contiguous - realms may include discontinuities. While a realm may represent an actual physical space, any part of a realm may be imaginary.
  • a "set” includes at least one member.
  • FIGS. 1-7 the following section describes various embodiments relating to Systems and Methods for Enabling Shared Experiences of a Realm for Users in a Group. Reference numerals recited in this section relate to FIGs. 1-7. The fact that the same reference numeral is used in the section below relating to an
  • the inventors have developed systems and methods that enable users in a group to experience the same realm concurrently, and also to synchronize, either by sharing or commanding and controlling, their experiences of the realm.
  • the systems include a tour platform, with at least one server.
  • the tour platform stores sets of images, each set corresponding to a different realm.
  • a user selects a realm and creates a session for experiencing the realm.
  • the tour platform transmits code, which includes the set of images for the realm, to the computing devices of the users.
  • the devices store and execute the code, thereby enable the user to interact with the images to experience the realm.
  • Local execution of the code improves the speed, performance, and overall quality of the user's experiences. Moreover, this local execution enables the tour platform to synchronize experiences of the realm for two or more users efficiently simply by transmitting the locations of interest within the realm to the computing devices of the users, instead of the full images corresponding to the locations.
  • the users' computing devices may process the locations in conjunction with the set of images for the realm to identify the view (e.g., image data) to display to the user.
  • the systems and methods described herein decrease the likelihood of disruption in a user's experience, due to errors in image data transmission or lapses in connectivity over the network.
  • Fig. 1 depicts a block diagram representing a tour platform 100 including at least one server 105 that provides individual and synchronized experiences of a realm for a group of users via each user's respective computing device 110. Although operations described herein are discussed with respect to the server 105, any given operation may be performed by one or more servers in a group of servers.
  • the server 105 connects to the user's computing devices 110 over a wide area network 115.
  • the server 105 stores sets of images, either on the server 105 itself or on another entity (e.g., another server, a database 106), and each set corresponding to a different realm.
  • the server 105 also stores code that, when executed on a computing device 110, enables a user to interact with a set of images.
  • users may transmit sets of images corresponding to realms to the server 105 for storage (for the sake of simplicity, user activity discussed herein will be described in reference to the user initiating the activity, even though the user's computing device may perform the activity itself).
  • the set of images may include information such as the relationships between the images and the criteria for switching from one image to another.
  • any given image in the set may have embedded objects, which the user may interact with.
  • Fig. 2A is an exemplary flow diagram depicting the initiation of a session in which a group of users experience the same realm.
  • a user may send a request to create a session to the server 105 (step 201).
  • the request identifies at least one particular realm to associate with the session, and as the session creator, the user has administrative privileges, described in further detail below (session creators are also referred to herein as
  • the server 105 creates the session (step 205), and associates the administrative user and the realm(s) with the session.
  • the administrative user sends invitations to at least one other user to join the session (step 210).
  • the invitations include an identifier for the session, such as an
  • the administrative user sends the invitations to the server 105 (step 215), and the server forwards the invitations to the users (step 220).
  • the administrative user may send the invitations directly to the computing devices of the users (step 220).
  • a user may accept an invitation to join a session (step 225) by, for example, accessing the URL in an invitation, which directs the computing device 110 of the user to the server 105.
  • the server 105 adds the user to the session (step 230).
  • the server 105 transmits the set(s) of images
  • the computing device 110 stores the set of images and executable code.
  • each image in the set of images is a panoramic image.
  • the computing device 110 displays image data from one panoramic image at a time. To provide a more natural experience for the user, the computing device 110 displays only a subset of the panoramic image, which corresponds to the user's view of the represented scene. Thus, any given subset of the panoramic image may be associated with a unique view direction.
  • the computing device 110 may initially display the panoramic image from a default view direction. For example, each image in the set may be stored in association with a default view direction.
  • the computing device 110 identifies a subset of the panoramic image to display, based on the default view direction. Thereafter, the user is allowed to change the view direction. For example, the user may drag the panoramic image to pan to the left or right, or tilt upwards or downwards. In response to the user control, the computing device 110 updates the view direction and the corresponding subset of the panoramic image to display. [0045] Furthermore, the images in the set may be interrelated in a predetermined manner. Examples of such relationships are depicted in Fig. 3. The relationships constrain the sequence in which the user may view the images for a realm. For example, when the computing device 110 loads the set of images depicted in Fig. 3, the computing device 110 may display the image labeled "1" first.
  • the user may only opt for image "2".
  • the computing device 110 displays image "2"
  • the user is permitted to switch solely to either image "1" or "3". Should the user traverse the images to select image "7”, the user would be permitted to select image "6", "8", or "10" to view.
  • the user may not be permitted to change images unless the view direction for the displayed image encompasses a pathway between the locations in the realm represented by the displayed image and another image. For example, suppose images "1" and "2" represented adjacent rooms separated by a doorway. When the user is exploring the scene in image "1", to provide a natural experience of the realm, the user is not be permitted to select image "2" unless the view direction for image "1" encompasses the door separating the rooms. In this manner, the user would not be permitted to select image "2” unless the view direction falls within a predetermined range of view directions, during which the door would be visible. In some embodiments, the computing device 100 would not display an icon, such as an arrow, for changing images until the view direction for the displayed image meets this criterion.
  • an icon such as an arrow
  • the computing device 110 When the user sends the computing device 110 an instruction to display another image, the computing device 110 creates and displays transitional images to simulate motion between the locations represented by the two images in the set.
  • the server 105 may create the transitional images and send them to the computing device 110 to display. Exemplary systems and methods for creating transitional images are described in Application No. 11/271, 159, which is incorporated by reference in its entirety.
  • the transitional images may represent the pathway between the view direction for the prior image, and a default view direction for the selected image, which may depend on the identity of the prior image. For example, an image may have multiple default view directions, and referring to Fig.
  • image “2" may have one default view direction if a user has selected the image from image “1” and another default view direction if a user has selected the image from image "3". If a user is viewing image “1” and selects image “2", the computing device 110 uses the current view direction for image “1” and the default view direction for image “2” when the user is transitioning from image “1", and creates transition images to simulate motion between the two images.
  • the images in the set are not interrelated. Instead, the user may switch from one image in the set to any other image. For example, the user may invoke a control for changing images, and in response, the computing device 110 may display thumbnails representing the images in the set. When the user selects a thumbnail, the computing device 110 may display the corresponding image. In these embodiments, each image may include a default view direction, and the device 110 loads the image data for the default view direction. Because the images are not interrelated, the computing device 110 may switch from one image to another, instead of creating transitional images. Alternatively, the computing device 110 may transition between images by, for example, morphing one image into another. In some embodiments, a user may configure settings provided by the code to select the manner in which the computing device 110 displays images sequentially.
  • the computing device 110 sends updates about the state of the user's location to the server 105 (step 240).
  • the computing device 110 may update the server 105 regarding the image on display and the view direction therein.
  • Each communication may include an identifier for the session, an identifier for the user (e.g., the user's computing device 110), an identifier for the image in the set that is being viewed, a yaw angle for the view direction, a pitch angle for the view direction, or any combination thereof.
  • the server 105 receives updated locations in the realm for each user in the session, and the server 105 may store the updates for each user.
  • the computing device 110 may send updates directly to another user and/or all of the other users in the group.
  • the computing device 110 sends updates about the state of the user's location directly to the other users in the group (step 240'), thereby bypassing the server 105.
  • the server 105 transmits each update from each user to all of the other users in the group (step 245).
  • the server 105 may send updates solely to the administrative user. When the administrative user receives this information, his or her computing device 110 may store the information, thereby saving a history of the users' locations for the session (step 250).
  • the tour platform 100 allows users to synchronize other users' experiences of the realm.
  • any given user also referred to herein as the
  • commanding user may require another user (also referred to herein as the "following user") to follow his or her experience of the realm.
  • the computing device 110 of the following user displays the realm according to the location of the commanding user, and in some embodiments, the computing device 110 may become unresponsive to instructions from the following user to select an image or view direction in the realm.
  • a user may elect to follow the experience of the realm for another user. Thus, the user voluntarily becomes a "following user", but may opt to stop following at any time.
  • Fig. 4A is an exemplary flow diagram depicting one user commanding and controlling the experience of the realm for another user (also referred to herein as
  • the commanding user may invoke the option to command and control other users.
  • the computing device 110 may display an icon for commanding and controlling other users' experience, and in response to user selection of this icon, the computing device 1 10 may display the users in the group for the session.
  • the commanding user may select individuals to be following users (step 405), and transmit an instruction to command and control the experiences of the selected users to the server 105 (step 410).
  • the administrative user may be allowed to command and control all of the other users in the group, but a non-administrative user may be permitted to command and control a more limited number of users.
  • the server 105 sends instructions to the selected users to follow the commanding user (step 415). In Fig.
  • the computing device 110 transmits instructions to command and control directly to the computing devices 110 of the selected users (step 410'), thereby bypassing the server 105.
  • the computing device 110 of a following user may confirm receipt of the instruction to the server 105 (step 420).
  • the computing device 110 may display a notification to the following user that he or she is now following the commanding user, and may also begin disregarding any commands that the following user attempts to input (e.g., selection of an image to view, selection of a view direction).
  • the computing device 110 of a following user begins displaying the realm according to the commanding user's experience (step 425).
  • the instruction from the server 105 includes a location of the commanding user.
  • the computing device 110 obtains the location of the commanding user in the course of the commanding user's broadcast of his or her updated location.
  • the server 105 may transmit each update from each user to all of the other users in the group (step 245).
  • the user's computing device 110 may ignore these updates, or store the updates to create a local copy of the session history.
  • his or her computing device 110 may begin using the updated information associated with the commanding user to determine the image data in the set of images to display.
  • a user When a user becomes a following user, the computing device 110 may jump to the location of the commanding user directly. Alternatively, the computing device 110 may create transitional images to simulate motion between the following and commanding users' locations.
  • Fig. 5 is an exemplary flow diagram depicting the manner in which a following user's computing device 110 updates the image data that it displays, based on the location of the commanding user. The computing device 110 may execute code to perform these steps when it first begins following the commanding user, and any time the
  • commanding user changes his or her location thereafter.
  • the computing device 110 of the following user may receive an update to the commanding user's location (step 505) from the server 105.
  • the update may identify an image in the set and a view direction.
  • the computing device 110 compares the image in the update with the image that it is currently displaying. If the images are the same (step 510), using the view direction associated with the image data on display and the newly received view direction, the computing device 110 creates and displays transitional images to simulate motion from the current view direction to the new one (step 515).
  • the computing device 110 displays the image data for the newly received view direction until the device 110 receives another update (step 520).
  • the computing device 110 retrieves the new image from memory, as well as a default view direction (step 525). If the image has multiple default view directions, the computing device 110 may select the default that is closest to the view direction in the update and display image data associated with this default. The computing device 110 creates and displays transitional images to simulate motion from the default view direction to the view direction in the update (step 530). The computing device 110 displays the image data for the updated view direction until the device 110 receives a further update (step 520).
  • the computing device 110 of a following user may continue performing the steps in Fig. 5 until the commanding user releases control. While the commanding user compels other users to following his or her experience of the realm, his or her computing device 110 displays an icon for releasing control of other users. If the commanding user selects this icon, the computing device 110 sends an instruction to the server 105 to terminate the control over the other users, and the server 105 forwards the commands to the computing devices 110 of those users.
  • a computing device 110 may display a notification to its user that the commanding user has terminated its control.
  • the computing device 110 disregards location updates from the commanding user, and becomes receptive to instructions from the user, again.
  • the computing device 110 remains at the most recent location selected by the commanding user, and allows the following user to navigate from there.
  • the computing device 110 may have stored the most recently selected location of the following user prior to the instruction to follow the commanding user, and after the commanding user releases control, the computing device 110 restores the selected location of the user.
  • a user may command and control other users' experiences of the realm, in some embodiments, the user may simply share his or her experience, while allowing the other users to retain control.
  • the sharing user may perform similar steps to those described in Fig. 4, but with instructions to share the user's experience with the targeted users.
  • the computing devices 110 of the recipients reserve an area of the display for depicting the experience of the sharing user (e.g., a "picture-in-picture" display), and the devices 110 execute code both for updating the image data displayed in this reserved area, and updating the image data for locations selected by the users.
  • the devices 110 may downsample the images in the set to determine the image data to display for the experience of the sharing user, but process the images received from the server 105 to determine the image data to display for the user of the computing device 110.
  • a user may opt to follow another user.
  • the computing device 110 may include a control for following a user, and when this control is activated, the device 110 displays a list of users to follow. By selecting a user, the computing device 110 begins displaying image data corresponding to the followed user's location, whether the device 110 receives the location as part of the followed user's broadcast or sends a request to the server 105 to receive the location of the followed user whenever this location is updated.
  • the user can opt to either replace the image data on its computing device 110 with the experience of the followed user, or display the experience on a reserved area, thereby viewing different locations in the realm simultaneously. While the user is following another user, the computing device 110 displays an "unfollow" control, whose selection reverts the display data to that chosen by the user.
  • the administrative user may wish to select a different realm for the session.
  • the administrative user may be a real estate agent, and upon completing a tour of a property with a client, the real estate agent may wish to show another property.
  • the client may express disinterest with a property, and the parties may select another property to determine if that property will fit the client's needs more appropriately.
  • Fig. 6A is an exemplary flow diagram depicting the administrative user's selection of a new realm for a session, and the propagation of that realm to the other users in the group.
  • the administrative user activates an icon on the computing device 110 to change realms.
  • the computing device 110 displays representations of realms for the session, such as thumbnails of the first images for each of the realms.
  • the administrative user selects one of the realms, and the computing device 110 loads the set of images associated with the selected realm (step 610).
  • the computing device 110 displays the first image in this set, based on the default view position for the image and/or beginning of a tour of the new realm.
  • the computing device 110 sends an update to the server 105 regarding the new realm for the session (step 615).
  • the server 105 sends an update to the other users in the group regarding the new realm for the session (step 620).
  • the computing device 110 send the update directly to the other users in the group (step 615'), thereby bypassing the server 105.
  • their computing devices 110 process the update (step 625), retrieve the set of images for the new realm from memory, and display the first image in this set (step 630).
  • the users may be informed of realms for the session in real-time, and obtain new sets of images from the server 105 whenever the administrative user selects a new realm for the experience.
  • the code may identify the realms via alphanumeric identifiers, by way of example, but may not contain their sets of images (nevertheless, the code may include the set for the first realm of the session).
  • the computing device 110 may send a request with the alphanumeric identifier to the server 105, and in response, the server 105 may return the images. In this manner, the computing devices 110 do not store sets of images for realms until such sets are needed.
  • the server 105 may store a user history of the session.
  • Fig. 7 is an exemplary flow diagram for creating a session history in real time. The steps are similar to those described in Fig. 4. However, each time a user sends an updated location to the server 105, the server 105 records the change (step 720), and may also store the time that the update was received. In this manner, the server 105 creates a log of every location that each user visited, and the length of time that the user remained at that location. Thus, the server 105 may preserve the history of the session.
  • the administrative user may run analytics on the session history to ascertain the other users' interests. For example, the administrative user may instruct the server 105 to determine the percentage of time that the users spent in each image, and well as particular sections of each image. If a realm corresponds to a house, analytics may reveal the percentage of time that a user spent in the kitchen, master bedroom, bathroom, and living room. Analytics may also determine that, while the user was in the kitchen, the user spent part of the time examining the appliances, but a significant amount of time enjoying the view into the backyard. Similarly, while the user remained in the basement, the user may have spent little time looking at the heating system, but a significant amount of time looking at the foundation. In this manner, the administrative user may discern items that interest or concern a user based on their location, and select other houses (and realms) accordingly.
  • the inventors have developed an apparatus and methods that enable anyone with a mobile computing device (e.g., a smartphone) to obtain high quality panoramic images.
  • the apparatus is configured to house a mobile computing device, and the apparatus further leverages features of existing panoramic image capturing software (also referred to herein as "panoramic software") on the device to ensure that the device captures excellent panoramic images.
  • panoramic software also referred to herein as "panoramic software”
  • the apparatus uses the panoramic software's reliance on the device's accelerometer(s) to determine whether the device is in motion.
  • the apparatus may ensure that the mobile computing device captures the images needed to create a panoramic image, but only when the apparatus has halted motion at the designated positions.
  • FIG. 8 Elements of an exemplary apparatus 100 that leverages features of panoramic image capturing programs on mobile computing devices to facilitate the capture of panoramic images are depicted in Figs. 8 and 9.
  • the apparatus 100 includes a receptacle 110 that is configured to house a mobile computing device (not shown).
  • a user may mount the receptacle 110 on a platform 120 by coupling the receptacle 110 to its member 125.
  • the platform may include a motor 130 that is coupled to the member 125, as depicted in Fig. 10.
  • the motor 130 and member 125 are coupled by a drivetrain, collectively represented by the gearbox 126 and drive gear(s) 127, though other
  • the motor 130 By driving the member to different positions, the motor 130 consequently drives the mobile computing device in the receptacle 110 to different positions, as well.
  • a user inserts his or her mobile computing device in the receptacle 110 and mounts the receptacle 110 on the platform 120 (alternatively, the receptacle 110 may be permanently affixed to the platform 120, and the user may merely insert the device therein). Then, the user initiates the panoramic software of the mobile computing device 190 and activates the motor 130 to begin driving the member 125.
  • the user activates the motor by depressing a control, such as a button or any other physically tactile component (not shown).
  • the control may activate the motor 130 by detecting that the mobile computing device has been coupled to the platform 120.
  • control may be a sensor, or sensor system, that detects the presence of a mounted receptacle 110, a mobile computing device within the receptacle 110, or both, and which activates the motor 130 in response to such detection.
  • the sensor may monitor the weight applied to the member 125 and activate the motor 130 when the weight exceeds a threshold.
  • the motor 130 drives the member 125, and consequently, the mobile computing device, to move to different positions, and the motor 130 also halts motion at the positions for predefined periods of time.
  • the motor's 125 activities are coordinated with the panoramic software of the mobile computing device.
  • the motor 130 may drive the member 125 to move at a velocity that is sufficiently high so as to be detectable by the mobile computing device 190. Because the device 190 determines that it is in motion, the device 190 refrains from capturing images for the panoramic image. Once the motor 130 halts, the motor 130 remains stationary for a sufficiently long period of time to trigger the panoramic software of the mobile computing device. In response to this stillness for a threshold period of time, the mobile computing device prepares its camera (e.g., focusing, color balancing, etc.) according to the scene in view, and captures an image.
  • its camera e.g., focusing, color balancing, etc.
  • the motor 130 drives the member 125 on a continual basis.
  • the configuration of the drivetrain (e.g., gearbox 126 and drive gears 127) and the resultant movement of the member 125 is coordinated with the panoramic software of the mobile computing device, in lieu of the motor 130.
  • the drivetrain moves the member 125 to different positions, and also halts motion at the positions for predefined periods of time.
  • the mobile computing device 190 recognizes that it has remained stationary for a sufficiently long period of time to trigger the panoramic software of the mobile computing device 190, the mobile computing device 190 prepares its camera according to the scene in view, and captures an image.
  • the apparatus 100 provides a period of stillness that is coordinated with the period of time that mobile computing device uses to recognize its physically still state and to capture an image.
  • the apparatus 100 resumes driving the member 125 and mobile computing device to its next image capture position.
  • the apparatus 100 and mobile computing device do not communicate with one another regarding their own operations. At no point does the mobile computing device recognize that it has been coupled to an apparatus 100. Moreover, even if a mobile computing device has not been mounted on the apparatus 100, if a user activates the motor 130, the motor 130 and/or drivetrain stills drive the member 125 through its sequence of positions, halting motion therebetween. Thus, the apparatus 100 and mobile computing device operate independently of one another, even though the apparatus 100 tailors its parameters to the features of panoramic software on the mobile computing device.
  • the motor 130 and/or drivetrain may drive the member 125 angularly or linearly.
  • the member 125 may be a turntable that rotates around an axis 128.
  • the motor 130 and/or drivetrain may alternate between driving the member 125 to rotate by a particular number of degrees and halting the member 125 for a predefined period of time. Because the apparatus 100 facilitates capture of a panoramic image, the apparatus 100 may rotate the mobile computing device by a total of 360 degrees, throughout the sequence of its motions.
  • the apparatus 100 may move the member 125 and mobile computing device to ten (10) positions for capturing images for a panoramic image.
  • the motor 130 and/or drivetrain drives the member 125 to rotate thirty-six (36) degrees before halting for a predefined period of time.
  • the motor 130 and/or drivetrain drives the member 125 to rotate by another thirty-six (36) degrees before halting.
  • the motor 130 and/or drivetrain repeats this sequence until the member 125 has been rotated a full three hundred and sixty (360) degrees.
  • the motor 130 and/or drivetrain may halt motion altogether, and the panoramic software of the mobile computing device may recognize that the camera can no longer be expected to move, and the device begins aggregating the captured images into a single panoramic image.
  • this example describes angular increments of thirty-six (36) degrees for a total of 10 positions for capturing images, any other angular increment, and corresponding number of positions, may be used.
  • the member 125 may be a rotary table. Furthermore, the receptacle 110 and member 125 may be configured such that, when a mobile computing device 190 is inserted into the receptacle 110, the device 190 is offset radially from the axis of rotation 128 of the member 125 by a predefined distance, so as to align the entrance pupil 192 of the lens system 195 with the axis 128.
  • the motor 130 and/or drivetrain drives the member 125 to rotate around its axis 128, the lens system 195 rotates around the axis 128 while the device 190 rotates around the axis 128 at an offset.
  • the offset ensures that the mobile computing device 190 captures images suited for panoramic images, and the offset may be selected to optimize the quality of the images for this purposes. In some embodiments, the offset may be less than about 0.5 inches.
  • Figs. 11 and 12 depict two exemplary views of a receptacle 110 mounted on a member 125 that illustrates a radial offset between the device 190 and the axis of rotation 128 of the member 125, which ensures alignment between the axis 128 and the lens system 195.
  • Fig. 13 depicts an overhead view of the receptacle 110 demonstrating that the imaging system of a mobile computing device would align with the entrance pupil 192 of the receptacle 110.
  • the member 125 may progress along a linear path predefined by a groove in the platform.
  • the motor 130 and/or drivetrain drives the member 125 to move down the path by a particular distance, and halts the member 125 for a predetermined period of time before driving the member 125 further down the path.
  • the member 125 may move at a constant velocity, although in some embodiments, the motor 130 and/or drivetrain may vary the velocity of the member 125.
  • the member 125 may be coupled to the motor 130 via a set of gears.
  • Figs. 14-17 depict top, side, bottom, and isometric views of an exemplary set of gears that may be used in the platform 120 of the apparatus 100.
  • the motor 130 drives a gearbox 805 to rotate a first gear 810.
  • the teeth of the first gear 810 are engaged with the teeth of a second gear 815, which in turn are engaged with the teeth of a third gear 820 attached to the member 125 (in this case, a rotary table).
  • a third gear 820 attached to the member 125 (in this case, a rotary table).
  • rotation of the first gear 810 via the motor 130 drives the rotation of the rotary table 125.
  • the configuration of teeth for at least one of the gears may determine the positions that the member 125 and mobile computing device will assume.
  • one of the gears may have teeth of different length.
  • motion of the member 125 begins, and motion halts when the shorter teeth of the gear fail to engage with the other gear in the set.
  • the position where the member 125 halts depends on the position of the shorter teeth on the gear, as well as the gear's relationship to others in the set (e.g., relative size).
  • the amount of time that the member 125 remains stationary may depend on the form factor of the engaged teeth.
  • the positions where the member 125 halts motion, as well as the duration of those periods, can be controlled by selecting the size of the gear, the angular frequency of the larger teeth, and the form factors of the larger teeth, among other parameters.
  • activating the motor 130 may cause the motor 130 to drive the set of gears for a predetermined period of time, but the configuration of the gears, both individually and as a set, define the positions that the member 125 and mobile computing device attain, and the length of time they remain in such positions.
  • the platform 120 may include a processor and a memory for storing computer instructions.
  • the processor may load and execute computer instructions from the memory, and these instructions may control the manner in which the motor 130 drives the member 125. For example, based on the instructions, the processor may alternate between driving the member 125 at a particular velocity for a period of time, and halting all motion. Because the parameters for motion and stillness are encoded in the computer instructions, these embodiments require simpler sets of gears than the embodiments that rely exclusively on the set of gears to determine positions for the member 125 and the length(s) of time for halting motion at the positions.
  • the lens system 192 When the receptacle 110 has a lens system 192, the lens system 192 must be positioned so that the lens of an inserted mobile computing device 190 aligns with the lens system 192 on the receptacle 110. As shown in Fig. 13, the receptacle 110 may be configured to position a mobile computing device 190 to ensure this alignment. Enabling the device to leverage its own lens and the lens system 192 of the receptacle 110 can improve the quality of images obtained for panoramic images. As discussed above, and depicted in Fig. 11, the lens system 192 of the receptacle 100 must also be positioned so that its entrance pupil 192 is aligned with the axis of rotation 128 of the member 125.
  • the embodiments depicted in Figs. 9 and 11-13 illustrate a lens system 195 affixed to the receptacle 110, and positioned to align with the lens of the mobile computing device
  • the receptacle 110 does not have an attached lens system 195.
  • the mobile computing device includes the sole lens (or lenses) used for image capture.
  • the receptacle 110 would still be configured to position the entrance pupil of the lens of the mobile computing device according to the same principles guiding the positioning of the lens system 192 affixed to a receptacle 110.
  • the receptacle 110 includes an open sleeve 111 with arms 112, 113 configured to receive a mobile computing device 190.
  • a user may insert the device into the sleeve 111, and the arms 112, 113 hold the device in place.
  • the receptacle 110 may include a slot 111 whose front and back faces define a space for housing the device.
  • the receptacle 110 may include clamps that hold the device in place.
  • the receptacle 110 may include flexible molded components that define at least part of the perimeter of the space for housing the device. When a user presses the device into the space, the component flex to accept and secure the device.
  • the receptacle 110 may be mounted on the platform 120 in any manner desired.
  • the receptacle 110 is coupled to the member 125 by sliding a portion of the receptacle 110 into a groove of the member 125.
  • any other method of mounting or coupling may be used.
  • the receptacle 110 may include one or more pins, and the member 125 may include one or more holes that receive the pins of the receptacle 110.
  • a portable device for facilitating capture of panoramic images on a mobile computing device comprising:
  • a receptacle configured to house a mobile computing device
  • a platform comprising (1) a member, with an axis of rotation, configured to interface with the receptacle to offset radially the mobile computing device from the axis of rotation by a predefined distance such that an entrance pupil of a lens system of the mobile computing device is aligned with the axis, and (2) a motor that drives the member at different velocities so that the mobile computing device captures an image for a panoramic image when a velocity of the member is substantially zero for a threshold period of time.
  • P5. The portable device of claim PI, wherein the motor is coupled to a processor that executes code to drive the member.
  • P7 The portable device of claim P6, wherein the motor drives the member to rotate by a predetermined number of degrees.
  • the drivetrain comprises at least one drive gear with teeth of different lengths, wherein the teeth of different lengths determine the different velocities at which the member is driven.
  • PI 1 The portable device of claim PI, wherein the predefined distance of radial offset between the mobile computing device and the axis of rotation of the member is less than 0.5 inches.
  • PI 2 A method for facilitating capture of panoramic images on a mobile computing device, the method comprising:
  • driving the member comprises: driving, by the motor, the member from the first location to the second location at a non-zero angular velocity.
  • driving the member comprises: driving, by the motor, the member to rotate from the first location to the second location.
  • driving the member comprises: driving, by the motor, the member to rotate by a predetermined number of degrees.
  • driving the member comprises: driving, by the motor, a drivetrain coupled to the member, wherein the drivetrain comprises at least one drive gear with teeth of different lengths and positions of the teeth of different lengths determine when the member halts.
  • a computer-readable medium having computer program code non-transitorily stored thereon, the computer program code for facilitating capture of panoramic images on a mobile computing device, the program code comprising:
  • program code for monitoring motion of the mobile computing device including substantially zero and substantially non-zero velocities of the mobile computing device
  • program code for capturing at least one image of a scene in response to determinations that the mobile computing device has remained at a substantially zero velocity for a threshold period of time.
  • a portable device facilitates capture of panoramic images on a mobile computing device.
  • the device includes a receptacle configured to house a mobile computing device.
  • the device also includes a platform with a member, with an axis of rotation, that is configured to interface with the receptacle to offset radially the mobile computing device from the axis of rotation by a predefined distance, such that an entrance pupil of a lens system of the mobile computing device is aligned with the axis.
  • the platform also includes a motor that drives the member at different velocities so that the mobile computing device captures an image for a panoramic image when a velocity of the member is substantially zero for a threshold period of time.
  • the velocities may be angular velocities.
  • the member may be a circular member.
  • the motor may be a mechanical motor.
  • the motor may be coupled to a processor that executes code to drive the member.
  • the motor may drive the member to alternate between rotating and remaining still.
  • the motor may drive the member to rotate by a predetermined number of degrees.
  • the predetermined number of degrees may be thirty-six (36).
  • the predefined distance of radial offset between the mobile computing device and the axis of rotation of the member may be less than 0.5 inches.
  • a method facilitates capture of panoramic images on a mobile computing device.
  • the method initiates, by a mobile computing device, a program to capture images of a scene for a panoramic image when the mobile computing device is stationary.
  • the method also captures, by a mobile computing device, a first image of a scene at a first location.
  • the method further drives, by a motor, a member configured to interface with a receptacle containing the mobile computing device from the first location to a second location.
  • the method additionally halts, by the motor, further motion of the member for a predetermined period of time.
  • the method determines, by a sensor of the mobile computing device, a lack of motion of the mobile computing device for a threshold period of time. In response to the lack of motion, the method captures, by the mobile computing device, a second image of the scene at the second location. Lastly, the method creates the panoramic image from the first and second images of the scene.
  • the method may drive, by the motor, the member from the first location to the second location at a non-zero angular velocity.
  • the method may drive, by the motor, the member to rotate by a predetermined number of degrees.
  • a computer- readable medium has computer program code non-transitorily stored thereon, and the computer program code facilitates capture of panoramic images on a mobile computing device.
  • the program code includes code for monitoring motion of the mobile computing device, including substantially zero and substantially non-zero velocities of the mobile computing device.
  • the program code also includes code for capturing at least one image of a scene in response to determinations that the mobile computing device has remained at a substantially zero velocity for a threshold period of time.

Abstract

In accordance with one embodiment of the invention, a method coordinates experiences of a realm for a group of users, including at least an administrative user and at least one participating user. The method includes transmitting, from the server to an administrative user, a session identifier associated with the realm. The method also includes receiving, by the server, a user acceptance to join a session using the session identifier so that the at least one participating user can experience the realm. In response to receiving the acceptance, the method also includes transmitting to a computing device of the participating user, by the server, executable code including data describing a series of locations in the realm. The series of locations comprises at least a first location and a second location, and the first location is associated with a first panoramic image and the second location is associated with a second panoramic image. The executable code is configured to receive instructions from at least one computing device of at least one user within the group to display the realm at at least one selected location, and display, on the computing device, data corresponding to the realm at at least one of the locations selected by at least one of the users within the group.

Description

Systems and Methods for Providing Individual and/or Synchronized Virtual Tours
Through a Realm for a Group of Users
Cross-Reference to Related Application's)
[0001] This PCT patent application claims priority from the following United States patent applications, each of which is hereby incorporated herein by reference to the extent permitted:
[0002] United States Provisional Patent Application No. 62/368,471, entitled "Systems and Methods for Providing Individual and/or Synchronized Virtual Tours through a Realm for a Group of Users", filed on 7/29/16, and naming as inventors James
Schoonmaker, Matthew Dunn, and Keno Mullings (Attorney Docket No. 2894/124).
[0003] United States Patent Application No. 15/224,077, entitled "Systems and Methods for Providing Individual and/or Synchronized Virtual Tours through a Realm for a Group of Users", filed on 7/29/16, and naming as inventors James Schoonmaker, Matthew Dunn, and Keno Mullings (Attorney Docket No. 2894/126).
[0004] United States Patent Application No. 15/224,068, entitled "Apparatus and Method for Facilitating Capture of Panoramic Images on a Mobile Computing Device", filed on 7/29/16, and naming as inventors James Schoonmaker, Matthew Dunn, and Stephen Seibert (Attorney Docket No. 2894/125).
Technical Field
[0005] The present invention relates to providing individual and/or synchronized virtual tours through locations within a realm for a group of users within a networked environment, and more particularly, to allowing users to either select their own locations or rely on other users to select locations for experiencing the realm.
Background
[0006] Under conventional systems and methods, users examine a realm merely by viewing static images. A user cannot interact with the images, aside from selecting a particular image to view. Furthermore, if the images do not make the relationships between different locations of the realm clear, the user is left to envision how the locations connect to one another. For example, suppose a house hunter views a listing, which provides a group of images for a property. Each image may represent a room from one particular point of view, but the image may nevertheless omit some information about the room, due to the limitations of the camera that captured the image. Additionally, the house hunter may need to switch back and forth between images to determine how rooms relate to one another by, for example, matching features (e.g., picture windows, chandeliers) that appear in multiple images. Moreover, the house hunter may view the images on his or her own. Through conventional methods of experiencing realms, other parties, such as real estate agents, cannot share the house hunter's experience of the realm and cannot control what the house hunter views. In this manner, each individual may have a unique, personal experience of the realm that is not readily communicated with others.
Summary of the Embodiments
[0007] In accordance with one embodiment of the invention, a method coordinates experiences of a realm for a group of users, including at least an administrative user and at least one participating user. The method includes transmitting, from the server to an administrative user, a session identifier associated with the realm. The method also includes receiving, by the server, a user acceptance to join a session using the session identifier so that the at least one participating user can experience the realm. In response to receiving the acceptance, the method also includes transmitting to a computing device of the participating user, by the server, executable code including data describing a series of locations in the realm. The series of locations comprises at least a first location and a second location, and the first location is associated with a first panoramic image and the second location is associated with a second panoramic image. The executable code is configured to receive instructions from at least one computing device of at least one user within the group to display the realm at at least one selected location, and display, on the computing device, data corresponding to the realm at at least one of the locations selected by at least one of the users within the group.
[0008] In some embodiments, the executable code is further configured to receive instructions from the server regarding locations selected by a user within the group. Further, the method may include receiving, by the server from the administrative user, a request to synchronize an experience of the realm for the participating user with an experience of the realm for the administrative user. The request may include a location in the realm selected by the administrative user. The method may include transmitting, by the server, the location in the realm selected by the administrative user to a computing device of the participating user. The location causes the executable code running on the computing device of the participating user to display data corresponding to the location selected by the administrative user.
[0009] The method may include transmitting, by the server to a computing device of the participating user, locations in the realm selected by the users of the group. Additionally, the method may include receiving, by the server from the administrative user, a request to synchronize an experience of the realm for the participating user with an experience of the realm for the administrative user. In response to the request from the administrative user, the method may include transmitting, by the server to the computing device of the participating user, an instruction to display data corresponding to the locations in the realm selected by the administrative user.
[0010] The computing device of the participating user may create at least one virtual image representing a view of the realm between a location selected by the participating user and the location selected by the administrative user. Also, the computing device may display data corresponding to the virtual image and the location selected by the administrative user to simulate motion in the realm from the location selected by the participating user to the location selected by the administrative user.
[0011] The method may include receiving, by the server from the computing device of the administrative user, an updated location selected by the administrative user. The method may also include transmitting, by the server to the computing device of the participating user, the updated selected location so that the computing device of the participating user displays data corresponding to the updated selected location in the realm. In some embodiments, the instruction to synchronize the experiences of the participating user and the administrative user causes the computing device of the participating user to disregard instructions from the participating user.
[0012] The method may include receiving, by the server from a first user in the group, a request to synchronize an experience of the realm for a second user in the group with an experience of the realm for the first user. In some embodiments, the selected location in the realm includes a view direction.
[0013] The method may include receiving, by the server from an administrative user, a request to view a list of users who have joined the experience of the realm. Also, the method may include transmitting, by the server to a computing device of the administrative user, the list of users for display.
[0014] In many embodiments, the method includes receiving, by the server from the administrative user, a request to view a history of selected locations in the realm for a user in the group of users. The method may include transmitting, by the server to a computing device of the administrative user, the history of selected locations in the realm for the user.
[0015] The method may include receiving, by the server from the administrative user, a request to follow the view of the participating user. The method may also include sending, by the server to the computing device of the participating user, an instruction to provide, on a continual basis, updates on the location in the realm selected by the participating user. The method may further include forwarding, by the server to the computing device of the administrative user, the updated location selected by the participating user. The updated location causes the executable code running on the computing device of the administrative user to display data corresponding to the realm at the updated selected location of the participating user.
[0016] In some embodiments, the method also includes transmitting, by the server to a computing device of the administrative user, locations in the realm selected by the users of the group. The executable code on the computing device of the administrative user may be configured to receive a request to follow the view of the participating user and display data corresponding to the locations in the realm received from the server that are selected by the participating user.
[0017] The method may include receiving, by the server from an administrative user, an instruction to display a second realm to the group of users. Additionally, the method may include transmitting, by the server to each computing device for the group of users, executable code including data describing a series of locations in the second realm. Brief Description of the Drawings
[0018] The foregoing features of embodiments will be more readily understood by reference to the following detailed description, taken with reference to the accompanying drawings, in which:
[0019] Fig. 1 depicts a block diagram representing a tour platform including at least one server that provides individual and synchronized experiences of a realm for a group of users;
[0020] Fig. 2A is an exemplary flow diagram depicting the initiation of a session in which a group of users experience the same realm;
[0021] Fig. 2B is an exemplary flow diagram depicting the initiation of the session of Fig. 2A, in which users bypass the server to update one another regarding their locations in the realm;
[0022] Fig. 3 depicts exemplary relationships between images in a set of images for a realm;
[0023] Fig. 4A is an exemplary flow diagram depicting one user commanding and controlling the experience of the realm for another user;
[0024] Fig. 4B is an exemplary flow diagram depicting one user bypassing the server to directly command and control the experience of the realm for another user;
[0025] Fig. 5 is an exemplary flow diagram depicting the manner in which the image data for one user is updated, based on the location of another user;
[0026] Fig. 6 A is an exemplary flow diagram depicting a user's selection of a new realm for a session, and the propagation of that realm to the other users in the group;
[0027] Fig. 6B is an exemplary flow diagram depicting the initiation of the session of Fig. 6A, in which users bypass the server to update one another regarding their locations in the realm; and
[0028] Fig. 7 is an exemplary flow diagram for creating a session history in real time.
[0029] Fig. 8 depicts a platform for an exemplary apparatus that leverages features of panoramic image capturing programs on mobile computing devices to facilitate the capture of panoramic images;
[0030] Fig. 9 depicts an isometric view of a receptacle for the mobile computing device when mounted on a rotating member of a platform; [0031] Fig. 10 depicts exemplary elements of the platform of Fig. 8;
[0032] Figs. 11-13 depict side, front, and top views of the receptacle and rotating member, illustrating the configuration of the receptacle to offset the mobile computing device from the axis of rotation of the member; and
[0033] Figs. 14-17 depict top, side, bottom, and isometric views of an exemplary set of gears that may be used in the platform of the apparatus of Fig. 8.
Detailed Description of Specific Embodiments
[0034] Definitions. As used in this description and the accompanying claims, the following terms shall have the meanings indicated, unless the context otherwise requires:
[0035] A "realm" means a three-dimensional space. Parts of a realm need not be contiguous - realms may include discontinuities. While a realm may represent an actual physical space, any part of a realm may be imaginary.
[0036] A "set" includes at least one member.
[0037] With reference to FIGS. 1-7, the following section describes various embodiments relating to Systems and Methods for Enabling Shared Experiences of a Realm for Users in a Group. Reference numerals recited in this section relate to FIGs. 1-7. The fact that the same reference numeral is used in the section below relating to an
Apparatus and Method for Facilitating Capture of Panoramic Images on a Mobile Computing Device should be disregarded.
[0038] The inventors have developed systems and methods that enable users in a group to experience the same realm concurrently, and also to synchronize, either by sharing or commanding and controlling, their experiences of the realm. The systems include a tour platform, with at least one server. The tour platform stores sets of images, each set corresponding to a different realm. A user selects a realm and creates a session for experiencing the realm. When other users join the session, the tour platform transmits code, which includes the set of images for the realm, to the computing devices of the users. The devices store and execute the code, thereby enable the user to interact with the images to experience the realm.
[0039] Local execution of the code improves the speed, performance, and overall quality of the user's experiences. Moreover, this local execution enables the tour platform to synchronize experiences of the realm for two or more users efficiently simply by transmitting the locations of interest within the realm to the computing devices of the users, instead of the full images corresponding to the locations. The users' computing devices may process the locations in conjunction with the set of images for the realm to identify the view (e.g., image data) to display to the user. As a result, the systems and methods described herein decrease the likelihood of disruption in a user's experience, due to errors in image data transmission or lapses in connectivity over the network.
[0040] Fig. 1 depicts a block diagram representing a tour platform 100 including at least one server 105 that provides individual and synchronized experiences of a realm for a group of users via each user's respective computing device 110. Although operations described herein are discussed with respect to the server 105, any given operation may be performed by one or more servers in a group of servers.
[0041] The server 105 connects to the user's computing devices 110 over a wide area network 115. The server 105 stores sets of images, either on the server 105 itself or on another entity (e.g., another server, a database 106), and each set corresponding to a different realm. The server 105 also stores code that, when executed on a computing device 110, enables a user to interact with a set of images. In various embodiments, from their computing devices 110, users may transmit sets of images corresponding to realms to the server 105 for storage (for the sake of simplicity, user activity discussed herein will be described in reference to the user initiating the activity, even though the user's computing device may perform the activity itself). The set of images may include information such as the relationships between the images and the criteria for switching from one image to another. Furthermore, any given image in the set may have embedded objects, which the user may interact with.
[0042] Fig. 2A is an exemplary flow diagram depicting the initiation of a session in which a group of users experience the same realm. A user may send a request to create a session to the server 105 (step 201). The request identifies at least one particular realm to associate with the session, and as the session creator, the user has administrative privileges, described in further detail below (session creators are also referred to herein as
"administrative users"). If the request includes more than one realm, the request identifies the first realm for the session. The server 105 creates the session (step 205), and associates the administrative user and the realm(s) with the session.
[0043] The administrative user sends invitations to at least one other user to join the session (step 210). The invitations include an identifier for the session, such as an
alphanumeric code, and may include a uniform resource location ("URL") for joining the session. In some embodiments, the administrative user sends the invitations to the server 105 (step 215), and the server forwards the invitations to the users (step 220). Alternatively, the administrative user may send the invitations directly to the computing devices of the users (step 220). A user may accept an invitation to join a session (step 225) by, for example, accessing the URL in an invitation, which directs the computing device 110 of the user to the server 105. When the server 105 receives an acceptance of an invitation, the server 105 adds the user to the session (step 230). The server 105 transmits the set(s) of images
corresponding to the realm(s) associated with the session to the user, as well as executable code for displaying and interacting with the set(s) of images. The computing device 110 stores the set of images and executable code.
[0044] The computing device 110 of the user executes the code to begin the user's experience of the realm (step 235), and to perform any of the steps described herein. In some embodiments, each image in the set of images is a panoramic image. The computing device 110 displays image data from one panoramic image at a time. To provide a more natural experience for the user, the computing device 110 displays only a subset of the panoramic image, which corresponds to the user's view of the represented scene. Thus, any given subset of the panoramic image may be associated with a unique view direction. Whenever the computing device 110 loads a panoramic image from the set, the computing device 110 may initially display the panoramic image from a default view direction. For example, each image in the set may be stored in association with a default view direction. Via the executable code, the computing device 110 identifies a subset of the panoramic image to display, based on the default view direction. Thereafter, the user is allowed to change the view direction. For example, the user may drag the panoramic image to pan to the left or right, or tilt upwards or downwards. In response to the user control, the computing device 110 updates the view direction and the corresponding subset of the panoramic image to display. [0045] Furthermore, the images in the set may be interrelated in a predetermined manner. Examples of such relationships are depicted in Fig. 3. The relationships constrain the sequence in which the user may view the images for a realm. For example, when the computing device 110 loads the set of images depicted in Fig. 3, the computing device 110 may display the image labeled "1" first. If the user wishes to view another image in the set, the user may only opt for image "2". When the computing device 110 displays image "2", the user is permitted to switch solely to either image "1" or "3". Should the user traverse the images to select image "7", the user would be permitted to select image "6", "8", or "10" to view.
[0046] Moreover, in some embodiments, the user may not be permitted to change images unless the view direction for the displayed image encompasses a pathway between the locations in the realm represented by the displayed image and another image. For example, suppose images "1" and "2" represented adjacent rooms separated by a doorway. When the user is exploring the scene in image "1", to provide a natural experience of the realm, the user is not be permitted to select image "2" unless the view direction for image "1" encompasses the door separating the rooms. In this manner, the user would not be permitted to select image "2" unless the view direction falls within a predetermined range of view directions, during which the door would be visible. In some embodiments, the computing device 100 would not display an icon, such as an arrow, for changing images until the view direction for the displayed image meets this criterion.
[0047] When the user sends the computing device 110 an instruction to display another image, the computing device 110 creates and displays transitional images to simulate motion between the locations represented by the two images in the set. Alternatively, the server 105 may create the transitional images and send them to the computing device 110 to display. Exemplary systems and methods for creating transitional images are described in Application No. 11/271, 159, which is incorporated by reference in its entirety. The transitional images may represent the pathway between the view direction for the prior image, and a default view direction for the selected image, which may depend on the identity of the prior image. For example, an image may have multiple default view directions, and referring to Fig. 3, image "2" may have one default view direction if a user has selected the image from image "1" and another default view direction if a user has selected the image from image "3". If a user is viewing image "1" and selects image "2", the computing device 110 uses the current view direction for image "1" and the default view direction for image "2" when the user is transitioning from image "1", and creates transition images to simulate motion between the two images.
[0048] Furthermore, in some embodiments, the images in the set are not interrelated. Instead, the user may switch from one image in the set to any other image. For example, the user may invoke a control for changing images, and in response, the computing device 110 may display thumbnails representing the images in the set. When the user selects a thumbnail, the computing device 110 may display the corresponding image. In these embodiments, each image may include a default view direction, and the device 110 loads the image data for the default view direction. Because the images are not interrelated, the computing device 110 may switch from one image to another, instead of creating transitional images. Alternatively, the computing device 110 may transition between images by, for example, morphing one image into another. In some embodiments, a user may configure settings provided by the code to select the manner in which the computing device 110 displays images sequentially.
[0049] In various embodiments, as the user interacts with the set of images for the realm, the computing device 110 sends updates about the state of the user's location to the server 105 (step 240). For example, the computing device 110 may update the server 105 regarding the image on display and the view direction therein. Each communication may include an identifier for the session, an identifier for the user (e.g., the user's computing device 110), an identifier for the image in the set that is being viewed, a yaw angle for the view direction, a pitch angle for the view direction, or any combination thereof. Thus, the server 105 receives updated locations in the realm for each user in the session, and the server 105 may store the updates for each user. Alternatively, in this situation and any others in which one computing device 110 sends information to the server 105 for forwarding to other users, the computing device 110 may send updates directly to another user and/or all of the other users in the group. For example, in Fig. 2B, in an alternative embodiment, the computing device 110 sends updates about the state of the user's location directly to the other users in the group (step 240'), thereby bypassing the server 105. [0050] In some embodiments, the server 105 transmits each update from each user to all of the other users in the group (step 245). Alternatively, the server 105 may send updates solely to the administrative user. When the administrative user receives this information, his or her computing device 110 may store the information, thereby saving a history of the users' locations for the session (step 250).
[0051] Additionally, the tour platform 100 allows users to synchronize other users' experiences of the realm. In particular, any given user (also referred to herein as the
"commanding user") may require another user (also referred to herein as the "following user") to follow his or her experience of the realm. During this period, the computing device 110 of the following user displays the realm according to the location of the commanding user, and in some embodiments, the computing device 110 may become unresponsive to instructions from the following user to select an image or view direction in the realm. In some embodiments, a user may elect to follow the experience of the realm for another user. Thus, the user voluntarily becomes a "following user", but may opt to stop following at any time.
[0052] Fig. 4A is an exemplary flow diagram depicting one user commanding and controlling the experience of the realm for another user (also referred to herein as
"refocusing" another user). On the computing device 110, the commanding user may invoke the option to command and control other users. For example, the computing device 110 may display an icon for commanding and controlling other users' experience, and in response to user selection of this icon, the computing device 1 10 may display the users in the group for the session. The commanding user may select individuals to be following users (step 405), and transmit an instruction to command and control the experiences of the selected users to the server 105 (step 410). In some embodiments, the administrative user may be allowed to command and control all of the other users in the group, but a non-administrative user may be permitted to command and control a more limited number of users. The server 105 sends instructions to the selected users to follow the commanding user (step 415). In Fig. 4B, in an alternative embodiment, the computing device 110 transmits instructions to command and control directly to the computing devices 110 of the selected users (step 410'), thereby bypassing the server 105. [0053] The computing device 110 of a following user may confirm receipt of the instruction to the server 105 (step 420). At this time, the computing device 110 may display a notification to the following user that he or she is now following the commanding user, and may also begin disregarding any commands that the following user attempts to input (e.g., selection of an image to view, selection of a view direction). The computing device 110 of a following user begins displaying the realm according to the commanding user's experience (step 425).
[0054] In some embodiments, the instruction from the server 105 includes a location of the commanding user. In other embodiments, the computing device 110 obtains the location of the commanding user in the course of the commanding user's broadcast of his or her updated location. As described in reference to Fig. 2 A, the server 105 may transmit each update from each user to all of the other users in the group (step 245). When a user is not being commanded and controlled by another user, the user's computing device 110 may ignore these updates, or store the updates to create a local copy of the session history.
However, when a user is selected as a following user, his or her computing device 110 may begin using the updated information associated with the commanding user to determine the image data in the set of images to display.
[0055] When a user becomes a following user, the computing device 110 may jump to the location of the commanding user directly. Alternatively, the computing device 110 may create transitional images to simulate motion between the following and commanding users' locations. Fig. 5 is an exemplary flow diagram depicting the manner in which a following user's computing device 110 updates the image data that it displays, based on the location of the commanding user. The computing device 110 may execute code to perform these steps when it first begins following the commanding user, and any time the
commanding user changes his or her location thereafter.
[0056] The computing device 110 of the following user may receive an update to the commanding user's location (step 505) from the server 105. The update may identify an image in the set and a view direction. The computing device 110 compares the image in the update with the image that it is currently displaying. If the images are the same (step 510), using the view direction associated with the image data on display and the newly received view direction, the computing device 110 creates and displays transitional images to simulate motion from the current view direction to the new one (step 515). The computing device 110 displays the image data for the newly received view direction until the device 110 receives another update (step 520).
[0057] Alternatively, if the image identified in the update does not match the image currently on display, the computing device 110 retrieves the new image from memory, as well as a default view direction (step 525). If the image has multiple default view directions, the computing device 110 may select the default that is closest to the view direction in the update and display image data associated with this default. The computing device 110 creates and displays transitional images to simulate motion from the default view direction to the view direction in the update (step 530). The computing device 110 displays the image data for the updated view direction until the device 110 receives a further update (step 520).
[0058] The computing device 110 of a following user may continue performing the steps in Fig. 5 until the commanding user releases control. While the commanding user compels other users to following his or her experience of the realm, his or her computing device 110 displays an icon for releasing control of other users. If the commanding user selects this icon, the computing device 110 sends an instruction to the server 105 to terminate the control over the other users, and the server 105 forwards the commands to the computing devices 110 of those users.
[0059] Upon receipt, a computing device 110 may display a notification to its user that the commanding user has terminated its control. The computing device 110 disregards location updates from the commanding user, and becomes receptive to instructions from the user, again. In some embodiments, the computing device 110 remains at the most recent location selected by the commanding user, and allows the following user to navigate from there. Alternatively, the computing device 110 may have stored the most recently selected location of the following user prior to the instruction to follow the commanding user, and after the commanding user releases control, the computing device 110 restores the selected location of the user.
[0060] Although a user may command and control other users' experiences of the realm, in some embodiments, the user may simply share his or her experience, while allowing the other users to retain control. The sharing user may perform similar steps to those described in Fig. 4, but with instructions to share the user's experience with the targeted users. The computing devices 110 of the recipients reserve an area of the display for depicting the experience of the sharing user (e.g., a "picture-in-picture" display), and the devices 110 execute code both for updating the image data displayed in this reserved area, and updating the image data for locations selected by the users. The devices 110 may downsample the images in the set to determine the image data to display for the experience of the sharing user, but process the images received from the server 105 to determine the image data to display for the user of the computing device 110.
[0061] Furthermore, a user may opt to follow another user. The computing device 110 may include a control for following a user, and when this control is activated, the device 110 displays a list of users to follow. By selecting a user, the computing device 110 begins displaying image data corresponding to the followed user's location, whether the device 110 receives the location as part of the followed user's broadcast or sends a request to the server 105 to receive the location of the followed user whenever this location is updated. Moreover, the user can opt to either replace the image data on its computing device 110 with the experience of the followed user, or display the experience on a reserved area, thereby viewing different locations in the realm simultaneously. While the user is following another user, the computing device 110 displays an "unfollow" control, whose selection reverts the display data to that chosen by the user.
[0062] In some scenarios, the administrative user may wish to select a different realm for the session. For example, the administrative user may be a real estate agent, and upon completing a tour of a property with a client, the real estate agent may wish to show another property. Alternatively, the client may express disinterest with a property, and the parties may select another property to determine if that property will fit the client's needs more appropriately.
[0063] Fig. 6A is an exemplary flow diagram depicting the administrative user's selection of a new realm for a session, and the propagation of that realm to the other users in the group. To select a new realm (step 605), the administrative user activates an icon on the computing device 110 to change realms. The computing device 110 displays representations of realms for the session, such as thumbnails of the first images for each of the realms. The administrative user selects one of the realms, and the computing device 110 loads the set of images associated with the selected realm (step 610). The computing device 110 displays the first image in this set, based on the default view position for the image and/or beginning of a tour of the new realm.
[0064] The computing device 110 sends an update to the server 105 regarding the new realm for the session (step 615). The server 105 sends an update to the other users in the group regarding the new realm for the session (step 620). In Fig. 6B, in an alternative embodiment, the computing device 110 send the update directly to the other users in the group (step 615'), thereby bypassing the server 105. When the other users receive this update, their computing devices 110 process the update (step 625), retrieve the set of images for the new realm from memory, and display the first image in this set (step 630).
[0065] Although the figure describes storing the sets of images for realms on computing devices 110 when the users join the session, in alternate embodiments, the users may be informed of realms for the session in real-time, and obtain new sets of images from the server 105 whenever the administrative user selects a new realm for the experience. For example, when the computing devices 110 download the executable code, the code may identify the realms via alphanumeric identifiers, by way of example, but may not contain their sets of images (nevertheless, the code may include the set for the first realm of the session). To retrieve the set of images for a realm, the computing device 110 may send a request with the alphanumeric identifier to the server 105, and in response, the server 105 may return the images. In this manner, the computing devices 110 do not store sets of images for realms until such sets are needed.
[0066] Throughout any of these exemplary sessions, the server 105 may store a user history of the session. Fig. 7 is an exemplary flow diagram for creating a session history in real time. The steps are similar to those described in Fig. 4. However, each time a user sends an updated location to the server 105, the server 105 records the change (step 720), and may also store the time that the update was received. In this manner, the server 105 creates a log of every location that each user visited, and the length of time that the user remained at that location. Thus, the server 105 may preserve the history of the session.
[0067] After the session ends, the administrative user may run analytics on the session history to ascertain the other users' interests. For example, the administrative user may instruct the server 105 to determine the percentage of time that the users spent in each image, and well as particular sections of each image. If a realm corresponds to a house, analytics may reveal the percentage of time that a user spent in the kitchen, master bedroom, bathroom, and living room. Analytics may also determine that, while the user was in the kitchen, the user spent part of the time examining the appliances, but a significant amount of time enjoying the view into the backyard. Similarly, while the user remained in the basement, the user may have spent little time looking at the heating system, but a significant amount of time looking at the foundation. In this manner, the administrative user may discern items that interest or concern a user based on their location, and select other houses (and realms) accordingly.
[0068] With reference to FIGs. 8-17, the following section describes various additional or alternative embodiments relating to an Apparatus and Method for
Facilitating Capture of Panoramic Images on a Mobile Computing Device. Reference numerals recited in this section relate to FIGs. 8-17. The fact that the same reference numeral is used in the section above relating to Systems and Methods for Enabling Shared Experiences of a Realm for Users in a Group should be disregarded.
[0069] Conventional cameras for creating panoramic images are expensive, and they can require meticulous preparation to obtain quality images. For example, some of these cameras include two lenses, each coupled to its own sensor, that face opposite directions. The cameras capture separate images using each lens-sensor pair and aggregate the images to create a panoramic image. An alternate conventional camera may include a single lens and sensor, and the camera executes software to drive a motor for rotating the lens and sensor to capture two or more images for a panoramic image. When a user deploys this type of camera, the quality of the camera set-up greatly impacts the quality of the images captured, and consequently, the resulting panoramic image.
[0070] These conventional cameras pose a number of drawbacks for users. Because they are specialized for panoramic images, they have limited versatility but still require users to incur significant costs in purchasing them. Furthermore, since the quality of the final panoramic images depends on the camera set-up, users must be skilled at preparing the cameras for image capture. Thus, conventional cameras pose a number of obstacles to obtaining panoramic image.
[0071] The inventors have developed an apparatus and methods that enable anyone with a mobile computing device (e.g., a smartphone) to obtain high quality panoramic images. The apparatus is configured to house a mobile computing device, and the apparatus further leverages features of existing panoramic image capturing software (also referred to herein as "panoramic software") on the device to ensure that the device captures excellent panoramic images. In particular, the apparatus uses the panoramic software's reliance on the device's accelerometer(s) to determine whether the device is in motion. By moving the mobile computing device to different positions for image capture, and then halting motion for predetermined periods of time, the apparatus may ensure that the mobile computing device captures the images needed to create a panoramic image, but only when the apparatus has halted motion at the designated positions.
[0072] Leveraging existing features of panoramic software obviates the need for the apparatus to communicate with the mobile computing device. Since the user does not need to initiate or ensure such communication, use of the apparatus requires less skill, especially compared to a conventional camera for capturing panoramic images. Furthermore, because the apparatus is simple, portable, and lightweight, a user may acquire the apparatus at low cost and couple it to the user's own mobile computing device for obtaining panoramic images with ease.
[0073] Elements of an exemplary apparatus 100 that leverages features of panoramic image capturing programs on mobile computing devices to facilitate the capture of panoramic images are depicted in Figs. 8 and 9. The apparatus 100 includes a receptacle 110 that is configured to house a mobile computing device (not shown). A user may mount the receptacle 110 on a platform 120 by coupling the receptacle 110 to its member 125. The platform may include a motor 130 that is coupled to the member 125, as depicted in Fig. 10. In some embodiments, the motor 130 and member 125 are coupled by a drivetrain, collectively represented by the gearbox 126 and drive gear(s) 127, though other
configurations for drivetrains may be used. By driving the member to different positions, the motor 130 consequently drives the mobile computing device in the receptacle 110 to different positions, as well.
[0074] To use the apparatus 100, a user inserts his or her mobile computing device in the receptacle 110 and mounts the receptacle 110 on the platform 120 (alternatively, the receptacle 110 may be permanently affixed to the platform 120, and the user may merely insert the device therein). Then, the user initiates the panoramic software of the mobile computing device 190 and activates the motor 130 to begin driving the member 125. In some embodiments, the user activates the motor by depressing a control, such as a button or any other physically tactile component (not shown). Alternatively, the control may activate the motor 130 by detecting that the mobile computing device has been coupled to the platform 120. For example, the control may be a sensor, or sensor system, that detects the presence of a mounted receptacle 110, a mobile computing device within the receptacle 110, or both, and which activates the motor 130 in response to such detection. In some embodiments, the sensor may monitor the weight applied to the member 125 and activate the motor 130 when the weight exceeds a threshold.
[0075] Upon activation, the motor 130 drives the member 125, and consequently, the mobile computing device, to move to different positions, and the motor 130 also halts motion at the positions for predefined periods of time. The motor's 125 activities are coordinated with the panoramic software of the mobile computing device. For example, the motor 130 may drive the member 125 to move at a velocity that is sufficiently high so as to be detectable by the mobile computing device 190. Because the device 190 determines that it is in motion, the device 190 refrains from capturing images for the panoramic image. Once the motor 130 halts, the motor 130 remains stationary for a sufficiently long period of time to trigger the panoramic software of the mobile computing device. In response to this stillness for a threshold period of time, the mobile computing device prepares its camera (e.g., focusing, color balancing, etc.) according to the scene in view, and captures an image.
[0076] Alternatively, upon activation, the motor 130 drives the member 125 on a continual basis. In these embodiments, the configuration of the drivetrain (e.g., gearbox 126 and drive gears 127) and the resultant movement of the member 125 is coordinated with the panoramic software of the mobile computing device, in lieu of the motor 130. Thus, for example, even though the motor 130 operates continuously, the drivetrain moves the member 125 to different positions, and also halts motion at the positions for predefined periods of time. When the device 190 recognizes that it has remained stationary for a sufficiently long period of time to trigger the panoramic software of the mobile computing device 190, the mobile computing device 190 prepares its camera according to the scene in view, and captures an image. [0077] Thus, as these exemplary embodiments demonstrate, the apparatus 100 provides a period of stillness that is coordinated with the period of time that mobile computing device uses to recognize its physically still state and to capture an image.
Thereafter, the apparatus 100 resumes driving the member 125 and mobile computing device to its next image capture position.
[0078] In this manner, the apparatus 100 and mobile computing device do not communicate with one another regarding their own operations. At no point does the mobile computing device recognize that it has been coupled to an apparatus 100. Moreover, even if a mobile computing device has not been mounted on the apparatus 100, if a user activates the motor 130, the motor 130 and/or drivetrain stills drive the member 125 through its sequence of positions, halting motion therebetween. Thus, the apparatus 100 and mobile computing device operate independently of one another, even though the apparatus 100 tailors its parameters to the features of panoramic software on the mobile computing device.
[0079] In various embodiments, the motor 130 and/or drivetrain may drive the member 125 angularly or linearly. When the motor 130 and/or drivetrain drives the member 125 angularly, the member 125 may be a turntable that rotates around an axis 128. The motor 130 and/or drivetrain may alternate between driving the member 125 to rotate by a particular number of degrees and halting the member 125 for a predefined period of time. Because the apparatus 100 facilitates capture of a panoramic image, the apparatus 100 may rotate the mobile computing device by a total of 360 degrees, throughout the sequence of its motions.
[0080] For example, the apparatus 100 may move the member 125 and mobile computing device to ten (10) positions for capturing images for a panoramic image. Thus, upon activation of the motor 130, the motor 130 and/or drivetrain drives the member 125 to rotate thirty-six (36) degrees before halting for a predefined period of time. After pausing to allow the panoramic software of the device 190 to prepare its settings and capture an image, the motor 130 and/or drivetrain drives the member 125 to rotate by another thirty-six (36) degrees before halting. The motor 130 and/or drivetrain repeats this sequence until the member 125 has been rotated a full three hundred and sixty (360) degrees. At this point, the motor 130 and/or drivetrain may halt motion altogether, and the panoramic software of the mobile computing device may recognize that the camera can no longer be expected to move, and the device begins aggregating the captured images into a single panoramic image. Although this example describes angular increments of thirty-six (36) degrees for a total of 10 positions for capturing images, any other angular increment, and corresponding number of positions, may be used.
[0081] As previously mentioned, in embodiments that drive the receptacle 110 angularly, the member 125 may be a rotary table. Furthermore, the receptacle 110 and member 125 may be configured such that, when a mobile computing device 190 is inserted into the receptacle 110, the device 190 is offset radially from the axis of rotation 128 of the member 125 by a predefined distance, so as to align the entrance pupil 192 of the lens system 195 with the axis 128. As a result, as the motor 130 and/or drivetrain drives the member 125 to rotate around its axis 128, the lens system 195 rotates around the axis 128 while the device 190 rotates around the axis 128 at an offset. The offset ensures that the mobile computing device 190 captures images suited for panoramic images, and the offset may be selected to optimize the quality of the images for this purposes. In some embodiments, the offset may be less than about 0.5 inches.
[0082] Figs. 11 and 12 depict two exemplary views of a receptacle 110 mounted on a member 125 that illustrates a radial offset between the device 190 and the axis of rotation 128 of the member 125, which ensures alignment between the axis 128 and the lens system 195. Fig. 13 depicts an overhead view of the receptacle 110 demonstrating that the imaging system of a mobile computing device would align with the entrance pupil 192 of the receptacle 110.
[0083] When the motor 130 and/or drivetrain drives the member 125 linearly, the member 125 may progress along a linear path predefined by a groove in the platform. In these embodiments, the motor 130 and/or drivetrain drives the member 125 to move down the path by a particular distance, and halts the member 125 for a predetermined period of time before driving the member 125 further down the path. The member 125 may move at a constant velocity, although in some embodiments, the motor 130 and/or drivetrain may vary the velocity of the member 125.
[0084] In any of the above embodiments, the member 125 may be coupled to the motor 130 via a set of gears. Figs. 14-17 depict top, side, bottom, and isometric views of an exemplary set of gears that may be used in the platform 120 of the apparatus 100. In this embodiment, the motor 130 drives a gearbox 805 to rotate a first gear 810. The teeth of the first gear 810 are engaged with the teeth of a second gear 815, which in turn are engaged with the teeth of a third gear 820 attached to the member 125 (in this case, a rotary table). Thus, rotation of the first gear 810 via the motor 130 drives the rotation of the rotary table 125.
[0085] In many embodiments, the configuration of teeth for at least one of the gears may determine the positions that the member 125 and mobile computing device will assume. For example, one of the gears may have teeth of different length. When one of the longer teeth of a gear engages another gear in the set, motion of the member 125 begins, and motion halts when the shorter teeth of the gear fail to engage with the other gear in the set. The position where the member 125 halts depends on the position of the shorter teeth on the gear, as well as the gear's relationship to others in the set (e.g., relative size). Further, the amount of time that the member 125 remains stationary may depend on the form factor of the engaged teeth. As a result, the positions where the member 125 halts motion, as well as the duration of those periods, can be controlled by selecting the size of the gear, the angular frequency of the larger teeth, and the form factors of the larger teeth, among other parameters. In this manner, activating the motor 130 may cause the motor 130 to drive the set of gears for a predetermined period of time, but the configuration of the gears, both individually and as a set, define the positions that the member 125 and mobile computing device attain, and the length of time they remain in such positions.
[0086] Alternatively, the platform 120 may include a processor and a memory for storing computer instructions. When the motor 130 is activated, the processor may load and execute computer instructions from the memory, and these instructions may control the manner in which the motor 130 drives the member 125. For example, based on the instructions, the processor may alternate between driving the member 125 at a particular velocity for a period of time, and halting all motion. Because the parameters for motion and stillness are encoded in the computer instructions, these embodiments require simpler sets of gears than the embodiments that rely exclusively on the set of gears to determine positions for the member 125 and the length(s) of time for halting motion at the positions.
[0087] Other Features:
[0088] When the receptacle 110 has a lens system 192, the lens system 192 must be positioned so that the lens of an inserted mobile computing device 190 aligns with the lens system 192 on the receptacle 110. As shown in Fig. 13, the receptacle 110 may be configured to position a mobile computing device 190 to ensure this alignment. Enabling the device to leverage its own lens and the lens system 192 of the receptacle 110 can improve the quality of images obtained for panoramic images. As discussed above, and depicted in Fig. 11, the lens system 192 of the receptacle 100 must also be positioned so that its entrance pupil 192 is aligned with the axis of rotation 128 of the member 125.
[0089] Although the embodiments depicted in Figs. 9 and 11-13 illustrate a lens system 195 affixed to the receptacle 110, and positioned to align with the lens of the mobile computing device, in various embodiments, the receptacle 110 does not have an attached lens system 195. Instead, the mobile computing device includes the sole lens (or lenses) used for image capture. The receptacle 110 would still be configured to position the entrance pupil of the lens of the mobile computing device according to the same principles guiding the positioning of the lens system 192 affixed to a receptacle 110.
[0090] As depicted in Figs. 9 and 13, the receptacle 110 includes an open sleeve 111 with arms 112, 113 configured to receive a mobile computing device 190. A user may insert the device into the sleeve 111, and the arms 112, 113 hold the device in place. However, any other mechanism for maintaining the position of the mobile computing device may be used. For example, the receptacle 110 may include a slot 111 whose front and back faces define a space for housing the device. In another example, the receptacle 110 may include clamps that hold the device in place. Alternatively, the receptacle 110 may include flexible molded components that define at least part of the perimeter of the space for housing the device. When a user presses the device into the space, the component flex to accept and secure the device.
[0091] Furthermore, the receptacle 110 may be mounted on the platform 120 in any manner desired. In Figs. 8, 9, 11, and 12, the receptacle 110 is coupled to the member 125 by sliding a portion of the receptacle 110 into a groove of the member 125. However, any other method of mounting or coupling may be used. For example, the receptacle 110 may include one or more pins, and the member 125 may include one or more holes that receive the pins of the receptacle 110.
[0092] Various embodiments of the present invention may be characterized by the potential claims listed in the paragraphs following this paragraph (and before the actual claims provided at the end of this application). These potential claims form a part of the written description of this application. Accordingly, subject matter of the following potential claims may be presented as actual claims in later proceedings involving this application or any application claiming priority based on this application. Inclusion of such potential claims should not be construed to mean that the actual claims do not cover the subject matter of the potential claims. Thus, a decision to not present these potential claims in later proceedings should not be construed as a donation of the subject matter to the public.
[0093] Without limitation, potential subject matter that may be claimed (prefaced with the letter "P" so as to avoid confusion with the actual claims presented below) includes:
[0094] PI . A portable device for facilitating capture of panoramic images on a mobile computing device, the device comprising:
[0095] a receptacle configured to house a mobile computing device; and
[0096] a platform comprising (1) a member, with an axis of rotation, configured to interface with the receptacle to offset radially the mobile computing device from the axis of rotation by a predefined distance such that an entrance pupil of a lens system of the mobile computing device is aligned with the axis, and (2) a motor that drives the member at different velocities so that the mobile computing device captures an image for a panoramic image when a velocity of the member is substantially zero for a threshold period of time.
[0097] P2. The portable device of claim PI, wherein the velocities are angular velocities.
[0098] P3. The portable device of claim PI, wherein the member is a circular member.
[0099] P4. The portable device of claim PI, wherein the motor is a mechanical motor.
[00100] P5. The portable device of claim PI, wherein the motor is coupled to a processor that executes code to drive the member.
[00101] P6. The portable device of claim PI, wherein the motor drives the member to alternate between rotating and remaining still.
[00102] P7. The portable device of claim P6, wherein the motor drives the member to rotate by a predetermined number of degrees. [00103] P8. The portable device of claim P7, wherein the predetermined number of degrees is thirty-six (36).
[00104] P9. The portable device of claim PI, further comprising a drivetrain coupled to the motor and the member, wherein the motor drives the member by driving the drivetrain.
[00105] P10. The portable device of claim P9, wherein the drivetrain comprises at least one drive gear with teeth of different lengths, wherein the teeth of different lengths determine the different velocities at which the member is driven.
[00106] PI 1. The portable device of claim PI, wherein the predefined distance of radial offset between the mobile computing device and the axis of rotation of the member is less than 0.5 inches.
[00107] PI 2. A method for facilitating capture of panoramic images on a mobile computing device, the method comprising:
[00108] initiating, by a mobile computing device, a program to capture images of a scene for a panoramic image when the mobile computing device is stationary;
[00109] capturing, by a mobile computing device, a first image of a scene at a first location;
[00110] driving, by a motor, a member configured to interface with a receptacle containing the mobile computing device from the first location to a second location;
[00111] halting, by the motor, further motion of the member for a
predetermined period of time;
[00112] determining, by a sensor of the mobile computing device, a lack of motion of the mobile computing device for a threshold period of time;
[00113] in response to the lack of motion, capturing, by the mobile computing device, a second image of the scene at the second location; and
[00114] creating the panoramic image from the first and second images of the scene.
[00115] P13. The method of claim PI 2, wherein driving the member comprises: driving, by the motor, the member from the first location to the second location at a non-zero angular velocity. [00116] P14. The method of claim PI 2, wherein driving the member comprises: driving, by the motor, the member to rotate from the first location to the second location.
[00117] P15. The method of claim PI 2, wherein driving the member comprises: driving, by the motor, the member to rotate by a predetermined number of degrees.
[00118] P16. The method of claim PI 2, wherein driving the member comprises: driving, by the motor, a drivetrain coupled to the member, wherein the drivetrain comprises at least one drive gear with teeth of different lengths and positions of the teeth of different lengths determine when the member halts.
[00119] PI 7. A computer-readable medium having computer program code non-transitorily stored thereon, the computer program code for facilitating capture of panoramic images on a mobile computing device, the program code comprising:
[00120] program code for monitoring motion of the mobile computing device, including substantially zero and substantially non-zero velocities of the mobile computing device; and
[00121] program code for capturing at least one image of a scene in response to determinations that the mobile computing device has remained at a substantially zero velocity for a threshold period of time.
[00122] Thus, in accordance with one embodiment of the invention, a portable device facilitates capture of panoramic images on a mobile computing device. The device includes a receptacle configured to house a mobile computing device. The device also includes a platform with a member, with an axis of rotation, that is configured to interface with the receptacle to offset radially the mobile computing device from the axis of rotation by a predefined distance, such that an entrance pupil of a lens system of the mobile computing device is aligned with the axis. The platform also includes a motor that drives the member at different velocities so that the mobile computing device captures an image for a panoramic image when a velocity of the member is substantially zero for a threshold period of time.
[00123] The velocities may be angular velocities. The member may be a circular member. The motor may be a mechanical motor. The motor may be coupled to a processor that executes code to drive the member. The motor may drive the member to alternate between rotating and remaining still. Furthermore, the motor may drive the member to rotate by a predetermined number of degrees. The predetermined number of degrees may be thirty-six (36). The predefined distance of radial offset between the mobile computing device and the axis of rotation of the member may be less than 0.5 inches.
[00124] In accordance with another embodiment of the invention, a method facilitates capture of panoramic images on a mobile computing device. The method initiates, by a mobile computing device, a program to capture images of a scene for a panoramic image when the mobile computing device is stationary. The method also captures, by a mobile computing device, a first image of a scene at a first location. The method further drives, by a motor, a member configured to interface with a receptacle containing the mobile computing device from the first location to a second location. The method additionally halts, by the motor, further motion of the member for a predetermined period of time. The method determines, by a sensor of the mobile computing device, a lack of motion of the mobile computing device for a threshold period of time. In response to the lack of motion, the method captures, by the mobile computing device, a second image of the scene at the second location. Lastly, the method creates the panoramic image from the first and second images of the scene.
[00125] The method may drive, by the motor, the member from the first location to the second location at a non-zero angular velocity. Alternatively, the method may drive, by the motor, the member to rotate by a predetermined number of degrees.
[00126] In accordance with another embodiment of the invention, a computer- readable medium has computer program code non-transitorily stored thereon, and the computer program code facilitates capture of panoramic images on a mobile computing device. The program code includes code for monitoring motion of the mobile computing device, including substantially zero and substantially non-zero velocities of the mobile computing device. The program code also includes code for capturing at least one image of a scene in response to determinations that the mobile computing device has remained at a substantially zero velocity for a threshold period of time.
[00127] The embodiments of the invention described above are intended to be merely exemplary; numerous variations and modifications will be apparent to those skilled in the art. All such variations and modifications are intended to be within the scope of the present invention as defined in any appended claims.

Claims

What is claimed is:
1. A method of coordinating experiences of a realm for a group of users wherein the group includes at least an administrative user and at least one participating user, the method comprising:
transmitting, from the server to an administrative user, a session identifier associated with the realm;
receiving, by the server, a user acceptance to join a session using the session identifier so that the at least one participating user can experience the realm;
in response to receiving the acceptance, transmitting to a computing device of the participating user, by the server, executable code including data describing a series of locations in the realm, the series of locations comprising at least a first location and a second location, wherein the first location is associated with a first panoramic image and the second location is associated with a second panoramic image, wherein the executable code is configured to (1) receive instructions from at least one computing device of at least one user within the group to display the realm at at least one selected location, and (2) display, on the computing device, data corresponding to the realm at at least one of the locations selected by at least one of the users within the group.
2. The method of claim 1, wherein the executable code is further configured to (3) receive instructions from the server regarding locations selected by a user within the group.
3. The method of claim 1, further comprising:
receiving, by the server from the administrative user, a request to synchronize an experience of the realm for the participating user with an experience of the realm for the administrative user, the request including a location in the realm selected by the administrative user; transmitting, by the server, the location in the realm selected by the administrative user to a computing device of the participating user, the location causing the executable code running on the computing device of the participating user to display data
corresponding to the location selected by the administrative user.
4. The method of claim 1, further comprising:
transmitting, by the server to a computing device of the participating user, locations in the realm selected by the users of the group;
receiving, by the server from the administrative user, a request to synchronize an experience of the realm for the participating user with an experience of the realm for the administrative user;
in response to the request from the administrative user, transmitting, by the server to the computing device of the participating user, an instruction to display data corresponding to the locations in the realm selected by the administrative user.
5. The method of claim 3, wherein the computing device of the participating user creates at least one virtual image representing a view of the realm between a location selected by the participating user and the location selected by the administrative user, and
displays data corresponding to the virtual image and the location selected by the administrative user to simulate motion in the realm from the location selected by the participating user to the location selected by the administrative user.
6. The method of claim 4, wherein the computing device of the participating user creates at least one virtual image representing a view of the realm between a location selected by the participating user and the location selected by the administrative user, and displays data corresponding to the virtual image and the location selected by the administrative user to simulate motion in the realm from the location selected by the participating user to the location selected by the administrative user.
7. The method of claim 3, further comprising:
receiving, by the server from the computing device of the administrative user, an updated location selected by the administrative user; and
transmitting, by the server to the computing device of the participating user, the updated selected location so that the computing device of the participating user displays data corresponding to the updated selected location in the realm.
8. The method of claim 4, further comprising:
receiving, by the server from the computing device of the administrative user, an updated location selected by the administrative user; and
transmitting, by the server to the computing device of the participating user, the updated selected location so that the computing device of the participating user displays data corresponding to the updated selected location in the realm.
9. The method of claim 3, wherein the instruction to synchronize the experiences of the participating user and the administrative user causes the computing device of the participating user to disregard instructions from the participating user.
10. The method of claim 4, wherein the instruction to synchronize the experiences of the participating user and the administrative user causes the computing device of the participating user to disregard instructions from the participating user.
11. The method of claim 1, further comprising: receiving, by the server from a first user in the group, a request to synchronize an experience of the realm for a second user in the group with an experience of the realm for the first user.
12. The method of claim 1, wherein the selected location in the realm includes a view direction.
13. The method of claim 1, further comprising:
receiving, by the server from an administrative user, a request to view a list of users who have joined the experience of the realm; and
transmitting, by the server to a computing device of the administrative user, the list of users for display.
14. The method of claim 1, further comprising:
receiving, by the server from the administrative user, a request to view a history of selected locations in the realm for a user in the group of users; and
transmitting, by the server to a computing device of the administrative user, the history of selected locations in the realm for the user.
15. The method of claim 1, further comprising:
receiving, by the server from the administrative user, a request to follow the view of the participating user;
sending, by the server to the computing device of the participating user, an instruction to provide, on a continual basis, updates on the location in the realm selected by the participating user;
forwarding, by the server to the computing device of the administrative user, the updated location selected by the participating user, the updated location causing the executable code running on the computing device of the administrative user to display data corresponding to the realm at the updated selected location of the participating user.
16. The method of claim 1, further comprising transmitting, by the server to a computing device of the administrative user, locations in the realm selected by the users of the group, wherein the executable code on the computing device of the administrative user is configured to (1) receive a request to follow the view of the participating user and (2) display data corresponding to the locations in the realm received from the server that are selected by the participating user.
17. The method of claim 1, further comprising:
receiving, by the server from an administrative user, an instruction to display a second realm to the group of users; and
transmitting, by the server to each computing device for the group of users, executable code including data describing a series of locations in the second realm.
18. A method of coordinating experiences of a realm for a group of users wherein the group includes at least an administrative user and at least one participating user, the method comprising:
transmitting, from the server to an administrative user, a session identifier associated with the realm;
receiving, by the server, a user acceptance to join a session using the session identifier so that the at least one participating user can experience the realm;
in response to receiving the acceptance, transmitting to a computing device of the participating user, by the server, executable code including a set of panoramic images associated with the realm, wherein the executable code is configured to (1) receive instructions from at least one computing device of at least one user within the group to display the realm at at least one selected location, and (2) display, on the computing device, data from the set of panoramic images corresponding to the at least one selected location.
PCT/US2017/044342 2016-07-29 2017-07-28 Systems and methods for providing individual and/or synchronized virtual tours through a realm for a group of users WO2018022977A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CA3071398A CA3071398A1 (en) 2016-07-29 2017-07-28 Systems and methods for providing individual and/or synchronized virtual tours through a realm for a group of users
EP17749318.6A EP3491589A1 (en) 2016-07-29 2017-07-28 Systems and methods for providing individual and/or synchronized virtual tours through a realm for a group of users

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US201662368471P 2016-07-29 2016-07-29
US15/224,068 2016-07-29
US15/224,077 US11153355B2 (en) 2016-07-29 2016-07-29 Systems and methods for providing individual and/or synchronized virtual tours through a realm for a group of users
US62/368,471 2016-07-29
US15/224,077 2016-07-29
US15/224,068 US20180035048A1 (en) 2016-07-29 2016-07-29 Apparatus and Method for Facilitating Capture of Panoramic Images on a Mobile Computing Device

Publications (1)

Publication Number Publication Date
WO2018022977A1 true WO2018022977A1 (en) 2018-02-01

Family

ID=59558518

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2017/044342 WO2018022977A1 (en) 2016-07-29 2017-07-28 Systems and methods for providing individual and/or synchronized virtual tours through a realm for a group of users

Country Status (3)

Country Link
EP (1) EP3491589A1 (en)
CA (1) CA3071398A1 (en)
WO (1) WO2018022977A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070271338A1 (en) * 2006-05-18 2007-11-22 Thomas Anschutz Methods, systems, and products for synchronizing media experiences
US20100306671A1 (en) * 2009-05-29 2010-12-02 Microsoft Corporation Avatar Integrated Shared Media Selection
WO2014087555A1 (en) * 2012-12-06 2014-06-12 Nec Corporation Communication system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070271338A1 (en) * 2006-05-18 2007-11-22 Thomas Anschutz Methods, systems, and products for synchronizing media experiences
US20100306671A1 (en) * 2009-05-29 2010-12-02 Microsoft Corporation Avatar Integrated Shared Media Selection
WO2014087555A1 (en) * 2012-12-06 2014-06-12 Nec Corporation Communication system

Also Published As

Publication number Publication date
EP3491589A1 (en) 2019-06-05
CA3071398A1 (en) 2018-02-01

Similar Documents

Publication Publication Date Title
US10269089B2 (en) Multi-user virtual reality processing
US8447070B1 (en) Approaches for device location and communication
US10981061B2 (en) Method and system for directing user attention to a location based game play companion application
US20220038512A1 (en) Systems and Methods for Providing Individual and/or Synchronized Virtual Tours Through a Realm for a Group of Users
US8655980B2 (en) Networked computer system for communicating and operating in a virtual reality environment
CN100451897C (en) Remotely-operated robot, and robot self position identifying method
US10410320B2 (en) Course profiling and sharing
US9798382B2 (en) Systems and methods of eye tracking data analysis
JP2023504570A (en) Cross-reality system with simplified programming of virtual content
CN107066092B (en) VR running space dynamic detection and parameterized virtual scene reconstruction system and method
KR20170010815A (en) Methods and systems for augmented reality to display virtual representations of robotic device actions
CN105979133A (en) Tracking shooting method, mobile terminal and system
EP3469804A1 (en) Game play companion application
CN108255291B (en) Virtual scene data transmission method and device, storage medium and electronic device
EP2745488B1 (en) Method and apparatus for accessing a virtual object
WO2018022977A1 (en) Systems and methods for providing individual and/or synchronized virtual tours through a realm for a group of users
US10845875B1 (en) Gaze initiated interaction technique
CN103533149A (en) Mobile phone robot system and application method thereof
US20200380786A1 (en) System and method for analyzing and distributing volumetric video
Bedin et al. Robots and IoT devices for assistive automation
KR102571086B1 (en) Method and system for supporting collaboration among multiple users using virtual space
CN113813607B (en) Game view angle switching method and device, storage medium and electronic equipment
CN109213307A (en) A kind of gesture identification method and device, system
US20180035048A1 (en) Apparatus and Method for Facilitating Capture of Panoramic Images on a Mobile Computing Device
CN110505682B (en) Image display device control information transmission method and system based on Internet of things and computer readable medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17749318

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2017749318

Country of ref document: EP

Effective date: 20190228

ENP Entry into the national phase

Ref document number: 3071398

Country of ref document: CA