US20200020159A1 - Systems and methods for navigating a user in a virtual reality model - Google Patents
Systems and methods for navigating a user in a virtual reality model Download PDFInfo
- Publication number
- US20200020159A1 US20200020159A1 US16/581,741 US201916581741A US2020020159A1 US 20200020159 A1 US20200020159 A1 US 20200020159A1 US 201916581741 A US201916581741 A US 201916581741A US 2020020159 A1 US2020020159 A1 US 2020020159A1
- Authority
- US
- United States
- Prior art keywords
- route
- point positions
- point
- property
- introduction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000004891 communication Methods 0.000 description 16
- 230000006870 function Effects 0.000 description 8
- 239000003795 chemical substances by application Substances 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000009877 rendering Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 230000007704 transition Effects 0.000 description 4
- 241000196324 Embryophyta Species 0.000 description 3
- 238000005034 decoration Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000009432 framing Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 241000282575 Gorilla Species 0.000 description 1
- 241000124033 Salix Species 0.000 description 1
- 230000002730 additional effect Effects 0.000 description 1
- 235000021152 breakfast Nutrition 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000003278 mimic effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010422 painting Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000011282 treatment Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/16—Real estate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
- G06Q10/047—Optimisation of routes or paths, e.g. travelling salesman problem
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/16—Real estate
- G06Q50/163—Real estate management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/003—Navigation within 3D models or images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2210/00—Indexing scheme for image generation or computer graphics
- G06T2210/21—Collision detection, intersection
Definitions
- the present disclosure relates to systems and methods for conducting a property tour using virtual reality technology, and more particularly, to systems and methods for navigating a user in a three-dimensional model of a property through a selected route linking a plurality of point positions.
- House tours are often conducted in person. e.g., by the property owner or a real estate agent. During the tour, the visitor gets to see the property and the property owner/agent accompanying the tour will introduce features of the property to the visitor. For example, when the visitor is in the kitchen, features such as size of the room, cabinet and counter space, installed appliances, and lighting conditions may be introduced orally. While property tours are informative to the potential buyer/renter, it is also time consuming and inefficient.
- VR Virtual Reality
- a property typically includes multiple rooms or functional spaces, such as kitchen, living room, dining room, breakfast nook, study, bedrooms, and bathrooms.
- conventional VR touring technologies typically transitions the views directly from one room to another room, for example by suddenly switching the images. Such a transition contradicts the user experience in real life, where the visitor would have been ushered into the next room. For example, as he walks through the hallway and doors connecting the rooms, he would have continued to experience the layout of the property, as well as additional property views along the way. All these become lost due to the abrupt transition by the conventional VR touring technologies.
- Embodiments of the disclosure improve the existing VR touring systems and methods by navigating a user in a three-dimensional model of a property through a selected route linking a plurality of point positions.
- a system for navigating a user in a three-dimensional model of a property includes a storage device configured to store data associated with a plurality of point positions. Each point position corresponds to a camera position at which images and point cloud data of the property are captured by the camera.
- the system further includes at least one processor configured to determine connectivity among the point positions and determine a plurality of candidate routes connecting a first introduction position and a second introduction position. Each candidate route links a subset of the point positions based on the connectivity.
- the at least one processor is further configured to select a route from the candidate routes and sequentially display views corresponding to the point positions on the selected route. The views at each point position are rendered from the images captured at the camera position corresponding to the point position.
- a computer-implemented method for navigating a user in a three-dimensional model of a property includes receiving data associated with a plurality of point positions. Each point position corresponds to a camera position at which images of the property are captured by the camera. The method further includes determining, by at least one processor, connectivity among the point positions and determining, by the at least one processor, a plurality of candidate routes connecting a first introduction position and a second introduction position. Each candidate route links a subset of the point positions based on the connectivity. The method also includes selecting, by the at least one processor, a route from the candidate routes and sequentially displaying, on a display, views corresponding to the point positions on the selected route. The views at each point position are rendered by the at least one processor from the images captured at the camera position corresponding to the point position.
- a non-transitory computer-readable medium having stored thereon computer instructions when executed by at least one processor, perform a method for navigating a user in a three-dimensional model of a property.
- the method includes receiving data associated with a plurality of point positions. Each point position corresponds to a camera position at which images of the property are captured by the camera.
- the method further includes determining connectivity among the point positions and determining a plurality of candidate routes connecting a first introduction position and a second introduction position. Each candidate route links a subset of the point positions based on the connectivity.
- the method also includes selecting a route from the candidate routes and sequentially displaying views corresponding to the point positions on the selected route. The views at each point position are rendered from the images captured at the camera position corresponding to the point position.
- FIG. 1 illustrates a schematic diagram of an exemplary three-dimensional model including multiple rooms, according to embodiments of the disclosure.
- FIG. 2 illustrates a schematic diagram of an exemplary room in a three-dimensional model, according to embodiments of the disclosure.
- FIG. 3 is a block diagram of an exemplary system for navigating a user in a three-dimensional model, according to embodiments of the disclosure.
- FIG. 4 is a flowchart of an exemplary method for navigating a user in a three-dimensional model, according to embodiments of the disclosure.
- FIG. 5 is a flowchart of an exemplary method for determining connectivity among point positions, according to embodiments of the disclosure.
- FIG. 6 illustrates exemplary candidate routes, according to embodiments of the disclosure.
- FIG. 7 is a flowchart of an exemplary method for selecting a route from candidate routes, according to embodiments of the disclosure.
- FIG. 1 illustrates a schematic diagram of an exemplary three-dimensional model 100 including multiple rooms, according to embodiments of the disclosure.
- three-dimensional model 100 may be a model of a property, such as a house, an apartment, a townhouse, a garage, a warehouse, an office building, a hotel, and a store, etc. As shown in FIG.
- three-dimensional model 100 virtually recreates the real-world property including its layout (e.g., the framing structures that divide the property into several rooms such as walls and counters), finishing (e.g., kitchen/bathroom cabinets, bathtub, island, etc.), fixtures installed (e.g., appliances, window treatments, chandeliers, etc.), and furniture and decorations (e.g., beds, desks, tables and chairs, sofas, TV stands, bookshelves, wall paintings, mirrors, plants, etc.)
- layout e.g., the framing structures that divide the property into several rooms such as walls and counters
- finishing e.g., kitchen/bathroom cabinets, bathtub, island, etc.
- fixtures installed e.g., appliances, window treatments, chandeliers, etc.
- furniture and decorations e.g., beds, desks, tables and chairs, sofas, TV stands, bookshelves, wall paintings, mirrors, plants, etc.
- Three-dimensional model 100 may be divided into multiple rooms or functional spaces by interior walls. Some rooms may have multiple functions.
- three-dimensional model 100 may include a great room 110 that has combined functions of a living room and a kitchen. Three-dimensional model 100 may further include two bedrooms 120 and 130 , their attached bathrooms, as well as balconies.
- FIG. 2 illustrates a schematic diagram of an exemplary great room 110 in three-dimensional model 100 , according to embodiments of the disclosure. The rooms may be connected by hallways and doors.
- Each room of three-dimensional model 100 includes at least one introduction positions.
- great room 110 has an introduction position 112 and bedroom 120 has an introduction position 115 .
- a “introduction position” is a position in the three-dimensional model that corresponds to a physical position that the user would stand in the room when listening to introductions by the property owner/agent.
- the user of the VR tool views the rooms from the introduction positions.
- audio content may be displayed to the user at the introduction positions to orally introduce the features of the respective rooms.
- the introduction may be provided in real-time by property owner/agent, or pre-recorded and replayed when the user is at the introduction position.
- point positions 111 and 113 are located in great room 110
- point position 114 is located in bedroom 120 .
- a “point position” is a position in the three-dimensional model that corresponds to a camera position where images and videos of the property are captured.
- introduction positions 112 and 115 may also be point positions.
- a user 210 may be placed at introduction position 112 in great room 110 for the room features to be introduced to him.
- User 210 may view images or videos of great room 110 and listen to oral introductions at introduction position 112 .
- a camera 220 may be placed at point position 113 and/or 112 to capture images and/or videos of great room 110 .
- camera 220 may be a digital camera, a binocular camera, a video camera, a panoramic camera, etc.
- camera 220 may be configured to capture three-dimensional information, (e.g., including depth information) of the room.
- Camera 220 may be mounted on a tripod or be carried by an operator. Camera 220 may be rotated or otherwise adjusted to capture images or videos from different angles and heights.
- the VR tool may transition from one room to another room of three-dimensional model, during the VR house tour. For example, when user 210 finishes touring great room 110 , he may want to move on to bedroom 120 . Consistent with the present disclosure, systems and methods for navigating user 210 in three-dimensional model 100 of a property (e.g., from great room 110 to bedroom 120 ) is provided. Instead of transitioning from views at introduction position 112 in the first room directly to views at introduction position 115 in the second room, the disclosed systems and methods navigate user 210 from introduction position 112 to introduction position 115 along a route that passes one or more point positions, e.g., point positions 113 and 114 . In some embodiments, the disclosed systems and methods selects the route from multiple candidate routes. The candidate routes are different paths that the user can follow to get from introduction position 112 in great room 110 to introduction position 115 in bedroom 120 . Each candidate passes different sets of point positions.
- the candidate routes are different paths that the user can follow to get from introduction position 112 in great room 110 to introduction
- the disclosed systems and methods first determine connectivity among the relevant point positions and then establish the candidate routes connecting the introduction positions 112 and 115 by linking a set of the point positions that are determined to be connectable. The disclosed systems and methods then select the best route from the candidate routes based on certain criteria. Views corresponding to the point positions on the selected route may be sequentially displayed to the user to provide a more real experience as the user walks from great room 110 to introduction position 115 through the route.
- the views at each point position are rendered from the images previously captured at the camera position corresponding to the point position.
- the views are oriented in the direction the user is facing as he walks towards each point position. For example, the user may be facing the direction pointing from introduction position 112 to point position 113 , and thus the views at point position 113 will be rendered in that direction when shown to the user.
- the exemplary three-dimensional model shown in FIG. 1 illustrates only indoor space and structures, it is contemplated that the model could also include outdoor space and structures, such as the deck, the front/back yard, the garage, as well as the neighboring environment.
- a user may be navigated from an introduction position indoor to an introduction outdoor or from an introduction position outdoor to an introduction indoor door through, e.g., doors.
- a person of ordinary skill can adapt the disclosed systems and methods without undue experimentation for outdoor showings.
- FIG. 3 is a block diagram of an exemplary system 300 for navigating a user in a three-dimensional model, according to embodiments of the disclosure.
- system 300 may be implemented by a physical server or a service in the cloud.
- system 300 may be implemented a computer or a consumer electronic device such as a mobile phone, a pad, or a wearable device.
- system 300 may include a communication interface 302 , a processor 304 , a memory 306 , a storage 308 , and a bus 310 .
- system 300 may have different modules in a single device, such as an integrated circuit (IC) chip (implemented as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA)), or separate devices with dedicated functions.
- IC integrated circuit
- ASIC application-specific integrated circuit
- FPGA field-programmable gate array
- Components of system 300 may be in an integrated device, or distributed at different locations but communicate with each other through a network (not shown).
- the various components of system 300 may be connected to and communicate with each other through bus 310 .
- Communication interface 302 may send data to and receive data from components such as camera 220 and database 301 via direct communication links, a Wireless Local Area Network (WLAN), a Wide Area Network (WAN), wireless communication networks using radio waves, a cellular network, and/or a local wireless network (e.g., BluetoothTM or WiFi), or other communication methods.
- communication interface 302 can be an integrated services digital network (ISDN) card, cable modem, satellite modem, or a modem to provide a data communication connection.
- ISDN integrated services digital network
- communication interface 302 can be a local area network (LAN) card to provide a data communication connection to a compatible LAN.
- Wireless links can also be implemented by communication interface 302 .
- communication interface 302 can send and receive electrical, electromagnetic or optical signals that carry digital data streams representing various types of information via a network.
- communication interface 302 may receive a three-dimensional model of a property.
- the model may be constructed based on point cloud data as well as images captured of the property.
- the three-dimensional model may be rendered in real-time of the VR tour or offline prior to the tour.
- communication interface 302 may receive data of camera positions in the property, and images/videos of the property taken at those camera positions.
- the three-dimensional model, the camera positions, and the images/videos may be stored in database 301 .
- Communication interface 302 may provide the received information or data to memory 306 and/or storage 308 for storage or to processor 304 for processing.
- Processor 304 may include any appropriate type of general-purpose or special-purpose microprocessor, digital signal processor, or microcontroller. Processor 304 may be configured as a separate processor module dedicated to providing VR house tours. Alternatively, processor 404 may be configured as a shared processor module for performing other functions related to or unrelated to VR house tours. For example, VR house tour is just one application installed on a versatile device.
- processor 304 may include multiple modules, such as a point position determination unit 340 , a candidate route determination unit 342 , a route selection unit 344 , and a view rendering unit 346 , and the like. These modules (and any corresponding sub-modules or sub-units) can be hardware units (e.g., portions of an integrated circuit) of processor 304 designed for use with other components or to execute part of a program.
- the program may be stored on a computer-readable medium (e.g., memory 306 and/or storage 308 ), and when executed by processor 304 , it may perform one or more functions.
- FIG. 3 shows units 340 - 346 all within one processor 304 , it is contemplated that these units may be distributed among multiple processors located near or remotely with each other.
- Memory 306 and storage 308 may include any appropriate type of mass storage provided to store any type of information that processor 304 may need to operate.
- Memory 306 and storage 308 may be a volatile or non-volatile, magnetic, semiconductor, tape, optical, removable, non-removable, or other type of storage device or tangible (i.e., non-transitory) computer-readable medium including, but not limited to, a ROM, a flash memory, a dynamic RAM, and a static RAM.
- Memory 306 and/or storage 308 may be configured to store one or more computer programs that may be executed by processor 304 to perform user navigation functions in a three-dimensional model disclosed herein.
- memory 306 and/or storage 308 may be configured to store program(s) that may be executed by processor 304 to provide a VR tour of a property to a user by introducing property features to the user at introduction positions in a three-dimensional model and navigating the user in the model among the introduction positions.
- Memory 306 and/or storage 308 may be further configured to store information and data used by processor 304 .
- memory 306 and/or storage 308 may be configured to store various camera positions and images/videos of a property captured by camera 220 at the camera positions.
- Memory 306 and/or storage 308 may also be configured to store intermediate data generated by processor 304 , such as point positions in a three-dimensional model corresponding to the camera positions, connectivity of the point positions, candidate routes connection two introduction positions, and rendered views at point positions.
- the various types of data may be stored permanently, removed periodically, or disregarded immediately after each frame of data is processed.
- system 300 may optionally include a display 303 .
- display 303 may be external to system 300 but be connected with system 300 .
- Display 303 may display the views in the three-dimensional model to the users.
- display 303 may further function as a user interface to receive user input.
- Display 303 may include a Liquid Crystal Display (LCD), a Light Emitting Diode Display (LED), a plasma display, or any other type of display, and provide a Graphical User Interface (GUI) presented on the display for user input and data display.
- the display may include a number of different types of materials, such as plastic or glass, and may be touch-sensitive to receive commands from the user.
- the display may include a touch-sensitive material that is substantially rigid, such as Gorilla GlassTM, or substantially pliable, such as Willow GlassTM.
- FIG. 4 is a flowchart of an exemplary method 400 for navigating a user in a three-dimensional model, according to embodiments of the disclosure.
- Method 400 may include steps S 402 -S 420 as described below. It is to be appreciated that some of the steps may be optional to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown in FIG. 4 .
- method 400 will be described as to navigate a user from great room 110 to bedroom 120 of three-dimensional model 100 (as shown in FIG. 1 ). Method 400 , however, can be implemented for navigating between other rooms of the model, between indoor and outdoor spaces, or for different models for the properties.
- method 400 may be triggered when system 300 receives an instruction to move the user from the current room to the next room. In some embodiments, system 300 may automatically initiate method 400 when it detects that the introduction of the current room is about to finish.
- communication interface 302 may receive a three-dimensional model of a property, such as three-dimensional model 100 .
- the three-dimensional model may be a model of a property, such as a house, an apartment, a townhouse, a garage, a warehouse, an office building, a hotel, and a store, etc.
- the three-dimensional model may be constructed based on point cloud data of the property as well as image/videos captured of the property. The three-dimensional model may virtually recreate the property including its layout, finishing, fixtures, and furniture and decorations of the property.
- step S 404 communication interface 302 may receive camera position information.
- camera 220 may be placed in multiple positions in the property to take images and videos.
- point position determination unit 340 may determine point positions in the three-dimensional model corresponding to the camera positions in the property.
- the point positions are determined with a coordinate transformation of the camera positions.
- coordinates of the camera positions may be transformed from the property coordinate system to the coordinate system of the three-dimensional model.
- point position determination unit 340 may be further configured to determine connectivity among the point positions.
- two point positions are considered “connectable” if the connecting line between them does not intercept any obstacles in the three-dimensional model.
- obstacles may include framing structures (e.g., walls and counters), furniture (e.g., table and chairs, sofas, beds), or other objects (e.g., plants).
- the connectivity may be recorded in a two-dimensional table. For example, connectivity of two point positions may be recorded in a cell corresponding to the two point positions.
- connectivity can be recorded as a binary number, with “1” corresponding to “connectable” and “0” corresponding to “non-connectable.”
- FIG. 5 is a flowchart of an exemplary method 500 for determining connectivity among point positions, according to embodiments of the disclosure.
- Method 500 may include steps S 502 -S 512 as described below. It is to be appreciated that some of the steps may be optional to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown in FIG. 5 .
- the connectivity between two points positions may be determined based on the layout of the property. For example, the layout may specify the dividing structures, such as walls, counters, and doors. In some embodiments, the connectivity may be additionally determined based on other obstacles in the property, such as furniture, plants, and decorative pieces.
- point position determination unit 340 may connect a pair of point positions with a connecting line.
- point position determination unit 340 may determine if the connecting line intersects any obstacle in the three-dimensional model. If no, the pair of point positions is determined to be connectable in step S 506 . Otherwise, the pair of point positions is determined to be non-connectable in step S 508 . For example, point positions 113 and 114 are connectable through a door. However, introduction positions 112 and 115 are not connectable because line connecting the two positions intersects with the wall.
- step S 510 point position determination unit 340 may determine if all pairs of point positions are considered. If so, method 500 concludes and method 400 may proceed to step S 410 . Otherwise, method 500 will return to step S 502 to consider the next pair of point positions, and determine the connectivity using steps S 504 -S 508 .
- candidate route determination unit 342 may determine candidate routes connecting a first introduction position and a second introduction position based on the connectivity.
- the first introduction position may be introduction position 112 in great room 110
- the second introduction position may be introduction position 115 in bedroom 120 .
- Each candidate route links point positions that are connectable.
- multiple candidate routes between the first and second introduction positions may be available.
- FIG. 6 illustrates exemplary candidate routes between introduction positions 601 and 602 , according to embodiments of the disclosure.
- the first candidate route links point positions 611 , 612 , and 613 .
- the second candidate route links point positions 621 and 622 .
- the third candidate route links point positions 631 and 632 .
- step S 412 route selection unit 344 selects a route from the candidate routes based on some predetermined criteria.
- the predetermined criteria may be that the number of point positions on the route is the smallest, and/or the length of the route is the shortest.
- FIG. 7 is a flowchart of an exemplary method 700 for implementing step S 412 , according to embodiments of the disclosure. Method 700 may include steps S 702 -S 708 as described below. It is to be appreciated that some of the steps may be optional to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown in FIG. 7 .
- route selection unit 344 may determine the number of point positions on each candidate route. For example, as shown in FIG. 6 , the first candidate route has three point positions between the first and second introduction positions. Both the second candidate route and the third candidate route have two point positions between the first and second introduction positions.
- route selection unit 344 may identify a subset of candidate routes with less than threshold number of point positions. For example, when the threshold number is 3, route selection unit 344 will identify the second and third candidate routes shown in FIG. 6 . If after the thresholding, there is only one candidate route remaining, method 700 may conclude without implementing steps S 706 and S 708 , and return the one candidate route as the selected route.
- step S 706 route selection unit 344 calculates the length of each candidate route in the subset.
- route selection unit 344 identifies the route with the shortest length. For example, if L ⁇ L′, route selection unit 344 will identify the second candidate route as the best route.
- the two criteria may be applied in a flipped order, e.g., in step S 704 , the length of each candidate route may be compared with a threshold and those with lengths shorter than the threshold may be identified, and in step S 706 , the route with the smallest number of point positions may be selected from the remaining candidate routes.
- Other predetermined criteria may also be used to select the route. For example, in order to provide more seamless views when the user is being navigated to the second introduction position, route selection unit 344 may instead select the candidate route linking the largest number of point position. The selected route may be provided as the result of step S 412 , and method 700 concludes.
- view rendering unit 346 may detect that showing at the first introduction position ends or is about to end, which indicates that the user will soon move to the next room.
- showing of the property at the introduction positions may including displaying views rendered from images/videos captured at the camera positions corresponding the introduction positions.
- showing of the property at the introduction positions may be further coupled with playing audio content to the user to introduce the room features.
- the audio content may introduce features such as the size of the room, the function of the room, as well as other information such as when the room was renovated and the furniture and decorations in the room.
- the audio content may be pre-recorded so that view rendering unit 346 knows when the replay is towards the end.
- an indication that the user would like to view another room may be provided as an input by the user, e.g., by pressing buttons on the VR tool.
- view rendering unit 346 may render views corresponding to a point position on the selected route and display the views to the user.
- the views may be rendered based on images/videos captured at the point position.
- the views are rendered to be oriented in the direction the user is facing as he walks towards the next point position. For example, the user may be facing the direction pointing from introduction position 112 to point position 113 , and thus the views at point position 113 will be rendered in that direction when shown to the user.
- view rendering unit 346 may sequentially render and display views for all the point positions between the two introduction positions.
- step S 418 view rendering unit 346 determines if views of all point positions on the selected route have been rendered and displayed. If so, method 400 proceeds to step S 420 . Otherwise, method 700 returns to step S 416 to display views corresponding to the next point position on the selected route. Method 700 repeats steps S 416 and S 418 until all the views are displayed the user is navigated to the second introduction position.
- step S 420 system 300 starts showing features of the next room at the second introduction position. For example, after views at point positions 113 and 114 are displayed, user 210 arrives at introduction position 115 in bedroom 120 . The user will be shown views of bedroom 120 at introduction position 115 , and audio content may be optionally played to the user.
- the disclosed systems and methods make the VR tour more closely mimic a real house tour, thus improving user experience.
- the computer-readable medium may include volatile or non-volatile, magnetic, semiconductor, tape, optical, removable, non-removable, or other types of computer-readable medium or computer-readable storage devices.
- the computer-readable medium may be the storage device or the memory module having the computer instructions stored thereon, as disclosed.
- the computer-readable medium may be a disc or a flash drive having the computer instructions stored thereon.
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Tourism & Hospitality (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Human Resources & Organizations (AREA)
- Economics (AREA)
- Strategic Management (AREA)
- Software Systems (AREA)
- Marketing (AREA)
- General Business, Economics & Management (AREA)
- Health & Medical Sciences (AREA)
- Primary Health Care (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Remote Sensing (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- Radar, Positioning & Navigation (AREA)
- Development Economics (AREA)
- Game Theory and Decision Science (AREA)
- Entrepreneurship & Innovation (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Processing Or Creating Images (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
Description
- This application is a continuation of U.S. application Ser. No. 16/421,472, filed on May 24, 2019, which claims the benefits of priority to Chinese Application No. CN201810540326.1 filed on May 30, 2018, the content of which is hereby incorporated by reference in its entirety.
- The present disclosure relates to systems and methods for conducting a property tour using virtual reality technology, and more particularly, to systems and methods for navigating a user in a three-dimensional model of a property through a selected route linking a plurality of point positions.
- In the real estate market, when a person wants to buy or rent a property, he typically would ask to tour the property first in order to visually experience the layout and furnishing. House tours are often conducted in person. e.g., by the property owner or a real estate agent. During the tour, the visitor gets to see the property and the property owner/agent accompanying the tour will introduce features of the property to the visitor. For example, when the visitor is in the kitchen, features such as size of the room, cabinet and counter space, installed appliances, and lighting conditions may be introduced orally. While property tours are informative to the potential buyer/renter, it is also time consuming and inefficient.
- Recently, Virtual Reality (VR) technology has been applied to conduct these tours virtually. Videos, images, and point cloud data of the property are taken in advance. A three-dimensional model is rendered based on the point cloud data and/or the images. As a result, a visitor can remotely view the property without having to do it in person. On the other hand, the property owner/agent may log into the system to introduce the property features in real-time. VR tours thus save both visitor and property owner/agent's time by recreating the in-person experience virtually online.
- However, challenges remain in VR tours. In particular, a property typically includes multiple rooms or functional spaces, such as kitchen, living room, dining room, breakfast nook, study, bedrooms, and bathrooms. When the visitor finishes touring the current room and would like to view the next room, conventional VR touring technologies typically transitions the views directly from one room to another room, for example by suddenly switching the images. Such a transition contradicts the user experience in real life, where the visitor would have been ushered into the next room. For example, as he walks through the hallway and doors connecting the rooms, he would have continued to experience the layout of the property, as well as additional property views along the way. All these become lost due to the abrupt transition by the conventional VR touring technologies.
- Embodiments of the disclosure improve the existing VR touring systems and methods by navigating a user in a three-dimensional model of a property through a selected route linking a plurality of point positions.
- In one aspect, a system for navigating a user in a three-dimensional model of a property is disclosed. The system includes a storage device configured to store data associated with a plurality of point positions. Each point position corresponds to a camera position at which images and point cloud data of the property are captured by the camera. The system further includes at least one processor configured to determine connectivity among the point positions and determine a plurality of candidate routes connecting a first introduction position and a second introduction position. Each candidate route links a subset of the point positions based on the connectivity. The at least one processor is further configured to select a route from the candidate routes and sequentially display views corresponding to the point positions on the selected route. The views at each point position are rendered from the images captured at the camera position corresponding to the point position.
- In another aspect, a computer-implemented method for navigating a user in a three-dimensional model of a property is disclosed. The method includes receiving data associated with a plurality of point positions. Each point position corresponds to a camera position at which images of the property are captured by the camera. The method further includes determining, by at least one processor, connectivity among the point positions and determining, by the at least one processor, a plurality of candidate routes connecting a first introduction position and a second introduction position. Each candidate route links a subset of the point positions based on the connectivity. The method also includes selecting, by the at least one processor, a route from the candidate routes and sequentially displaying, on a display, views corresponding to the point positions on the selected route. The views at each point position are rendered by the at least one processor from the images captured at the camera position corresponding to the point position.
- In yet another aspect, a non-transitory computer-readable medium having stored thereon computer instructions is disclosed. The computer instructions, when executed by at least one processor, perform a method for navigating a user in a three-dimensional model of a property. The method includes receiving data associated with a plurality of point positions. Each point position corresponds to a camera position at which images of the property are captured by the camera. The method further includes determining connectivity among the point positions and determining a plurality of candidate routes connecting a first introduction position and a second introduction position. Each candidate route links a subset of the point positions based on the connectivity. The method also includes selecting a route from the candidate routes and sequentially displaying views corresponding to the point positions on the selected route. The views at each point position are rendered from the images captured at the camera position corresponding to the point position.
- It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
-
FIG. 1 illustrates a schematic diagram of an exemplary three-dimensional model including multiple rooms, according to embodiments of the disclosure. -
FIG. 2 illustrates a schematic diagram of an exemplary room in a three-dimensional model, according to embodiments of the disclosure. -
FIG. 3 is a block diagram of an exemplary system for navigating a user in a three-dimensional model, according to embodiments of the disclosure. -
FIG. 4 is a flowchart of an exemplary method for navigating a user in a three-dimensional model, according to embodiments of the disclosure. -
FIG. 5 is a flowchart of an exemplary method for determining connectivity among point positions, according to embodiments of the disclosure. -
FIG. 6 illustrates exemplary candidate routes, according to embodiments of the disclosure. -
FIG. 7 is a flowchart of an exemplary method for selecting a route from candidate routes, according to embodiments of the disclosure. - Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts.
-
FIG. 1 illustrates a schematic diagram of an exemplary three-dimensional model 100 including multiple rooms, according to embodiments of the disclosure. In some embodiments, three-dimensional model 100 may be a model of a property, such as a house, an apartment, a townhouse, a garage, a warehouse, an office building, a hotel, and a store, etc. As shown inFIG. 1 , three-dimensional model 100 virtually recreates the real-world property including its layout (e.g., the framing structures that divide the property into several rooms such as walls and counters), finishing (e.g., kitchen/bathroom cabinets, bathtub, island, etc.), fixtures installed (e.g., appliances, window treatments, chandeliers, etc.), and furniture and decorations (e.g., beds, desks, tables and chairs, sofas, TV stands, bookshelves, wall paintings, mirrors, plants, etc.) - Three-
dimensional model 100 may be divided into multiple rooms or functional spaces by interior walls. Some rooms may have multiple functions. For example, three-dimensional model 100 may include agreat room 110 that has combined functions of a living room and a kitchen. Three-dimensional model 100 may further include twobedrooms FIG. 2 illustrates a schematic diagram of an exemplarygreat room 110 in three-dimensional model 100, according to embodiments of the disclosure. The rooms may be connected by hallways and doors. - Each room of three-
dimensional model 100 includes at least one introduction positions. For example,great room 110 has anintroduction position 112 andbedroom 120 has anintroduction position 115. Consistent with the present disclosure, a “introduction position” is a position in the three-dimensional model that corresponds to a physical position that the user would stand in the room when listening to introductions by the property owner/agent. In other words, the user of the VR tool views the rooms from the introduction positions. In some embodiments, audio content may be displayed to the user at the introduction positions to orally introduce the features of the respective rooms. The introduction may be provided in real-time by property owner/agent, or pre-recorded and replayed when the user is at the introduction position. - Several point positions may be located within each room of three-
dimensional model 100. For example, point positions 111 and 113 are located ingreat room 110, andpoint position 114 is located inbedroom 120. Consistent with the present disclosure, a “point position” is a position in the three-dimensional model that corresponds to a camera position where images and videos of the property are captured. In some embodiments, introduction positions 112 and 115 may also be point positions. - As shown in
FIG. 2 , auser 210 may be placed atintroduction position 112 ingreat room 110 for the room features to be introduced to him.User 210 may view images or videos ofgreat room 110 and listen to oral introductions atintroduction position 112. Acamera 220 may be placed atpoint position 113 and/or 112 to capture images and/or videos ofgreat room 110. In some embodiments,camera 220 may be a digital camera, a binocular camera, a video camera, a panoramic camera, etc. In some embodiments,camera 220 may be configured to capture three-dimensional information, (e.g., including depth information) of the room.Camera 220 may be mounted on a tripod or be carried by an operator.Camera 220 may be rotated or otherwise adjusted to capture images or videos from different angles and heights. - In some embodiments, the VR tool may transition from one room to another room of three-dimensional model, during the VR house tour. For example, when
user 210 finishes touringgreat room 110, he may want to move on tobedroom 120. Consistent with the present disclosure, systems and methods for navigatinguser 210 in three-dimensional model 100 of a property (e.g., fromgreat room 110 to bedroom 120) is provided. Instead of transitioning from views atintroduction position 112 in the first room directly to views atintroduction position 115 in the second room, the disclosed systems and methods navigateuser 210 fromintroduction position 112 tointroduction position 115 along a route that passes one or more point positions, e.g., point positions 113 and 114. In some embodiments, the disclosed systems and methods selects the route from multiple candidate routes. The candidate routes are different paths that the user can follow to get fromintroduction position 112 ingreat room 110 tointroduction position 115 inbedroom 120. Each candidate passes different sets of point positions. - In some embodiments, the disclosed systems and methods first determine connectivity among the relevant point positions and then establish the candidate routes connecting the introduction positions 112 and 115 by linking a set of the point positions that are determined to be connectable. The disclosed systems and methods then select the best route from the candidate routes based on certain criteria. Views corresponding to the point positions on the selected route may be sequentially displayed to the user to provide a more real experience as the user walks from
great room 110 tointroduction position 115 through the route. In some embodiments, the views at each point position are rendered from the images previously captured at the camera position corresponding to the point position. In some embodiments, the views are oriented in the direction the user is facing as he walks towards each point position. For example, the user may be facing the direction pointing fromintroduction position 112 to pointposition 113, and thus the views atpoint position 113 will be rendered in that direction when shown to the user. - Although the exemplary three-dimensional model shown in
FIG. 1 illustrates only indoor space and structures, it is contemplated that the model could also include outdoor space and structures, such as the deck, the front/back yard, the garage, as well as the neighboring environment. A user may be navigated from an introduction position indoor to an introduction outdoor or from an introduction position outdoor to an introduction indoor door through, e.g., doors. A person of ordinary skill can adapt the disclosed systems and methods without undue experimentation for outdoor showings. -
FIG. 3 is a block diagram of anexemplary system 300 for navigating a user in a three-dimensional model, according to embodiments of the disclosure. In some embodiments,system 300 may be implemented by a physical server or a service in the cloud. In some other embodiments,system 300 may be implemented a computer or a consumer electronic device such as a mobile phone, a pad, or a wearable device. As shown inFIG. 3 ,system 300 may include acommunication interface 302, aprocessor 304, amemory 306, astorage 308, and a bus 310. In some embodiments,system 300 may have different modules in a single device, such as an integrated circuit (IC) chip (implemented as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA)), or separate devices with dedicated functions. Components ofsystem 300 may be in an integrated device, or distributed at different locations but communicate with each other through a network (not shown). The various components ofsystem 300 may be connected to and communicate with each other through bus 310. -
Communication interface 302 may send data to and receive data from components such ascamera 220 anddatabase 301 via direct communication links, a Wireless Local Area Network (WLAN), a Wide Area Network (WAN), wireless communication networks using radio waves, a cellular network, and/or a local wireless network (e.g., Bluetooth™ or WiFi), or other communication methods. In some embodiments,communication interface 302 can be an integrated services digital network (ISDN) card, cable modem, satellite modem, or a modem to provide a data communication connection. As another example,communication interface 302 can be a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links can also be implemented bycommunication interface 302. In such an implementation,communication interface 302 can send and receive electrical, electromagnetic or optical signals that carry digital data streams representing various types of information via a network. - Consistent with some embodiments,
communication interface 302 may receive a three-dimensional model of a property. The model may be constructed based on point cloud data as well as images captured of the property. In some embodiments, the three-dimensional model may be rendered in real-time of the VR tour or offline prior to the tour. Consistent with the disclosure,communication interface 302 may receive data of camera positions in the property, and images/videos of the property taken at those camera positions. The three-dimensional model, the camera positions, and the images/videos may be stored indatabase 301.Communication interface 302 may provide the received information or data tomemory 306 and/orstorage 308 for storage or toprocessor 304 for processing. -
Processor 304 may include any appropriate type of general-purpose or special-purpose microprocessor, digital signal processor, or microcontroller.Processor 304 may be configured as a separate processor module dedicated to providing VR house tours. Alternatively,processor 404 may be configured as a shared processor module for performing other functions related to or unrelated to VR house tours. For example, VR house tour is just one application installed on a versatile device. - As shown in
FIG. 3 ,processor 304 may include multiple modules, such as a point position determination unit 340, a candidateroute determination unit 342, aroute selection unit 344, and aview rendering unit 346, and the like. These modules (and any corresponding sub-modules or sub-units) can be hardware units (e.g., portions of an integrated circuit) ofprocessor 304 designed for use with other components or to execute part of a program. The program may be stored on a computer-readable medium (e.g.,memory 306 and/or storage 308), and when executed byprocessor 304, it may perform one or more functions. AlthoughFIG. 3 shows units 340-346 all within oneprocessor 304, it is contemplated that these units may be distributed among multiple processors located near or remotely with each other. -
Memory 306 andstorage 308 may include any appropriate type of mass storage provided to store any type of information thatprocessor 304 may need to operate.Memory 306 andstorage 308 may be a volatile or non-volatile, magnetic, semiconductor, tape, optical, removable, non-removable, or other type of storage device or tangible (i.e., non-transitory) computer-readable medium including, but not limited to, a ROM, a flash memory, a dynamic RAM, and a static RAM.Memory 306 and/orstorage 308 may be configured to store one or more computer programs that may be executed byprocessor 304 to perform user navigation functions in a three-dimensional model disclosed herein. For example,memory 306 and/orstorage 308 may be configured to store program(s) that may be executed byprocessor 304 to provide a VR tour of a property to a user by introducing property features to the user at introduction positions in a three-dimensional model and navigating the user in the model among the introduction positions. -
Memory 306 and/orstorage 308 may be further configured to store information and data used byprocessor 304. For instance,memory 306 and/orstorage 308 may be configured to store various camera positions and images/videos of a property captured bycamera 220 at the camera positions.Memory 306 and/orstorage 308 may also be configured to store intermediate data generated byprocessor 304, such as point positions in a three-dimensional model corresponding to the camera positions, connectivity of the point positions, candidate routes connection two introduction positions, and rendered views at point positions. The various types of data may be stored permanently, removed periodically, or disregarded immediately after each frame of data is processed. - In some embodiments,
system 300 may optionally include adisplay 303. In some embodiments,display 303 may be external tosystem 300 but be connected withsystem 300.Display 303 may display the views in the three-dimensional model to the users. In some embodiments,display 303 may further function as a user interface to receive user input.Display 303 may include a Liquid Crystal Display (LCD), a Light Emitting Diode Display (LED), a plasma display, or any other type of display, and provide a Graphical User Interface (GUI) presented on the display for user input and data display. The display may include a number of different types of materials, such as plastic or glass, and may be touch-sensitive to receive commands from the user. For example, the display may include a touch-sensitive material that is substantially rigid, such as Gorilla Glass™, or substantially pliable, such as Willow Glass™. - The modules of
processor 304 may be configured to execute program(s) stored inmemory 306/storage 308 to perform a method for providing a VR tour to a user. For example,FIG. 4 is a flowchart of an exemplary method 400 for navigating a user in a three-dimensional model, according to embodiments of the disclosure. Method 400 may include steps S402-S420 as described below. It is to be appreciated that some of the steps may be optional to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown inFIG. 4 . For description purpose, method 400 will be described as to navigate a user fromgreat room 110 tobedroom 120 of three-dimensional model 100 (as shown inFIG. 1 ). Method 400, however, can be implemented for navigating between other rooms of the model, between indoor and outdoor spaces, or for different models for the properties. - In some embodiments, method 400 may be triggered when
system 300 receives an instruction to move the user from the current room to the next room. In some embodiments,system 300 may automatically initiate method 400 when it detects that the introduction of the current room is about to finish. - In step S402,
communication interface 302 may receive a three-dimensional model of a property, such as three-dimensional model 100. In some embodiments, the three-dimensional model may be a model of a property, such as a house, an apartment, a townhouse, a garage, a warehouse, an office building, a hotel, and a store, etc. In some embodiments, the three-dimensional model may be constructed based on point cloud data of the property as well as image/videos captured of the property. The three-dimensional model may virtually recreate the property including its layout, finishing, fixtures, and furniture and decorations of the property. - In step S404,
communication interface 302 may receive camera position information. For example,camera 220 may be placed in multiple positions in the property to take images and videos. In step S406, point position determination unit 340 may determine point positions in the three-dimensional model corresponding to the camera positions in the property. In some embodiments, the point positions are determined with a coordinate transformation of the camera positions. For example, coordinates of the camera positions may be transformed from the property coordinate system to the coordinate system of the three-dimensional model. - In step S406, point position determination unit 340 may be further configured to determine connectivity among the point positions. In some embodiments, two point positions are considered “connectable” if the connecting line between them does not intercept any obstacles in the three-dimensional model. In some embodiments, obstacles may include framing structures (e.g., walls and counters), furniture (e.g., table and chairs, sofas, beds), or other objects (e.g., plants). In some embodiments, the connectivity may be recorded in a two-dimensional table. For example, connectivity of two point positions may be recorded in a cell corresponding to the two point positions. In some embodiments, connectivity can be recorded as a binary number, with “1” corresponding to “connectable” and “0” corresponding to “non-connectable.”
-
FIG. 5 is a flowchart of anexemplary method 500 for determining connectivity among point positions, according to embodiments of the disclosure.Method 500 may include steps S502-S512 as described below. It is to be appreciated that some of the steps may be optional to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown inFIG. 5 . In some embodiments, the connectivity between two points positions may be determined based on the layout of the property. For example, the layout may specify the dividing structures, such as walls, counters, and doors. In some embodiments, the connectivity may be additionally determined based on other obstacles in the property, such as furniture, plants, and decorative pieces. - In step S502, point position determination unit 340 may connect a pair of point positions with a connecting line. In step $504, point position determination unit 340 may determine if the connecting line intersects any obstacle in the three-dimensional model. If no, the pair of point positions is determined to be connectable in step S506. Otherwise, the pair of point positions is determined to be non-connectable in step S508. For example, point positions 113 and 114 are connectable through a door. However, introduction positions 112 and 115 are not connectable because line connecting the two positions intersects with the wall.
- In step S510, point position determination unit 340 may determine if all pairs of point positions are considered. If so,
method 500 concludes and method 400 may proceed to step S410. Otherwise,method 500 will return to step S502 to consider the next pair of point positions, and determine the connectivity using steps S504-S508. - Returning to
FIG. 4 , in step S410, candidateroute determination unit 342 may determine candidate routes connecting a first introduction position and a second introduction position based on the connectivity. For example, the first introduction position may beintroduction position 112 ingreat room 110, and the second introduction position may beintroduction position 115 inbedroom 120. Each candidate route links point positions that are connectable. In some embodiments, multiple candidate routes between the first and second introduction positions may be available. For example,FIG. 6 illustrates exemplary candidate routes betweenintroduction positions positions positions positions - In step S412,
route selection unit 344 selects a route from the candidate routes based on some predetermined criteria. In some embodiments, the predetermined criteria may be that the number of point positions on the route is the smallest, and/or the length of the route is the shortest. For example,FIG. 7 is a flowchart of anexemplary method 700 for implementing step S412, according to embodiments of the disclosure.Method 700 may include steps S702-S708 as described below. It is to be appreciated that some of the steps may be optional to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown inFIG. 7 . - In step S702,
route selection unit 344 may determine the number of point positions on each candidate route. For example, as shown inFIG. 6 , the first candidate route has three point positions between the first and second introduction positions. Both the second candidate route and the third candidate route have two point positions between the first and second introduction positions. - In step S704,
route selection unit 344 may identify a subset of candidate routes with less than threshold number of point positions. For example, when the threshold number is 3,route selection unit 344 will identify the second and third candidate routes shown inFIG. 6 . If after the thresholding, there is only one candidate route remaining,method 700 may conclude without implementing steps S706 and S708, and return the one candidate route as the selected route. - If more than one candidate routes remain after step S704,
method 700 proceeds to step S706. In step S706,route selection unit 344 calculates the length of each candidate route in the subset. Consistent with the present disclosure, the “length” of a route is the accumulated distances between the point positions on the route. For example, inFIG. 6 , if the distance L1 betweenintroduction position 601 andpoint position 621, the distance L2 betweenpoint position 621 andpoint position 622, and the distance L3 between thepoint position 622 andintroduction position 602, the length of the second candidate route is the sum of these distances, i.e., L=L1+L2+L3. Similarly, the length of the third candidate route can be calculated as L′=L1′+L2′+L3′, where L1′, L2′, and L3′ are the distances between the consecutive introduction/point positions - In step S708,
route selection unit 344 identifies the route with the shortest length. For example, if L<L′,route selection unit 344 will identify the second candidate route as the best route. - It is contemplated that the two criteria may be applied in a flipped order, e.g., in step S704, the length of each candidate route may be compared with a threshold and those with lengths shorter than the threshold may be identified, and in step S706, the route with the smallest number of point positions may be selected from the remaining candidate routes. Other predetermined criteria may also be used to select the route. For example, in order to provide more seamless views when the user is being navigated to the second introduction position,
route selection unit 344 may instead select the candidate route linking the largest number of point position. The selected route may be provided as the result of step S412, andmethod 700 concludes. - Returning to
FIG. 4 , in step S414,view rendering unit 346 may detect that showing at the first introduction position ends or is about to end, which indicates that the user will soon move to the next room. In some embodiments, showing of the property at the introduction positions may including displaying views rendered from images/videos captured at the camera positions corresponding the introduction positions. In some embodiments, showing of the property at the introduction positions may be further coupled with playing audio content to the user to introduce the room features. For example, the audio content may introduce features such as the size of the room, the function of the room, as well as other information such as when the room was renovated and the furniture and decorations in the room. In some embodiments, the audio content may be pre-recorded so thatview rendering unit 346 knows when the replay is towards the end. In some embodiments, an indication that the user would like to view another room may be provided as an input by the user, e.g., by pressing buttons on the VR tool. - In step S416,
view rendering unit 346 may render views corresponding to a point position on the selected route and display the views to the user. In some embodiments, the views may be rendered based on images/videos captured at the point position. In some embodiments, the views are rendered to be oriented in the direction the user is facing as he walks towards the next point position. For example, the user may be facing the direction pointing fromintroduction position 112 to pointposition 113, and thus the views atpoint position 113 will be rendered in that direction when shown to the user. - In some embodiments,
view rendering unit 346 may sequentially render and display views for all the point positions between the two introduction positions. In step S418,view rendering unit 346 determines if views of all point positions on the selected route have been rendered and displayed. If so, method 400 proceeds to step S420. Otherwise,method 700 returns to step S416 to display views corresponding to the next point position on the selected route.Method 700 repeats steps S416 and S418 until all the views are displayed the user is navigated to the second introduction position. - In step S420,
system 300 starts showing features of the next room at the second introduction position. For example, after views atpoint positions user 210 arrives atintroduction position 115 inbedroom 120. The user will be shown views ofbedroom 120 atintroduction position 115, and audio content may be optionally played to the user. - By providing views at intermediate point positions between the two introduction positions, the disclosed systems and methods make the VR tour more closely mimic a real house tour, thus improving user experience.
- Another aspect of the disclosure is directed to a non-transitory computer-readable medium storing instruction which, when executed, cause one or more processors to perform the methods, as discussed above. The computer-readable medium may include volatile or non-volatile, magnetic, semiconductor, tape, optical, removable, non-removable, or other types of computer-readable medium or computer-readable storage devices. For example, the computer-readable medium may be the storage device or the memory module having the computer instructions stored thereon, as disclosed. In some embodiments, the computer-readable medium may be a disc or a flash drive having the computer instructions stored thereon.
- It will be apparent to those skilled in the art that various modifications and variations can be made to the disclosed system and related methods. Other embodiments will be apparent to those skilled in the art from consideration of the specification and practice of the disclosed system and related methods.
- It is intended that the specification and examples be considered as exemplary only, with a true scope being indicated by the following claims and their equivalents.
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/581,741 US10553027B1 (en) | 2018-05-30 | 2019-09-24 | Systems and methods for navigating a user in a virtual reality model |
Applications Claiming Priority (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810540326 | 2018-05-30 | ||
CN201810540326.1A CN108898516B (en) | 2018-05-30 | 2018-05-30 | Method, server and terminal for entering between functions in virtual three-dimensional room speaking mode |
CNCN201810540326.1 | 2018-05-30 | ||
US16/421,472 US10467810B1 (en) | 2018-05-30 | 2019-05-24 | Systems and methods for navigating a user in a virtual reality model |
US16/581,741 US10553027B1 (en) | 2018-05-30 | 2019-09-24 | Systems and methods for navigating a user in a virtual reality model |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/421,472 Continuation US10467810B1 (en) | 2018-05-30 | 2019-05-24 | Systems and methods for navigating a user in a virtual reality model |
Publications (2)
Publication Number | Publication Date |
---|---|
US20200020159A1 true US20200020159A1 (en) | 2020-01-16 |
US10553027B1 US10553027B1 (en) | 2020-02-04 |
Family
ID=64343697
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/421,472 Active US10467810B1 (en) | 2018-05-30 | 2019-05-24 | Systems and methods for navigating a user in a virtual reality model |
US16/581,741 Active US10553027B1 (en) | 2018-05-30 | 2019-09-24 | Systems and methods for navigating a user in a virtual reality model |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/421,472 Active US10467810B1 (en) | 2018-05-30 | 2019-05-24 | Systems and methods for navigating a user in a virtual reality model |
Country Status (5)
Country | Link |
---|---|
US (2) | US10467810B1 (en) |
EP (1) | EP3662436A4 (en) |
JP (1) | JP6987998B2 (en) |
CN (1) | CN108898516B (en) |
WO (1) | WO2019228452A1 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11496824B2 (en) | 2019-04-30 | 2022-11-08 | Shenzhen Shokz Co., Ltd. | Acoustic output apparatus with drivers in multiple frequency ranges and bluetooth low energy receiver |
US12014433B1 (en) * | 2018-10-09 | 2024-06-18 | Corelogic Solutions, Llc | Generation and display of interactive 3D real estate models |
Families Citing this family (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108898516B (en) * | 2018-05-30 | 2020-06-16 | 贝壳找房(北京)科技有限公司 | Method, server and terminal for entering between functions in virtual three-dimensional room speaking mode |
CN110111427A (en) * | 2019-04-18 | 2019-08-09 | 贝壳技术有限公司 | Migration route automatic generation method and device in a kind of house virtual three-dimensional space |
CN111127627B (en) * | 2019-11-20 | 2020-10-27 | 贝壳找房(北京)科技有限公司 | Model display method and device in three-dimensional house model |
CN111145293B (en) * | 2019-12-18 | 2020-12-29 | 北京城市网邻信息技术有限公司 | Method and device for reconstructing outline of house-type graph and electronic equipment |
CN111460334B (en) * | 2020-04-03 | 2024-04-26 | 北京字节跳动网络技术有限公司 | Information display method and device and electronic equipment |
US10845681B1 (en) | 2020-08-17 | 2020-11-24 | Stephen Michael Buice | Camera apparatus for hiding a camera operator while capturing 360-degree images or video footage |
CN112068754B (en) * | 2020-08-28 | 2021-10-15 | 北京五八信息技术有限公司 | House resource display method and device |
CN112907755B (en) * | 2021-01-22 | 2022-04-15 | 贝壳找房(北京)科技有限公司 | Model display method and device in three-dimensional house model |
US11741618B2 (en) * | 2021-03-22 | 2023-08-29 | Everypoint, Inc. | Performing object modeling by combining visual data from images with motion data of the image acquisition device |
CN113129110B (en) * | 2021-05-08 | 2023-11-03 | 深圳新房网络科技有限公司 | Multi-sense VR house-seeing system based on virtual reality technology |
CN113963133B (en) * | 2021-09-18 | 2022-07-08 | 北京城市网邻信息技术有限公司 | Method and device for generating house source watching route, electronic equipment and readable medium |
Family Cites Families (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2001236396A (en) * | 2000-02-23 | 2001-08-31 | Wall:Kk | Mediation support system for real estate article |
JP2002064799A (en) * | 2000-08-22 | 2002-02-28 | Megafusion Corp | Image-providing system and image-providing method |
US20070198178A1 (en) | 2004-03-31 | 2007-08-23 | Trimby Martin W | Pathfinding system |
KR101673921B1 (en) | 2010-05-04 | 2016-11-08 | 삼성전자주식회사 | Apparatus and method for managing position information of mobile terminal |
EP2643821B1 (en) * | 2010-11-24 | 2019-05-08 | Google LLC | Path planning for street level navigation in a three-dimensional environment, and applications thereof |
CN104215237A (en) * | 2013-06-05 | 2014-12-17 | 北京掌尚无限信息技术有限公司 | Indoor map navigation route guidance algorithm |
CN103424113B (en) * | 2013-08-01 | 2014-12-31 | 毛蔚青 | Indoor positioning and navigating method of mobile terminal based on image recognition technology |
CN104598979B (en) * | 2013-10-31 | 2021-10-08 | Sap欧洲公司 | Time and location based delivery optimization |
CN103646419B (en) * | 2013-12-11 | 2017-10-10 | 上海赛图计算机科技有限公司 | The methods of exhibiting across floor path applied based on indoor map |
CN105333882B (en) * | 2014-08-07 | 2019-03-15 | 北京四维图新科技股份有限公司 | Route guiding planing method, device and terminal in a kind of building |
JP6304771B2 (en) * | 2015-08-20 | 2018-04-04 | 株式会社サイバーウォーカー | Route generation program, route generation method and route generation device |
US9824500B2 (en) * | 2016-03-16 | 2017-11-21 | Microsoft Technology Licensing, Llc | Virtual object pathing |
CN107402008A (en) | 2016-05-19 | 2017-11-28 | 阿里巴巴集团控股有限公司 | The method, apparatus and system of indoor navigation |
CN105910605A (en) * | 2016-05-30 | 2016-08-31 | 中国科学技术大学苏州研究院 | Indoor navigation dynamic route generation method |
CN105898712B (en) | 2016-06-15 | 2019-04-19 | 西北工业大学 | Indoor three-dimensional localization stepped approach under more floor environment |
US10030979B2 (en) * | 2016-07-29 | 2018-07-24 | Matterport, Inc. | Determining and/or generating a navigation path through a captured three-dimensional model rendered on a device |
CN106679668B (en) * | 2016-12-30 | 2018-08-03 | 百度在线网络技术(北京)有限公司 | Air navigation aid and device |
US10530997B2 (en) * | 2017-07-13 | 2020-01-07 | Zillow Group, Inc. | Connecting and using building interior data acquired from mobile devices |
CN108898516B (en) * | 2018-05-30 | 2020-06-16 | 贝壳找房(北京)科技有限公司 | Method, server and terminal for entering between functions in virtual three-dimensional room speaking mode |
-
2018
- 2018-05-30 CN CN201810540326.1A patent/CN108898516B/en active Active
-
2019
- 2019-05-24 US US16/421,472 patent/US10467810B1/en active Active
- 2019-05-30 WO PCT/CN2019/089258 patent/WO2019228452A1/en unknown
- 2019-05-30 EP EP19811203.9A patent/EP3662436A4/en active Pending
- 2019-05-30 JP JP2020536748A patent/JP6987998B2/en active Active
- 2019-09-24 US US16/581,741 patent/US10553027B1/en active Active
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US12014433B1 (en) * | 2018-10-09 | 2024-06-18 | Corelogic Solutions, Llc | Generation and display of interactive 3D real estate models |
US11496824B2 (en) | 2019-04-30 | 2022-11-08 | Shenzhen Shokz Co., Ltd. | Acoustic output apparatus with drivers in multiple frequency ranges and bluetooth low energy receiver |
Also Published As
Publication number | Publication date |
---|---|
US10553027B1 (en) | 2020-02-04 |
CN108898516B (en) | 2020-06-16 |
WO2019228452A1 (en) | 2019-12-05 |
EP3662436A4 (en) | 2021-09-08 |
EP3662436A1 (en) | 2020-06-10 |
CN108898516A (en) | 2018-11-27 |
US10467810B1 (en) | 2019-11-05 |
JP2021507423A (en) | 2021-02-22 |
JP6987998B2 (en) | 2022-01-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10553027B1 (en) | Systems and methods for navigating a user in a virtual reality model | |
US11494973B2 (en) | Generating floor maps for buildings from automated analysis of visual data of the buildings' interiors | |
US11935196B2 (en) | Presenting building information using building models | |
CA3139429C (en) | Automated determination of image acquisition locations in building interiors using determined room shapes | |
WO2021098582A1 (en) | System and method for displaying virtual reality model | |
AU2021290378B2 (en) | Automated determination of image acquisition locations in building interiors using multiple data capture devices | |
US11790648B2 (en) | Automated usability assessment of buildings using visual data of captured in-room images | |
AU2022200299B2 (en) | Automated direction of capturing in-room information for use in usability assessment of buildings | |
US20230196684A1 (en) | Presenting Building Information Using Video And Building Models | |
CA3187671A1 (en) | Presenting building information using building models | |
US20230063465A1 (en) | Systems and methods for constructing 3d model based on changing viewpiont | |
EP4358026A1 (en) | Automated determination of acquisition locations of acquired building images based on identified surrounding objects | |
CN117745983A (en) | Automated generation of visual data enhancements and presentation of visual data enhancements on camera view images captured in buildings | |
CN117615223A (en) | Automated generation and use of building video with accompanying narratives by analyzing acquired images and other building information |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: KE.COM (BEIJING) TECHNOLOGY CO., LTD, CHINA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:YANG, YUKE;REEL/FRAME:050479/0266 Effective date: 20190506 |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
AS | Assignment |
Owner name: REALSEE (BEIJING) TECHNOLOGY CO., LTD., CHINA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:KE.COM (BEIJING) TECHNOLOGY CO., LTD.;REEL/FRAME:059344/0106 Effective date: 20220318 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |