WO2023215976A1 - System and method for using virtual/augmented reality to design living space features - Google Patents

System and method for using virtual/augmented reality to design living space features Download PDF

Info

Publication number
WO2023215976A1
WO2023215976A1 PCT/CA2023/050629 CA2023050629W WO2023215976A1 WO 2023215976 A1 WO2023215976 A1 WO 2023215976A1 CA 2023050629 W CA2023050629 W CA 2023050629W WO 2023215976 A1 WO2023215976 A1 WO 2023215976A1
Authority
WO
WIPO (PCT)
Prior art keywords
floor
room
default
design
view
Prior art date
Application number
PCT/CA2023/050629
Other languages
French (fr)
Inventor
Sébastien Gagnon
Olivier D. LEBROHON
Original Assignee
Gagnon Sebastien
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Gagnon Sebastien filed Critical Gagnon Sebastien
Publication of WO2023215976A1 publication Critical patent/WO2023215976A1/en

Links

Classifications

    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09FDISPLAYING; ADVERTISING; SIGNS; LABELS OR NAME-PLATES; SEALS
    • G09F5/00Means for displaying samples
    • G09F5/04Cards of samples; Books of samples
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00Computer-aided design [CAD]
    • G06F30/10Geometric CAD
    • G06F30/13Architectural design, e.g. computer-aided architectural design [CAAD] related to design of buildings, bridges, landscapes, production plants or roads
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/06Buying, selling or leasing transactions
    • G06Q30/0601Electronic shopping [e-shopping]
    • G06Q30/0603Catalogue ordering
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/06Buying, selling or leasing transactions
    • G06Q30/0601Electronic shopping [e-shopping]
    • G06Q30/0621Item configuration or customization
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/06Buying, selling or leasing transactions
    • G06Q30/0601Electronic shopping [e-shopping]
    • G06Q30/0623Item investigation
    • G06Q30/0625Directed, with specific intent or strategy
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/06Buying, selling or leasing transactions
    • G06Q30/0601Electronic shopping [e-shopping]
    • G06Q30/0641Shopping interfaces
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/06Buying, selling or leasing transactions
    • G06Q30/0601Electronic shopping [e-shopping]
    • G06Q30/0641Shopping interfaces
    • G06Q30/0643Graphical representation of items or shoppers
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09FDISPLAYING; ADVERTISING; SIGNS; LABELS OR NAME-PLATES; SEALS
    • G09F5/00Means for displaying samples
    • G09F5/04Cards of samples; Books of samples
    • G09F2005/045Cards of samples; Books of samples for carpets

Definitions

  • the present generally concerns augmented reality applications and more particularly to use of augmented reality for designing home layouts.
  • Interior home design generally begins by physically measuring room dimensions. To install items such as carpets, flooring and the like, room measurements are taken, which if not carried out accurately can prove costly to correct. Selecting a desired carpet or flooring design generally involves using small samples. The designer must then use their imagination to determine what the end result will be. Clearly, this is haphazard and fraught with problems. Also, using small carpet or flooring samples to predict what the end result will be on a large scale can be the source of disappointment.
  • VR and AR are gaining popularity for use in a growing number of activities such as gaming and other entertainment purposes.
  • VR and AR has become increasingly popular for use in non- recreational simulations, examples of which are surgery, engineering projects and the like.
  • VR and AR use headsets to integrate visual information into a user's field of view to enhance their surroundings or allow them to step into immersive three- dimensional (3D) virtual environments.
  • 3D three- dimensional
  • the application is designed to scan a floor and then select a replacement floor.
  • the application uses a computing device to render a three- dimensional view of a room in a dwelling in virtual reality, and from the data collected, aid a designer in rendering an augmented three-dimensional virtual reality view of the room together with the selected new feature.
  • the reality of the floor selection and purchase process presents its share of challenges.
  • a method for modifying a room feature in virtual reality comprising: on a computing device having one or more cameras, collecting and processing two-dimensional floor measurement data for displaying, in a virtual reality space, a default room view having a default floor design; selecting a desired replacement floor design from a plurality of floor design image samples displayed in a catalog; and processing the selected desired replacement floor design so as to transition from the default room view to an augmented reality space view in which the default floor design is replaced by the selected replacement floor design.
  • the processing of the two-dimensional floor measurement data includes obtaining occlusion room data from the default room view and subtracting the occlusion room data from room obstacles to obtain the two-dimensional floor dimensions.
  • the processing of the two-dimensional floor measurement data includes providing accurate dimension measurements by locating markers in the default room view in the virtual reality space.
  • the method includes displaying room dimension data.
  • a user approves of the default room view and floor dimensions.
  • the catalog view is a virtual wheel having displayed therein the plurality of floor design image samples or the catalog view is an itemized list.
  • the user selects the desired floor sample, the desired floor sample being processed and displayed in the augmented reality space.
  • a cost display image is provided simultaneously with the selected floor sample.
  • the virtual wheel is moved in clockwise and counterclockwise directions.
  • the method further includes electronically contacting a contractor with the selected floor design and selecting a payment option.
  • a system for modifying a room feature in virtual reality comprising: a computing device having one or more cameras for collecting and processing two-dimensional floor measurement data for displaying, in a virtual reality space, a default room view having a default floor design, the two-dimensional floor measurement data being stored in memory; a plurality of floor design image samples displayed in a catalog, stored in memory, the catalog being operable to select a desired replacement floor design from; the selected desired replacement floor design being processed so as to transition from the default room view, stored memory, to an augmented reality space view in which the default floor design is replaced by the selected replacement floor design.
  • the processing of the two-dimensional floor measurement data includes obtaining occlusion room data from the default room view and subtracting the occlusion room data from room obstacles to obtain the two-dimensional floor dimensions.
  • the processing of the two-dimensional floor measurement data includes providing accurate dimension measurements by locating markers in the default room view in the virtual reality space.
  • the system includes displaying room dimension data.
  • a user approves of the default room view and floor dimensions.
  • the catalog view is a virtual wheel having displayed therein the plurality of floor design image samples or the catalog view is an itemized list.
  • the user selects the desired floor sample, the desired floor sample being processed and displayed in the augmented reality space.
  • a cost display image is provided simultaneously with the selected floor sample.
  • the virtual wheel is moved in clockwise and counterclockwise directions.
  • system further includes electronically contacting a contractor with the selected floor design and selecting a payment option
  • a non-transitory computer- readable medium comprising computer-readable instructions that, when executed by at least one processor of a computing device, cause the computing device to: on a computing device having one or more cameras, collect and process two- dimensional floor measurement data to display, in a virtual reality space, a default room view having a default floor design; select a desired replacement floor design from a plurality of floor design image samples displayed in a catalog; and process the selected desired replacement floor design so as to transition from the default room view to an augmented reality space view in which the default floor design is replaced by the selected replacement floor design.
  • Fig. i is a schematic illustration of a computer data processing system for collecting and processing room dimensions
  • Fig. 2 is a flow chart illustrating a process for selecting a replacement floor for a room
  • Fig. 2A is a diagrammatic representation of an Augmented Reality app
  • Fig. 2B to Fig. 2E illustrate a number of screenshots located on a mobile computing device interface
  • Fig. 3 is a diagrammatic representation of a mobile computing device showing a user selecting a floor design using a selection wheel
  • Fig. 4 is a diagrammatic representation of a mobile computing device showing a user selecting another floor design
  • Fig. 5 is a diagrammatic representation of a mobile computing device showing a selected floor design
  • Fig. 6 is a diagrammatic representation of a mobile computing device showing a user locating
  • Fig. 7 is a virtual reality (VR) representation of a three-dimensional room in virtual reality showing a first default floor design
  • Fig. 8 is a virtual reality (VR) representation of a three-dimensional room showing a second default floor design
  • Fig. 9 is a virtual reality (VR) representation of a floor design with occluding floor obstacles
  • Fig. 10 is a virtual reality (VR) representation of a catalog showing a plurality of floor designs for selection;
  • VR virtual reality
  • Fig. 11 is a diagrammatic default representation of a three-dimensional room showing a floor feature
  • Fig. 12 is a diagrammatic default representation of a three-dimensional room showing a floor feature
  • Fig.13 is a diagrammatic default representation of a three-dimensional room showing a floor feature
  • Fig.14 is a diagrammatic default representation of a three-dimensional room showing an exclusion zone for furniture
  • Fig.15 is a diagrammatic representation of a three-dimensional room showing the exclusion zone for furniture with optional add-on features;
  • Fig.16 is a diagrammatic representation of a three-dimensional room showing perimeter sliders
  • Fig.17 is a diagrammatic representation of a three-dimensional room showing a piece of furniture
  • Fig. 18 is a screenshot of a floor in a room with a floor design selection list
  • Fig. 19 is a screenshot of a floor design showing floor dimensions
  • Fig. 20 is a screenshot of a filter page showing a number of floor types and vendors
  • Fig. 21 is a screenshot showing a number of different floor designs showing check marks of selected floor designs
  • Fig. 22 is a screenshot of a replaced floor design showing floor dimensions
  • Fig. 23 is a screenshot of an installed floor design example showing floor dimensions and a number of floor designs
  • Fig. 24 is a screenshot of another installed floor design showing floor dimensions and a number of floor designs
  • Fig. 25 is a screenshot of another installed floor design showing floor dimensions and a number of floor designs
  • Fig. 26 is a screenshot of a floor design located around obstacles
  • Fig. 27 is a screenshot of floor designs showing varying floor pattern designs
  • Fig. 28 is a screenshot of a floor design installed around obstacles
  • Fig. 29 is a screenshot of a floor design in a finished location.
  • Fig. 30 is a screenshot of a human obstacle using LIDAR occlusion.
  • the term “consisting of’ is intended to mean including and limited to whatever follows the phrase “consisting of”. Thus, the phrase “consisting of’ indicates that the listed elements are required or mandatory and that no other elements may be present.
  • the main feature of the application (hereinafter the terms application and “app” are used interchangeably throughout) is to replace a room floor with another floor using augmented reality.
  • a catalog of different floors is available to users who wish to switch their current flooring with a replacement floor viewed and then selected from the catalog.
  • the app must also consider occlusion to not show the generated surface over furnishing adother obstacles.
  • the user can locate markers 116 at the corners of the room to generate a preview of the floor.
  • the markers 116 are used to allow for a more accurate calculation of the floor area using square footage as a measurement.
  • An additional feature is to use the metric system of measurement. Also, the floor will be generated within the marker bounds.
  • the application is a responsive application that is provided for users using a computing device 12, including handheld electronic computing devices such Tablets, Smart phones and the like.
  • a computing device 12 For users who wish to use traditional personal computers, another embodiment of the app design includes use of a feature list.
  • the computing device 12 has a visual display feature which enables landscape and portrait orientated displays of the room.
  • Several applications are available for programming. Currently, the app is programmed using a variety of commercially available packages including, for example, UNITY, which uses the most recent preview packages for AR. In one example, UNITY MARS is used to program the application. The program is further compatible with AR Core and AR Kit and is compatible with Apple’s LiDAR.
  • the app uses the latest AR technology in which depth sensing is provided.
  • FIGs. 1, 2 and 3 show an environment 100 for managing the process steps of the system described herein.
  • a main user interacts with the environment 100 to start the process of modifying a room feature in virtual reality.
  • the environment 100 includes a computer infrastructure 120 that can perform the processes described herein.
  • the computer infrastructure 120 includes a network 122 connected to client computers 124 having a display function 126 which provides a three-dimensional view of a room 128 including the location of a floor 130, ceiling 132 and walls 134.
  • a host computing system 136 is connected operably with an augmented reality module 138.
  • the client computer 124 such as a smartphone, tablet, has storage therein of a plurality of replacement floor images samples 140. When a desired replacement floor is selected, as described below, this is displayed in an augmented reality display 126.
  • a sample image wheel 104 which is located on the right of a screen viewer 106, can be rotated in clockwise (CW) CW and counterclockwise CCW directions.
  • the wheel 104 contains multiple illustrated floor materials that the user can select so as to change the current floor displayed in augmented realty.
  • one or more computing devices in the computer infrastructure 100 can communicate with one or more additional computing devices that are external to the computer infrastructure 100 using any type of communications link.
  • the communications link includes any combination of wired and/or wireless links; any combination of one or more types of networks, for example, the Internet, a wide area network, a local area network, a virtual private network and the like; and/or utilize any combination of transmission techniques and protocols.
  • Figs. 2, 3 to 6 a number of screenshots of the system are illustrated which provides access to information on the system via a smartphone.
  • the smartphone grants access to the system by way of Wi-Fi or Bluetooth connection.
  • Typical smartphones include an iPhone, Android or a tablet.
  • a display screen includes a plurality of icons that are activated by touchscreen technology.
  • GUI Graphical User Interface
  • a computing device having one or more cameras collects and processes two-dimensional floor measurement data for displaying, in a virtual reality space, a default room view having a default floor design.
  • the user starts by opening the application at block 12, which then takes the user to block 14 where the room view is displayed in three-dimensions.
  • the user can then select a floor analysis mode icon and depending on the accuracy required, add a series of markers around the room at block 18.
  • the accuracy of the room, and in particular the floor measurements are determined in an occlusion step which subtracts data acquired from the room including the occluding materials such as furnishings.
  • a block 20 loop provides the user with the option of re-setting or retaking the floor measurements.
  • the user moves on to block 24 where the room dimensions are displayed. Approval of the measurements at block 26 then gives the user the main option of displaying a floor selection tool.
  • the floor selection tool is a wheel image that includes a plurality of floor sample images at block 28.
  • the user selects a desired replacement floor design at block 30 and this is then displayed to transition as a conversion from a virtual reality default room view to an augmented reality space view of the room with a replaced floor design.
  • the user can view the calculated cost of the replacement floor design at block 34.
  • the user can approve the replacement floor design at block 36 and thereafter the user may then pay for the flooring and contact a contractor who then receives the order for the replacement floor design at block 38, whereupon the application ends at block 40.
  • the user may end the application at block 40.
  • the catalog view is a virtual wheel which displays the plurality of floor design image samples.
  • the user selects the desired floor sample, and the desired floor sample is processed and displayed in the augmented reality space.
  • the cost display image is provided simultaneously with the selected floor sample.
  • the virtual wheel is moved in CW and CCW directions using the user’s finger.
  • an electronic signal may be used to contact the contractor with the selected floor design together with a selected a payment option.
  • the platform consists of a technology engine delivered in the form of a mobile application, a server to support operations and a web portal to access certain elements on the computer.
  • Important features include the ability to visualize different material choices through advanced augmented reality, automatic surface calculation, project cost estimates, and the integration of new materials into the platform.
  • the mission is to facilitate the selection, estimation and selection processes of suppliers for the installation of new floor surfaces.
  • the app is aimed at the general public and will be available on the App Store (Apple) and the Play Store (Google).
  • the application provides a number of revenue sources including monthly subscriptions for pro members; external advertising; internal advertising; and requests for exclusivity from a partner
  • a monthly subscription allows entrepreneurs to have a business account, manage their profile, gain visibility and apply for assignments. Manufacturers and stores will have a different package offered at a different price. This account also allows a user to manage a business profile and maintain its expanded product catalogue. This type of user will have several management options and eventually, will have the opportunity to put products forward and gain visibility.
  • AdSense an external advertising network
  • Paying members will not be exposed to these advertisements. Integration will take place in different strategic screens in the app, for example in News Feed or after a number of uses of the enhanced viewing feature.
  • Technological tracking aims to clarify the spectrum of possibilities of the application and to guide development through technology choices. It is planned in the first few weeks to test the different approaches to meet the needs of the technology engine, as well as to analyze compatibility with newer devices. During our development of the application, we understand the quality of the main camera will have an impact on the assessment of the room preview. However, supported devices are not likely to be equipped with low-end devices that may interfere with the overall experience. In addition to the sensors and their nature, the processor (SoC) has another limitation on performance.
  • ARCore is a framework developed by Google that facilitates the design of augmented reality applications and whose first version was released in February 2018. The choice stopped on this platform as it maintained by Google and currently offers the widest range of supported devices. The development cycle is stable and documentation is plentiful.
  • ARCore's key technologies include motion tracking which allows the phone to understand and track its position in relation to the world: understanding the environment allows the phone to detect the size and scale of elements in the environment;_the light estimate allows the phone to estimate current lighting conditions in the environment; the prototype will be developed with Android Studio 3 and with the latest version of ARCore (1.18). The minimum version of Android supported will be Android 10 (20i9)._Currently, applications using this technology are i) IKEA Place; ii) Snapchat; and iii) Google
  • ARKit Augmented reality describes user experiences that add 2D or 3D elements to the live view from a camera in a way that gives the impression that these elements inhabit the real world. ARKit combines device motion tracking, camera scene capture, advanced scene processing and display features to simplify the task of creating an AR experience. It is possible to create many types of AR experiences with these technologies using the front or rear camera of an iOS device.
  • ARKit's main technologies include use of the LiDAR depth sensor; tracking (tracking) of the scene's object; and locating the device in its environment
  • the iOS prototype will be developed with Xcode and the latest version of ARKit (V4).
  • the minimum version of iOS supported will be iOS 14 (2020). This is not a concern as there is a high rate of adoption of updates from iOS devices.
  • Figs 14, 15, 16 and 17 we have further developed our application so that one or more perimeter sliders can be added to an image and thereafter edited.
  • a furniture exclusion zone furniture occlusion
  • the exclusion zone can be further modified.
  • the modifications can be height, movement and/or rotation of the exclusion zone.
  • An additional feature of the application is the ability to add or remove the exclusion zone, and during viewing, the user can turn the zone, rotates or move it back and forth, and tamp the floor from left to right, from the viewer’s perspective.
  • the viewer can save an image in the gallery being viewed with the photo function located at the top left of the screen. Also, the viewer (or user) can have floor lists to view and share among other app members.
  • a sample list (also known as a row list) 108 is located on the screen 106 and can be scrolled up or down with a plurality of floor design images 110 illustrated therein. A user can then select the desired floor design using, for example, a check mark 112 in the upper or lower corners of an image boundary 114.
  • the row list 108 is aesthetically pleasing and may be considered easier to use for those users who are less accustomed to smart phones. Using either the selection wheel 104 or the sample list 108, at the sample entrance, one or more sellers will take a *high definition* photograph of their products and then transfer them to us for storage on the database 134 as seen in Fig. 1.
  • the system selects samples on a large scale and incorporates the samples into the accessible database 134.
  • the system provides images that are photorealistic so that the customer is provided with a real life indication of a replacement floor design.
  • the term “photorealism” is intended to mean a technique in which a photograph, which has been previously taken, is reproduced as realistically as possible in another medium.
  • the medium is a computer graphic (image) which is stored on our database.
  • One or more users can “click” on a product advertisement on a news queue and immediately test the product coating of the seller, and view it in the location where the user is located. The user “click” redirects him/her directly into the catalog of the store (via the salesperson).
  • the users can click on a product advertisement on the news queue, and be able to immediately test the coating of the seller in question, view it on the location where the user is.
  • the click then redirects directly into the catalog of the salesperson (located in one or more stores).
  • the user has access to discounted flooring.
  • an occlusion such as human bodies or an animal, for example a pet
  • the system described herein is able to design around that occlusion so as to maintain an accurate floor measurement.
  • the application described encompass the floors, the interior and exterior layout, the walls, the ceilings, the roofs, the exterior of the buildings, the parking lots, courtyards, and landscaping, and the like.
  • the application permits the user to mark a location in a room using cursors, then to make the occlusion of a piece of furniture or an object, such as door, window, chandelier, light fixture, piece of furniture, frame, balcony, patio, stairs, shrub, landscaping, and the like
  • our app is used to design interior flooring and floor surfaces, it is also applicable to other surfaces, examples of which include, but are not limited to, walls and ceilings. Furthermore, our app is also applicable for use in designing exterior surfaces such as building exteriors and building rooves.
  • the user To use the app for designing one or more interior walls, the user first marks the wall using a cursor at a given location, such as the corners of the wall, and if desired to occlude a window, a frame or indeed any other object located therein or thereon. The user then chooses a sample from the scanned catalog. The procedure is to mark, using a cursor, at the chosen places, such as the corners of the wall and if desired, occlude a window or a frame or other object, scan choose a sample, scan and receive the area and rendering of the image to the screen of the device, either a mobile phone or a tablet.
  • the app can be used to design a ceiling using the same procedure as described above.
  • the user either mark the contours with the cursors, then mark or not if you want to occlude lights or others, choose the sample and scan to receive the area and the rendering of the image on the screen. .
  • the exterior of a building
  • the app can use the same process to re-design a building roof. The same process, you can mark with the cursors the contours of the roof, if you wish or not occlude a chimney outlet or roof window, etc., choose your samples then scan and receive the area and the rendering of the image on the screen.
  • the augmented reality engine environment is completely three-dimensional and measures along; X, Y and Z axes, internally. The app also estimates distances in the 3 axes to calculate areas.

Abstract

A method for modifying a room feature in virtual reality in which uses a computing device having a camera, to collect and process two-dimensional floor measurement data for displays, in a virtual reality space, a default room view which has a default floor design. A desired replacement floor design is selected from a number of floor design image samples that are displayed in a catalog. The replacement floor design is then processed so as to transition from the default room view in to an augmented reality space view where the default floor design is replaced by the selected replacement floor design.

Description

SYSTEM AND METHOD FOR USING VIRTUAL/AUGMENTED REALITY TO DESIGN LIVING SPACE FEATURES
TECHNICAL FIELD
The present generally concerns augmented reality applications and more particularly to use of augmented reality for designing home layouts.
BACKGROUND
Interior home design generally begins by physically measuring room dimensions. To install items such as carpets, flooring and the like, room measurements are taken, which if not carried out accurately can prove costly to correct. Selecting a desired carpet or flooring design generally involves using small samples. The designer must then use their imagination to determine what the end result will be. Clearly, this is haphazard and fraught with problems. Also, using small carpet or flooring samples to predict what the end result will be on a large scale can be the source of disappointment.
Multiple trips to and from a hardware store to obtain samples is time-consuming. Also, even with a sample provided, the homeowner does not usually have a full lifelike, real world end result. Lighting and the location of occluding walls and furnishings can result in expensive mistakes. Using a camera to take a photograph of a room is one way to quickly obtain information about the room, but currently the applicant is unaware of a system that exists which permits the user to select and overlay replacement items such as flooring, in a virtual world to obtain the impression of the end product
Virtual-reality (VR) and augmented-reality (AR) is gaining popularity for use in a growing number of activities such as gaming and other entertainment purposes. Recently though, VR and AR has become increasingly popular for use in non- recreational simulations, examples of which are surgery, engineering projects and the like. Typically, VR and AR use headsets to integrate visual information into a user's field of view to enhance their surroundings or allow them to step into immersive three- dimensional (3D) virtual environments. One application where VR and AR could be of enormous help would be in the field of interior design Thus, there is a need for a system and method for designing a room layout, particularly for flooring design, which uses accurate measurements and virtual/augmented reality interactive space.
BRIEF SUMMARY
We have designed a new and unobvious computer-based application, which uses virtual reality to represent design features in a room, and for which augmented reality is presented after a replacement design feature is selected. Advantageously, the use of virtual and augmented reality significantly reduces, or essentially, eliminates the need for manual measurements and for manual selection and delivery of the desired design feature. The application is designed to scan a floor and then select a replacement floor. To aid selection, the application uses a computing device to render a three- dimensional view of a room in a dwelling in virtual reality, and from the data collected, aid a designer in rendering an augmented three-dimensional virtual reality view of the room together with the selected new feature. Moreover, the reality of the floor selection and purchase process presents its share of challenges. Among the most important, transportation and the cost of replacing samples attract attention. With the augmented reality app, it will be possible to get a more complete, faster and lower cost overview. Facilitating this process is a major asset for promoting of our application to entrepreneurs. This need is the trigger for the project, but the full platform is more ambitious, with one of the social features to share achievements, recommend specialists and the possibility of forward certain profiles or materials
Accordingly, in one embodiment there is provided a method for modifying a room feature in virtual reality, the method comprising: on a computing device having one or more cameras, collecting and processing two-dimensional floor measurement data for displaying, in a virtual reality space, a default room view having a default floor design; selecting a desired replacement floor design from a plurality of floor design image samples displayed in a catalog; and processing the selected desired replacement floor design so as to transition from the default room view to an augmented reality space view in which the default floor design is replaced by the selected replacement floor design.
In one example, the processing of the two-dimensional floor measurement data includes obtaining occlusion room data from the default room view and subtracting the occlusion room data from room obstacles to obtain the two-dimensional floor dimensions.
In one example, the processing of the two-dimensional floor measurement data includes providing accurate dimension measurements by locating markers in the default room view in the virtual reality space.
In one example, the method includes displaying room dimension data.
In one example, a user approves of the default room view and floor dimensions.
In one example, the catalog view is a virtual wheel having displayed therein the plurality of floor design image samples or the catalog view is an itemized list. The user selects the desired floor sample, the desired floor sample being processed and displayed in the augmented reality space. A cost display image is provided simultaneously with the selected floor sample. The virtual wheel is moved in clockwise and counterclockwise directions.
In one example, the method further includes electronically contacting a contractor with the selected floor design and selecting a payment option.
Accordingly, in another embodiment, there is provided a system for modifying a room feature in virtual reality, the system comprising: a computing device having one or more cameras for collecting and processing two-dimensional floor measurement data for displaying, in a virtual reality space, a default room view having a default floor design, the two-dimensional floor measurement data being stored in memory; a plurality of floor design image samples displayed in a catalog, stored in memory, the catalog being operable to select a desired replacement floor design from; the selected desired replacement floor design being processed so as to transition from the default room view, stored memory, to an augmented reality space view in which the default floor design is replaced by the selected replacement floor design.
In one example, the processing of the two-dimensional floor measurement data includes obtaining occlusion room data from the default room view and subtracting the occlusion room data from room obstacles to obtain the two-dimensional floor dimensions.
In one example, the processing of the two-dimensional floor measurement data includes providing accurate dimension measurements by locating markers in the default room view in the virtual reality space.
In one example, the system includes displaying room dimension data.
In one example, a user approves of the default room view and floor dimensions.
In one example, the catalog view is a virtual wheel having displayed therein the plurality of floor design image samples or the catalog view is an itemized list. The user selects the desired floor sample, the desired floor sample being processed and displayed in the augmented reality space. A cost display image is provided simultaneously with the selected floor sample. The virtual wheel is moved in clockwise and counterclockwise directions.
In one example, the system further includes electronically contacting a contractor with the selected floor design and selecting a payment option
Accordingly, in another embodiment, there is provided a non-transitory computer- readable medium comprising computer-readable instructions that, when executed by at least one processor of a computing device, cause the computing device to: on a computing device having one or more cameras, collect and process two- dimensional floor measurement data to display, in a virtual reality space, a default room view having a default floor design; select a desired replacement floor design from a plurality of floor design image samples displayed in a catalog; and process the selected desired replacement floor design so as to transition from the default room view to an augmented reality space view in which the default floor design is replaced by the selected replacement floor design.
BRIEF DESCRIPTION OF THE DRAWINGS
These and other features of that described herein will become more apparent from the following description in which reference is made to the appended drawings wherein:
Fig. i is a schematic illustration of a computer data processing system for collecting and processing room dimensions;
Fig. 2 is a flow chart illustrating a process for selecting a replacement floor for a room;
Fig. 2A is a diagrammatic representation of an Augmented Reality app;
Fig. 2B to Fig. 2E illustrate a number of screenshots located on a mobile computing device interface;
Fig. 3 is a diagrammatic representation of a mobile computing device showing a user selecting a floor design using a selection wheel;
Fig. 4 is a diagrammatic representation of a mobile computing device showing a user selecting another floor design;
Fig. 5 is a diagrammatic representation of a mobile computing device showing a selected floor design;
Fig. 6 is a diagrammatic representation of a mobile computing device showing a user locating;
Fig. 7 is a virtual reality (VR) representation of a three-dimensional room in virtual reality showing a first default floor design; Fig. 8 is a virtual reality (VR) representation of a three-dimensional room showing a second default floor design;
Fig. 9 is a virtual reality (VR) representation of a floor design with occluding floor obstacles;
Fig. 10 is a virtual reality (VR) representation of a catalog showing a plurality of floor designs for selection;
Fig. 11 is a diagrammatic default representation of a three-dimensional room showing a floor feature
Fig. 12 is a diagrammatic default representation of a three-dimensional room showing a floor feature;
Fig.13 is a diagrammatic default representation of a three-dimensional room showing a floor feature;
Fig.14 is a diagrammatic default representation of a three-dimensional room showing an exclusion zone for furniture;
Fig.15 is a diagrammatic representation of a three-dimensional room showing the exclusion zone for furniture with optional add-on features;
Fig.16 is a diagrammatic representation of a three-dimensional room showing perimeter sliders;
Fig.17 is a diagrammatic representation of a three-dimensional room showing a piece of furniture;
Fig. 18 is a screenshot of a floor in a room with a floor design selection list;
Fig. 19 is a screenshot of a floor design showing floor dimensions;
Fig. 20 is a screenshot of a filter page showing a number of floor types and vendors;
Fig. 21 is a screenshot showing a number of different floor designs showing check marks of selected floor designs; Fig. 22 is a screenshot of a replaced floor design showing floor dimensions;
Fig. 23 is a screenshot of an installed floor design example showing floor dimensions and a number of floor designs;
Fig. 24 is a screenshot of another installed floor design showing floor dimensions and a number of floor designs;
Fig. 25 is a screenshot of another installed floor design showing floor dimensions and a number of floor designs;
Fig. 26 is a screenshot of a floor design located around obstacles;
Fig. 27 is a screenshot of floor designs showing varying floor pattern designs;
Fig. 28 is a screenshot of a floor design installed around obstacles;
Fig. 29 is a screenshot of a floor design in a finished location; and’
Fig. 30 is a screenshot of a human obstacle using LIDAR occlusion.
DETAILED DESCRIPTION
Definitions
Unless otherwise specified, the following definitions apply:
The singular forms “a”, “an” and “the” include corresponding plural references unless the context clearly dictates otherwise.
As used herein, the term “comprising” is intended to mean that the list of elements following the word “comprising” are required or mandatory but that other elements are optional and may or may not be present.
As used herein, the term “consisting of’ is intended to mean including and limited to whatever follows the phrase “consisting of”. Thus, the phrase “consisting of’ indicates that the listed elements are required or mandatory and that no other elements may be present. Broadly speaking, the main feature of the application (hereinafter the terms application and “app” are used interchangeably throughout) is to replace a room floor with another floor using augmented reality. A catalog of different floors is available to users who wish to switch their current flooring with a replacement floor viewed and then selected from the catalog. However, since most rooms are not perfectly square or rectangular, and often they have recesses and furniture located therein, the app must also consider occlusion to not show the generated surface over furnishing adother obstacles.
Advantageously, the user can locate markers 116 at the corners of the room to generate a preview of the floor. The markers 116 are used to allow for a more accurate calculation of the floor area using square footage as a measurement. An additional feature is to use the metric system of measurement. Also, the floor will be generated within the marker bounds.
As best illustrated in Fig. 1, the application is a responsive application that is provided for users using a computing device 12, including handheld electronic computing devices such Tablets, Smart phones and the like. For users who wish to use traditional personal computers, another embodiment of the app design includes use of a feature list. The computing device 12 has a visual display feature which enables landscape and portrait orientated displays of the room. Several applications are available for programming. Currently, the app is programmed using a variety of commercially available packages including, for example, UNITY, which uses the most recent preview packages for AR. In one example, UNITY MARS is used to program the application. The program is further compatible with AR Core and AR Kit and is compatible with Apple’s LiDAR.
For current and future uses, the app uses the latest AR technology in which depth sensing is provided.
Referring first to Figs. 1, 2 and 3, which show an environment 100 for managing the process steps of the system described herein. A main user interacts with the environment 100 to start the process of modifying a room feature in virtual reality. The environment 100 includes a computer infrastructure 120 that can perform the processes described herein. In particular, the computer infrastructure 120 includes a network 122 connected to client computers 124 having a display function 126 which provides a three-dimensional view of a room 128 including the location of a floor 130, ceiling 132 and walls 134. A host computing system 136 is connected operably with an augmented reality module 138. The client computer 124 such as a smartphone, tablet, has storage therein of a plurality of replacement floor images samples 140. When a desired replacement floor is selected, as described below, this is displayed in an augmented reality display 126.
As best seen in Fig. 3, in an embodiment of our app, to select a floor design 102, a sample image wheel 104, which is located on the right of a screen viewer 106, can be rotated in clockwise (CW) CW and counterclockwise CCW directions. The wheel 104 contains multiple illustrated floor materials that the user can select so as to change the current floor displayed in augmented realty.A person of ordinary skill in the art will readily understand that although a single computer infrastructure 100 is illustrated, two or more computing devices such as a server cluster, that communicate over any type of communications link, such as a network, a shared memory, or the like, can perform the method steps described herein. Furthermore, while performing the process or method steps described herein, one or more computing devices in the computer infrastructure 100 can communicate with one or more additional computing devices that are external to the computer infrastructure 100 using any type of communications link. The communications link includes any combination of wired and/or wireless links; any combination of one or more types of networks, for example, the Internet, a wide area network, a local area network, a virtual private network and the like; and/or utilize any combination of transmission techniques and protocols.
Referring now to Figs. 2, 3 to 6, a number of screenshots of the system are illustrated which provides access to information on the system via a smartphone. Typically, the smartphone grants access to the system by way of Wi-Fi or Bluetooth connection. Typical smartphones include an iPhone, Android or a tablet. A display screen includes a plurality of icons that are activated by touchscreen technology. At the beginning of the process, the user encounters an application which they open by interacting with a Graphical User Interface (GUI) on the display screen on the smartphone. Broadly speaking, a method and system is described for modifying a room feature in virtual reality. In the example illustrated herein, on a computing device having one or more cameras, collects and processes two-dimensional floor measurement data for displaying, in a virtual reality space, a default room view having a default floor design. At block 10, the user starts by opening the application at block 12, which then takes the user to block 14 where the room view is displayed in three-dimensions. At block 16, the user can then select a floor analysis mode icon and depending on the accuracy required, add a series of markers around the room at block 18. The accuracy of the room, and in particular the floor measurements, are determined in an occlusion step which subtracts data acquired from the room including the occluding materials such as furnishings. A block 20 loop provides the user with the option of re-setting or retaking the floor measurements. Once satisfied with the accuracy of the measurements, at block 22 after the final calculations are made, the user moves on to block 24 where the room dimensions are displayed. Approval of the measurements at block 26 then gives the user the main option of displaying a floor selection tool. The floor selection tool is a wheel image that includes a plurality of floor sample images at block 28. The user then selects a desired replacement floor design at block 30 and this is then displayed to transition as a conversion from a virtual reality default room view to an augmented reality space view of the room with a replaced floor design. Along with the display of the floor dimensions, typically in square footage, the user can view the calculated cost of the replacement floor design at block 34. Optionally, the user can approve the replacement floor design at block 36 and thereafter the user may then pay for the flooring and contact a contractor who then receives the order for the replacement floor design at block 38, whereupon the application ends at block 40. Alternatively, once the user calculates the cost at block 34, they may end the application at block 40.
As best seen in Fig. 3, 4, and 7, for example, the catalog view is a virtual wheel which displays the plurality of floor design image samples. The user selects the desired floor sample, and the desired floor sample is processed and displayed in the augmented reality space. The cost display image is provided simultaneously with the selected floor sample. As best seen in Fig. 3 and 4, the virtual wheel is moved in CW and CCW directions using the user’s finger. Depending on the options available to the user, an electronic signal may be used to contact the contractor with the selected floor design together with a selected a payment option.
In sum, our app provides an ecosystem targeting key players in the flooring industry. The platform consists of a technology engine delivered in the form of a mobile application, a server to support operations and a web portal to access certain elements on the computer. Important features include the ability to visualize different material choices through advanced augmented reality, automatic surface calculation, project cost estimates, and the integration of new materials into the platform. The mission is to facilitate the selection, estimation and selection processes of suppliers for the installation of new floor surfaces. The app is aimed at the general public and will be available on the App Store (Apple) and the Play Store (Google).
We have designed the application to run on two source codes, which will both offer the same features. The application is easy to use, accessible to all and very fast. Different types of accounts will be offered for potential customers, coating specialists and manufacturers and distributors of surface materials.
Conventional floor selection and purchase process presents its share of challenges. Among the most important, transportation and the cost of replacing samples are problematic. With the augmented reality app, it will be possible to get a more complete, faster and lower cost overview. Facilitating this process is a major asset for promotion to entrepreneurs.
Furthermore, the application provides a number of revenue sources including monthly subscriptions for pro members; external advertising; internal advertising; and requests for exclusivity from a partner
With the monthly subscription option, different packages can be imagined to align with the needs of entrepreneurs and offer a competitive and advantageous offer. A monthly subscription allows entrepreneurs to have a business account, manage their profile, gain visibility and apply for assignments. Manufacturers and stores will have a different package offered at a different price. This account also allows a user to manage a business profile and maintain its expanded product catalogue. This type of user will have several management options and eventually, will have the opportunity to put products forward and gain visibility.
In order to monetize users without a professional account, it is possible to integrate an external advertising network such as AdSense, for the display of targeted advertisements in the domains of construction and renovation. Paying members will not be exposed to these advertisements. Integration will take place in different strategic screens in the app, for example in News Feed or after a number of uses of the enhanced viewing feature.
Once the network effect is established, it can be exploited by offering the commercial account to promote their services and products. The principle will be based on the concept of ads featured to exchange temporary and targeted visibility in a region, for a pre-determined rate. This option is only available to professional members on the applicant’s business network.
To the extent that a chain or distributor requests exclusivity in an entire sector or platform, an agreement may be reached
At the heart of the buyer's experience is a simple way to visualize a wide range of materials in augmented reality, using the camera of a compatible smart camera. The user can browse the different types of materials (such as concrete coatings, floating floors, hardwood, etc.) and choose products and finally see the desired effect, directly at home. Subsequently, the application will publish the mandate and thus receive proposals from qualified local installers depending on the work to be completed.
It should be noted, however, that although we demonstrate the use of this application for replacement flooring, a person skilled in the art will readily recognize that any number of other interior design features can be accessed using our application.
Technological tracking aims to clarify the spectrum of possibilities of the application and to guide development through technology choices. It is planned in the first few weeks to test the different approaches to meet the needs of the technology engine, as well as to analyze compatibility with newer devices. During our development of the application, we understand the quality of the main camera will have an impact on the assessment of the room preview. However, supported devices are not likely to be equipped with low-end devices that may interfere with the overall experience. In addition to the sensors and their nature, the processor (SoC) has another limitation on performance.
Platform: native vs hybrid
Some technologies such as Cordova, Ionic, etc. allow the creation of hybrid applications compatible with iOS and Android. However, this choice comes with a few negatives, including greater complexity when using native modules or the device's physical sensors. As the use of motion sensors, cameras and graphics devices are at the heart of the project, it is considered preferable not to use these techniques and to develop two independent applications for both systems. The notable benefits are better performance and the ability to quickly integrate the new developments offered by Apple and Android. In the context of a technical prototype, these arguments become even more important.
Platform: ARCore (Android)
ARCore is a framework developed by Google that facilitates the design of augmented reality applications and whose first version was released in February 2018. The choice stopped on this platform as it maintained by Google and currently offers the widest range of supported devices. The development cycle is stable and documentation is plentiful.
ARCore's key technologies include motion tracking which allows the phone to understand and track its position in relation to the world: understanding the environment allows the phone to detect the size and scale of elements in the environment;_the light estimate allows the phone to estimate current lighting conditions in the environment; the prototype will be developed with Android Studio 3 and with the latest version of ARCore (1.18). The minimum version of Android supported will be Android 10 (20i9)._Currently, applications using this technology are i) IKEA Place; ii) Snapchat; and iii) Google
Platform: ARKit (IOS) Augmented reality describes user experiences that add 2D or 3D elements to the live view from a camera in a way that gives the impression that these elements inhabit the real world. ARKit combines device motion tracking, camera scene capture, advanced scene processing and display features to simplify the task of creating an AR experience. It is possible to create many types of AR experiences with these technologies using the front or rear camera of an iOS device.
ARKit's main technologies: include use of the LiDAR depth sensor; tracking (tracking) of the scene's object; and locating the device in its environment
Shared AR sessions.
The iOS prototype will be developed with Xcode and the latest version of ARKit (V4). The minimum version of iOS supported will be iOS 14 (2020). This is not a concern as there is a high rate of adoption of updates from iOS devices.
Applications which currently use this technology include i) Pokemon Go; ii) Inkhunter; and iii) Snapchat.
Referring now to Figs 14, 15, 16 and 17, we have further developed our application so that one or more perimeter sliders can be added to an image and thereafter edited. Once we have defined the perimeter of a room, we can use the perimeter sliders to delineate a furniture exclusion zone (furniture occlusion). Once the exclusion zone is defined, it can be further modified. The modifications can be height, movement and/or rotation of the exclusion zone. An additional feature of the application is the ability to add or remove the exclusion zone, and during viewing, the user can turn the zone, rotates or move it back and forth, and tamp the floor from left to right, from the viewer’s perspective.
Once finished, the viewer (or user) can save an image in the gallery being viewed with the photo function located at the top left of the screen. Also, the viewer (or user) can have floor lists to view and share among other app members.
As best seen in Figs. 18 through 29, in alternative embodiment of our app, a sample list (also known as a row list) 108 is located on the screen 106 and can be scrolled up or down with a plurality of floor design images 110 illustrated therein. A user can then select the desired floor design using, for example, a check mark 112 in the upper or lower corners of an image boundary 114. The row list 108 is aesthetically pleasing and may be considered easier to use for those users who are less accustomed to smart phones. Using either the selection wheel 104 or the sample list 108, at the sample entrance, one or more sellers will take a *high definition* photograph of their products and then transfer them to us for storage on the database 134 as seen in Fig. 1.
The system then selects samples on a large scale and incorporates the samples into the accessible database 134. Advantageously, the system provides images that are photorealistic so that the customer is provided with a real life indication of a replacement floor design.
Generally speaking, the term “photorealism” is intended to mean a technique in which a photograph, which has been previously taken, is reproduced as realistically as possible in another medium. In our case, the medium is a computer graphic (image) which is stored on our database.
One or more users can “click” on a product advertisement on a news queue and immediately test the product coating of the seller, and view it in the location where the user is located. The user “click” redirects him/her directly into the catalog of the store (via the salesperson).
Advantageously, the users can click on a product advertisement on the news queue, and be able to immediately test the coating of the seller in question, view it on the location where the user is. The click then redirects directly into the catalog of the salesperson (located in one or more stores). Furthermore, in cases where an end-of- season sale is available, the user has access to discounted flooring.
As best seen in Fig. 30, if during measurement of a room floor, an occlusion such as human bodies or an animal, for example a pet, moves across the floor during capture by the camera, or even if the user taking the photograph accidently includes a portion of his hand or foot in the photographs, the system described herein is able to design around that occlusion so as to maintain an accurate floor measurement. In summary, generally speaking, the application described encompass the floors, the interior and exterior layout, the walls, the ceilings, the roofs, the exterior of the buildings, the parking lots, courtyards, and landscaping, and the like. Furthermore, the application permits the user to mark a location in a room using cursors, then to make the occlusion of a piece of furniture or an object, such as door, window, chandelier, light fixture, piece of furniture, frame, balcony, patio, stairs, shrub, landscaping, and the like
Although our app is used to design interior flooring and floor surfaces, it is also applicable to other surfaces, examples of which include, but are not limited to, walls and ceilings. Furthermore, our app is also applicable for use in designing exterior surfaces such as building exteriors and building rooves.
1. Walls
To use the app for designing one or more interior walls, the user first marks the wall using a cursor at a given location, such as the corners of the wall, and if desired to occlude a window, a frame or indeed any other object located therein or thereon. The user then chooses a sample from the scanned catalog. The procedure is to mark, using a cursor, at the chosen places, such as the corners of the wall and if desired, occlude a window or a frame or other object, scan choose a sample, scan and receive the area and rendering of the image to the screen of the device, either a mobile phone or a tablet.
2. Ceilings
As with the walls, the app can be used to design a ceiling using the same procedure as described above. The user either mark the contours with the cursors, then mark or not if you want to occlude lights or others, choose the sample and scan to receive the area and the rendering of the image on the screen. . The exterior of a building
Using the same process, the application works both indoors and outdoors.
4. Roofs Finally, if desired, the app can use the same process to re-design a building roof. The same process, you can mark with the cursors the contours of the roof, if you wish or not occlude a chimney outlet or roof window, etc., choose your samples then scan and receive the area and the rendering of the image on the screen. The augmented reality engine environment is completely three-dimensional and measures along; X, Y and Z axes, internally. The app also estimates distances in the 3 axes to calculate areas.
Other Embodiments
From the foregoing description, it will be apparent to one of ordinary skill in the art that variations and modifications may be made to the embodiments described herein to adapt it to various usages and conditions.

Claims

CLAIMS What Is Claimed Is:
1. A method for modifying a room feature in virtual reality, the method comprising: on a computing device having one or more cameras, collecting and processing two-dimensional floor measurement data for displaying, in a virtual reality space, a default room view having a default floor design; selecting a desired replacement floor design from a plurality of floor design image samples displayed in a catalog; and processing the selected desired replacement floor design so as to transition from the default room view to an augmented reality space view in which the default floor design is replaced by the selected replacement floor design.
2. The method, according to claim 1, in which the processing of the two- dimensional floor measurement data includes obtaining occlusion room data from the default room view and subtracting the occlusion room data from room obstacles to obtain the two-dimensional floor dimensions.
3. The method, according to claim 1, in which the processing of the two- dimensional floor measurement data includes providing accurate dimension measurements by locating markers in the default room view in the virtual reality space.
4. The method, according to claim 1, includes displaying room dimension data.
5. The method, according to claim 1, in which a user approves of the default room view and floor dimensions.
6. The method, according to claim 1, in which the catalog view is a virtual wheel having displayed therein the plurality of floor design image samples or the catalog view is an itemized list.
7. The method, according to one of claims 6 or 7 above,, in which the user selects the desired floor sample, the desired floor sample being processed and displayed in the augmented reality space.
8. The method, according to claim 7, in which a cost display image is provided simultaneously with the selected floor sample.
9. The method, according to claim 6, in which the virtual wheel is moved in clockwise and counterclockwise directions.
10. The method, according to claim 1, further includes electronically contacting a contractor with the selected floor design and selecting a payment option.
11. A system for modifying a room feature in virtual reality, the system comprising: a computing device having one or more cameras for collecting and processing two-dimensional floor measurement data for displaying, in a virtual reality space, a default room view having a default floor design, the two-dimensional floor measurement data being stored in memory; a plurality of floor design image samples displayed in a catalog, stored in memory, the catalog being operable to select a desired replacement floor design from; the selected desired replacement floor design being processed so as to transition from the default room view, stored memory, to an augmented reality space view in which the default floor design is replaced by the selected replacement floor design.
12. The system, according to claim 11, in which the processing of the two- dimensional floor measurement data includes obtaining occlusion room data from the default room view and subtracting the occlusion room data from room obstacles to obtain the two-dimensional floor dimensions.
13. The system, according to claim 11, in which the processing of the two- dimensional floor measurement data includes providing accurate dimension measurements by locating markers in the default room view in the virtual reality space.
14- The system, according to claim n, includes displaying room dimension data.
15. The system, according to claim 11, in which a user approves of the default room view and floor dimensions.
16. The system, according to claim 11, in which the catalog view is a virtual wheel having displayed therein the plurality of floor design image samples.
17. The system, according to claim 6, in which the user selects the desired floor sample, the desired floor sample being processed and displayed in the augmented reality space.
18. The system, according to claim 7, in which a cost display image is provided simultaneously with the selected floor sample.
19. The system, according to claim 6, in which the virtual wheel is moved in CW and CCW directions.
20. The system, according to claim 11, further includes electronically contacting a contractor with the selected floor design and selecting a payment option.
21. A non-transitory computer-readable medium comprising computer-readable instructions that, when executed by at least one processor of a computing device, cause the computing device to: on a computing device having one or more cameras, collect and process two- dimensional floor measurement data to display, in a virtual reality space, a default room view having a default floor design; select a desired replacement floor design from a plurality of floor design image samples displayed in a catalog; and process the selected desired replacement floor design so as to transition from the default room view to an augmented reality space view in which the default floor design is replaced by the selected replacement floor design.
PCT/CA2023/050629 2022-05-09 2023-05-09 System and method for using virtual/augmented reality to design living space features WO2023215976A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202263339539P 2022-05-09 2022-05-09
US63/339,539 2022-05-09

Publications (1)

Publication Number Publication Date
WO2023215976A1 true WO2023215976A1 (en) 2023-11-16

Family

ID=88729279

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CA2023/050629 WO2023215976A1 (en) 2022-05-09 2023-05-09 System and method for using virtual/augmented reality to design living space features

Country Status (1)

Country Link
WO (1) WO2023215976A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190197599A1 (en) * 2017-12-22 2019-06-27 Houzz, Inc. Techniques for recommending and presenting products in an augmented reality scene
US20200302681A1 (en) * 2019-03-18 2020-09-24 Geomagical Labs, Inc. Virtual interaction with three-dimensional indoor room imagery
US20210334538A1 (en) * 2020-04-27 2021-10-28 State Farm Mutual Automobile Insurance Company Systems and methods for a 3d model for viewing potential placement of an object

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190197599A1 (en) * 2017-12-22 2019-06-27 Houzz, Inc. Techniques for recommending and presenting products in an augmented reality scene
US20200302681A1 (en) * 2019-03-18 2020-09-24 Geomagical Labs, Inc. Virtual interaction with three-dimensional indoor room imagery
US20210334538A1 (en) * 2020-04-27 2021-10-28 State Farm Mutual Automobile Insurance Company Systems and methods for a 3d model for viewing potential placement of an object

Similar Documents

Publication Publication Date Title
US11238652B2 (en) Presenting integrated building information using building models
US20220319106A1 (en) Virtual interaction with three-dimensional indoor room imagery
TWI628614B (en) Method for browsing house interactively in 3d virtual reality and system for the same
US10809066B2 (en) Automated mapping information generation from inter-connected images
US11676344B2 (en) Presenting building information using building models
US10970934B2 (en) Integrated operating environment
CA3058602C (en) Automated mapping information generation from inter-connected images
US20230161463A1 (en) Automated Tools For Generating Building Mapping Information
US20180374276A1 (en) System for multimedia spatial annotation, visualization, and recommendation
US20220164493A1 (en) Automated Tools For Generating Mapping Information For Buildings
US11481925B1 (en) Automated determination of image acquisition locations in building interiors using determined room shapes
US8108267B2 (en) Method of facilitating a sale of a product and/or a service
US11836973B2 (en) Automated direction of capturing in-room information for use in usability assessment of buildings
US20230419662A1 (en) Automated Usability Assessment Of Buildings Using Visual Data Of Captured In-Room Images
US10740870B2 (en) Creating a floor plan from images in spherical format
US10735708B2 (en) Transforming locations in a spherical image viewer
WO2001033446A1 (en) Housing space-related commodity sale assisting system, housing space-related commodity sale assisting method, program for assisting housing space-related commodity sale, and computer-readable recorded medium on which program for assisting housing space-related commodity sale is recorded
CA3121009A1 (en) Integrated operating environment
Racz et al. VR aided architecture and interior design
TWM514072U (en) Three-dimensional virtual reality interactive house browsing system
WO2015127534A1 (en) Computer-implemented method and system of real estate marketing
WO2023215976A1 (en) System and method for using virtual/augmented reality to design living space features
JP2001236418A (en) Business method using internet living space vr system, and storage medium for the same
US10529055B1 (en) Compensating for camera pose in a spherical image
US11847736B2 (en) Systems and methods for modifying lighting in three-dimensional models

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23802387

Country of ref document: EP

Kind code of ref document: A1

DPE2 Request for preliminary examination filed before expiration of 19th month from priority date (pct application filed from 20040101)